[ ... ]
And another question: how would you do it differently? Sure, in an ideal world I could assemble my own botnet and then blast my corporate network with a gigabit of distributed traffic multiple times for each webserver -- but obviously in the real world that's not going to happen.
IMO it's not necessary to simulate different connections from all over the world, but it is true that the benchmark client will consume a substantial amount of resources, which will affect your result. Both should be skewed about the same, so it should be a fair comparison, but things like "requests/sec" will not be meaningful.
Using the live Internet may cause your network connection to be the bottleneck, too, which could give bad results.
I think using a LAN/VLAN connection would be a fair measure. You could also consider running the tests as instances on Amazon EC2, which will let you lease a small pool of servers for a few hours for $20 or so.
---Scott.
