On 8/23/2018 4:59 PM, Derrick Stolee wrote:
When I choose my own metrics for performance tests, I like to run at
least 10 runs, remove the largest AND smallest runs from the samples,
and then take the average. I did this manually for 'git rev-list --all
--objects' on git/git and got the following results:
v2.18.0 v2.19.0-rc0 HEAD
--------------------------------
3.126 s 3.308 s 3.170 s
For full disclosure, here is a full table including all samples:
| | v2.18.0 | v2.19.0-rc0 | HEAD |
|------|---------|-------------|---------|
| | 4.58 | 3.302 | 3.239 |
| | 3.13 | 3.337 | 3.133 |
| | 3.213 | 3.291 | 3.159 |
| | 3.219 | 3.318 | 3.131 |
| | 3.077 | 3.302 | 3.163 |
| | 3.074 | 3.328 | 3.119 |
| | 3.022 | 3.277 | 3.125 |
| | 3.083 | 3.259 | 3.203 |
| | 3.057 | 3.311 | 3.223 |
| | 3.155 | 3.413 | 3.225 |
| Max | 4.58 | 3.413 | 3.239 |
| Min | 3.022 | 3.259 | 3.119 |
| Avg* | 3.126 | 3.30825 | 3.17025 |
(Note that the largest one was the first run, on v2.18.0, which is due
to a cold disk.)
I just kicked off a script that will run this test on the Linux repo
while I drive home. I'll be able to report a similar table of data
easily.
Here are the numbers for Linux:
| | v2.18.0 | v2.19.0-rc0 | HEAD |
|------|----------|-------------|--------|
| | 86.5 | 70.739 | 57.266 |
| | 60.582 | 101.928 | 56.641 |
| | 58.964 | 60.139 | 60.258 |
| | 59.47 | 61.141 | 58.213 |
| | 62.554 | 60.73 | 84.54 |
| | 59.139 | 85.424 | 57.745 |
| | 58.487 | 59.31 | 59.979 |
| | 58.653 | 69.845 | 60.181 |
| | 58.085 | 102.777 | 61.455 |
| | 58.304 | 60.459 | 62.551 |
| Max | 86.5 | 102.777 | 84.54 |
| Min | 58.085 | 59.31 | 56.641 |
| Avg* | 59.51913 | 71.30063 | 59.706 |
| Med | 59.0515 | 65.493 | 60.08 |
Thanks,
-Stolee
