Re: [RFC] New S3 Benchmark

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Ha, I hadn't thought to check if that existed which was probably pretty short-sighted on my part. :)


I suppose the good news is that it might be a good candidate for comparison, and once I've implemented an S3 client endpoint in CBT I should be able to target both pretty easily.


Mark


On 8/15/19 6:41 PM, David Byte wrote:
Mark, did the S3 engine for fio not work?

Sent from my iPhone. Typos are Apple's fault.

On Aug 15, 2019, at 6:37 PM, Mark Nelson <mark.a.nelson@xxxxxxxxx> wrote:

Hi Guys,

Earlier this week I was working on investigating the impact of OMAP performance on RGW and wanted to see if putting rocksdb on ramdisk would help speed up bucket index updates.  While running tests I found out that the benchmark tool I was using consumed roughly 15 cores of CPU to push 4K puts/second to RGW from 128 threads.  That wasn't really viable, so I started looking for alternate S3 benchmarking tools.  COSBench is sort of the most well known choice out there, but it's a bit cumbersome if you just want to run some quick tests from the command-line.

I managed to find a simple yet very nice benchmark written in go developed by Wasabi Inc called s3-benchmark.  While it works well, it really only targets single buckets and is designed more for AWS testing than for Ceph.  I forked the project and pretty much refactored the whole thing to be more useful for the kind of testing I want to do. It's now at the point where I think it might be ready to experiment with.  S3 benchmarking has been a semi-recurring topic on the list so I figured other folks might be interested in trying it too and hopefully providing feedback.

The new benchmark is called hsbench and it's available here:

https://github.com/markhpc/hsbench

See the README.md for some of the advantages over the original s3-benchmark program it was forked from.  I consider this release to be alpha level quality and disclaim all responsibility if it breaks and deletes every object in all of your buckets.  Consider yourself warned. :)

Mark
_______________________________________________
Dev mailing list -- dev@xxxxxxx
To unsubscribe send an email to dev-leave@xxxxxxx

_______________________________________________
Dev mailing list -- dev@xxxxxxx
To unsubscribe send an email to dev-leave@xxxxxxx
_______________________________________________
Dev mailing list -- dev@xxxxxxx
To unsubscribe send an email to dev-leave@xxxxxxx



[Index of Archives]     [CEPH Users]     [Ceph Devel]     [Ceph Large]     [Information on CEPH]     [Linux BTRFS]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]

  Powered by Linux