It will likely never be as fast as replication, but it can be much
faster than it is now. I put benchmarks in the original PR I
resurrected for partial stripe reads:
https://github.com/ceph/ceph/pull/52746
I'm trying to find some time this week to run some tests on Radek's PR
so we can do some comparisons.
Mark
On 7/2/24 02:28, Sebastian Wagner wrote:
Hi Bill,
I noticed you mentioned the PR a few moments after I clicked on send
🙈. I really hope we end up with EC being on par with replicated pools
without any meaningful performance impact.
Do you think we can put those design documents into the Ceph Github
organization? IIRC, this is the third proposal on this mailing list
that I saw just in a few months.
Best,
Sebastian
Am 01.07.24 um 12:01 schrieb Bill Scales:
Hi,
Yes, most of the optimizations for partial reads are part of that
pull request – it is referenced in the design document.
There is one further enhancement we would like to make which is to
reduce the size of the reads to each shard – the pull request
currently reads the whole chunk for any shard that needs to be read.
This is not an issue with the current default chunk size which is 4K,
but if you use a larger chunk size such as 256K then this means that
short client reads will read excess data and then discard it.
Currently there are many reasons why using a small chunk size is a
good idea, but once both partial reads and partial writes have been
implemented performance can be improved further by using a larger
chunk size because splitting I/Os into multiple chunks is expensive.
There’s more discussion about the chunk size and how to accommodate
small sized objects without large storage overheads later in the
document.
Cheers,
Bill.
bill_scales@xxxxxxxxxx <mailto:bill_scales@xxxxxxxxxx>
IBM Distinguished Engineer, IBM Storage
*From: *Sebastian Wagner <sebastian.wagner@xxxxxxxx>
*Date: *Monday, 1 July 2024 at 09:31
*To: *Bill Scales <bill_scales@xxxxxxxxxx>, dev@xxxxxxx <dev@xxxxxxx>
*Subject: *[EXTERNAL] Re: Erasure coding enhancements - design for review
Hi Bill, Is the Partial Reads section the same
as https: //github. com/ceph/ceph/pull/55196 ? Best, Sebastian Am
01. 07. 24 um 10: 08 schrieb Bill Scales: Hi, We are planning to
enhance the performance of erasure coding, in particular for use with
Hi Bill,
Is the Partial Reads section the same as
https://github.com/ceph/ceph/pull/55196 ?
Best,
Sebastian
Am 01.07.24 um 10:08 schrieb Bill Scales:
Hi,
We are planning to enhance the performance of erasure coding, in
particular for use with block and file. We've got a design
document https://github.com/bill-scales/CephErasureCodingDesign
that sets out what we are hoping to achieve. We welcome your
feedback, either posting your comments in Slack on #ceph-devel ,
raising issues in github or getting in contact with myself
Cheers,
Bill.
bill_scales@xxxxxxxxxx <mailto:bill_scales@xxxxxxxxxx>
IBM Distinguished Engineer, IBM Storage
Unless otherwise stated above:
IBM United Kingdom Limited
Registered in England and Wales with number 741598
Registered office: PO Box 41, North Harbour, Portsmouth, Hants.
PO6 3AU
_______________________________________________
Dev mailing list --dev@xxxxxxx
To unsubscribe send an email todev-leave@xxxxxxx
--
Head of Software Development
E-Mail: sebastian.wagner@xxxxxxxx
croit GmbH, Freseniusstr. 31h, 81247 Munich
CEO: Martin Verges, Andy Muthmann - VAT-ID: DE310638492
Com. register: Amtsgericht Munich HRB 231263
Web <https://croit.io/> | LinkedIn
<http://linkedin.com/company/croit> | Youtube
<https://www.youtube.com/channel/UCIJJSKVdcSLGLBtwSFx_epw> | Twitter
<https://twitter.com/croit_io>
TOP 100 Innovator Award Winner
<https://croit.io/blog/croit-receives-top-100-seal> by compamedia
Technology Fast50 Award
<https://croit.io/blog/deloitte-technology-fast-50-award> Winner by
Deloitte
Unless otherwise stated above:
IBM United Kingdom Limited
Registered in England and Wales with number 741598
Registered office: PO Box 41, North Harbour, Portsmouth, Hants. PO6 3AU
--
Head of Software Development
E-Mail: sebastian.wagner@xxxxxxxx
croit GmbH, Freseniusstr. 31h, 81247 Munich
CEO: Martin Verges, Andy Muthmann - VAT-ID: DE310638492
Com. register: Amtsgericht Munich HRB 231263
Web <https://croit.io/> | LinkedIn <http://linkedin.com/company/croit>
| Youtube <https://www.youtube.com/channel/UCIJJSKVdcSLGLBtwSFx_epw> |
Twitter <https://twitter.com/croit_io>
TOP 100 Innovator Award Winner
<https://croit.io/blog/croit-receives-top-100-seal> by compamedia
Technology Fast50 Award
<https://croit.io/blog/deloitte-technology-fast-50-award> Winner by
Deloitte
_______________________________________________
Dev mailing list -- dev@xxxxxxx
To unsubscribe send an email to dev-leave@xxxxxxx
--
Best Regards,
Mark Nelson
Head of Research and Development
Clyso GmbH
p: +49 89 21552391 12 | a: Minnesota, USA
w: https://clyso.com | e: mark.nelson@xxxxxxxxx
We are hiring: https://www.clyso.com/jobs/
_______________________________________________
Dev mailing list -- dev@xxxxxxx
To unsubscribe send an email to dev-leave@xxxxxxx
