On 2020-07-29 01:33, Виталий Филиппов wrote:
Wow, crimson osd has some big problems if its latency is 10ms. OK, let's just wait until it's fixed.
I really keen to see good crimson results, since I need a good reference numbers to understand where pech sucks, but each test I do I fail to get something significant.
Problem is that you don't know which data to resync without journaling.
Take simple RAID1 as an example. Nothing can be simpler. It does resync according to the bitmap of dirty blocks (when enabled). No journaling is involved. Why object data mutations (write IOs to an object) can't be tracked using old-school bitmap? Let's avoid any parallel access to the same image, here I consider only one particular case: VM opens its own image, i.e. one image per one client, thus no parallel access (with parallel access everything gets complicated, but IMHO not unsolvable). So having some restrictions we can track object data modifications (i.e. plain write IOs) in a bitmap.
AFAIK Linstor/drbd9 does a similar thing, something like "write intent journaling", to provide fast resyncs, even though they don't have EC (they have it in beta). So I think the client-based approach can only be possible without fully ditching atomicity. Maybe for example it should involve some additional lazy synchronization among OSDs themselves using pglogs from client-driven writes. Hm.. sounds like it's even not unreal in ceph..:-)
Unfortunately I can say nothing regarding linstor, but having simple (what is important to me is plain C :) and fast pech osd implementation of basic logic (transport and monitor client) it is possible to test and prove/disprove some interesting ideas.
Regarding the latency test, it also depends on network latency, so local test isn't the same as a clustered one.
True, is not the same, but we compare osds between each other, not the localhost network and cluster network. Just extrapolate the difference between osds to any other network setup and you will be almost certainly correct :) And frankly, now I do not have any real cluster in my disposal. So could test only on my local machine.
But thanks anyway. Pech OSD seems fast :-)
To reach current crimson state peering has to be implemented, though I never had plans to compete with crimson. What especially warms my heart is compilation time: 4 seconds on 8 cpus :) -- Roman _______________________________________________ Dev mailing list -- dev@xxxxxxx To unsubscribe send an email to dev-leave@xxxxxxx
