Reg. Persistent Metadata API

[Karthi <karthikeyani.palanisami@xxxxxxxxx>]

Hi everyone,

Our group works on a device mapper target called dmDedup that performs
inline data deduplication. The dmDedup target internally uses the Persistent Metadata API (located in linux/drivers/md/persistent-data)to manage its meta-data.

In our system, there is a main writer thread (or several of them) and a separate flusher thread. When a write request comes in, the writer thread checks for the duplicates in the data and updates corresponding meta-data. The flusher thread periodically (10 sec) flushes the meta-data to disk by calling the commit function in the transaction manager of Persistant Metadata API.

When we benchmark dmDedup target with a sequential unique data write workload, we observed that write performance periodically drops to 0 ops/sec for 8 to 15 seconds. It happens because the flusher thread blocks the writer thread. Our further analysis showed that, the writer thread is blocked on the dm_bufio mutex lock, which is acquired by the flusher thread on the dm_bufio_client  when the dirty buffers are being flushed as a part of transaction. Though these dirty buffers are flushed asynchronously, the function enters a loop with the lock acquired and exits only when it places the request for all the dirty buffers. (drivers/md/dm-bufio.c - dm_bufio_write_dirty_buffers()). The writer thread is blocked as it needs this lock to lookup the metadata. The underlying I/O queue size is 128. The flusher thread waits to place IO request for all dirty buffers (which is much greater than 128) onto the queue. This way the writes becomes synchronous during this entire period, thus blocking the other writer thread.

We want a metadata flush which is not affected by the writer thread so drastically. Specifically, we would like the flush to write out dirty buffers, while the writer still can update metadata (using COW to preserve consistency). Is it something that is possible with a current design of persistent metadata subsystem? Our current understanding is that during the commit all updates to the meta-data are blocked until all of the meta-data is written out.

We appreciate any suggestions!

Thank you!

--
dm-devel mailing list
dm-devel@xxxxxxxxxx
https://www.redhat.com/mailman/listinfo/dm-devel