On 03/02/2010 01:31 PM, Pete Zaitcev wrote:
This patch adds the self-check to Chunk. With it, the daemon can rescan all of its keys and drop those that fail to match their own checksums or throw an I/O error. Objects that are found found faulty are made invisible to applications (back-end files are renamed, so that bad blocks are not reused). This is intended to work in concert with applications that store redundant copies of their data objects. This patch includes the part that tracks the active I/O, since the self-check is the only user of it, so it made little sense to separate the two. We have to track the I/O so that self-check does not mistakenly assume a partially stored object to be faulty and kills it. Running the self-check can adversely affect performance. As a crude way to limit the problem, we limit the load to one check thread only. Still, as anyone who had mlocate or Beagle running in their desktop knows, the biggest issue is not additional I/O as such, but the blowing away the page cache and dentries in kernel. Also, our scheduling is between rudimentary and non-existing. We only provide a looping check with a randomized delay and an external control to start and verify the running of the checking. Therefore, to avoid surprises with sudden loss of objects and with performance anomalies, periodic self-check defaults to off. Still, self-check is an intergral part of the daemon, so we include unit tests for both the I/O tracking and self-check itself. Signed-off-by: Pete Zaitcev<zaitcev@xxxxxxxxxx> --- doc/setup.txt | 10 + include/Makefile.am | 2 include/chunk-private.h | 4 include/chunk_msg.h | 21 ++ include/chunkc.h | 4 include/objcache.h | 75 ++++++++ lib/chunkdc.c | 82 +++++++++ server/Makefile.am | 3 server/be-fs.c | 130 ++++++++++++++ server/chunkd.h | 28 +++ server/cldu.c | 1 server/config.c | 18 ++ server/objcache.c | 138 +++++++++++++++ server/object.c | 9 + server/selfcheck.c | 293 +++++++++++++++++++++++++++++++++ server/server.c | 135 +++++++++++++++ test/.gitignore | 2 test/Makefile.am | 7 test/objcache-unit.c | 64 +++++++ test/selfcheck-unit.c | 334 ++++++++++++++++++++++++++++++++++++++ test/test.h | 2 tools/chcli.c | 116 +++++++++++-- 22 files changed, 1454 insertions(+), 24 deletions(-)
applied... looks mostly OK. Things like self-check period still need to be removed, and a few other minor quibbles. But let's go ahead and get this in, then tackle those things.
-- To unsubscribe from this list: send the line "unsubscribe hail-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
