On Fri, Jan 13, 2012 at 14:33, Andrey Stepachev <octo47@xxxxxxxxx> wrote: > 2012/1/14 Noah Watkins <noahwatkins@xxxxxxxxx>: >> On Fri, Jan 13, 2012 at 07:12, Andrey Stepachev <octo47@xxxxxxxxx> wrote: >>> From: Andrey Stepachev <octo@xxxxxxxxxxxxxx> >>> >>> Hadoop always see wrong dates due of wrong timestamp calculation. >>> Possibly fixes #1666. >> >> Using coarse-grained timestamps might mask the problem in 1666. But it >> would only work in certain circumstances such as when clock sync skew >> is sufficiently small, and I think edge conditions would continue to >> exist. > > No. It is really bug. Not a coarse grained solution. > Here is really bug, because if we don't divide we get > not milliseconds, we'll got seconds. In my test environment > i have time difference up to several hours. > > Java uses millis, so we must divide nanos to get millis. > java_mtime += st.st_mtim.tv_nsec / 1000; > >> A more robust solution is probably needed for #1666 :( > > How about new function ceph_lstat_nocache, which > in turn will call lstat with flags == 0. In that case > _get_attr always will query MDS (if I understand code, > but I can be wrong). It is a performance hit, but > hadoop should work. > We can ever do lstat twice and get the biggest > mtime. Yeh, we need some way to refresh the cached stats at the client (if i remember 1666 correctly). If you are running into this issue there is a small hack for Hadoop that you an apply (it's on the wiki) until something for 1666 is figured out. By the way, these JNI wrappers are being rewritten to be Hadoop agnostic (pure libcephfs), and the code is at github.com/noahdesu/java-cephfs -- I was just getting ready to port the stat code, so it is nice you brought this issue back up :) > >> >>> >>> Signed-off-by: Andrey Stepachev <octo@xxxxxxxxxxxxxx> >>> >>> --- >>> src/client/hadoop/CephFSInterface.cc | 8 ++++---- >>> 1 files changed, 4 insertions(+), 4 deletions(-) >>> >>> - long long java_mtime(st.st_mtime); >>> + long long java_mtime(st.st_mtim.tv_sec); >> >> mtime vs mtim: the later is non-portable? > > Yeah. Yes. This can be safely removed from patch. > >> >> -Noah > > > > -- > Андрей. -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
