Re: Question on the rename atomicity behavior in gluster

Mohammed Rafi K C <rkavunga@xxxxxxxxxx> · Wed, 23 Mar 2016 11:55:30 +0530



    On 03/23/2016 04:10 AM, Rama Shenai
      wrote:

    
        Hi, We had some questions with respect to expectations of
          atomicity of rename in gluster.
        

        To elaborate : 
        

        We have setup with two machines (lets call them M1 and M2)
          which essentially access a file (F) on a gluster volume
          (mounted by M1 and M2)
        A program does the following steps sequentially on each of
          the two machines (M1 & M2) in an infinite loop
         1) Opens the file F in O_RDWR|O_EXCL mode, reads some data
          and closes (F)
         2)  Renames some other file F' => F
         
        Periodically either M1 or M2 sees a "Stalefile handle
          error" when it tries to read the file (step (1)) after opening
          the file in O_RDWR|O_EXCL (the open is successful)
        

        The specific error reported the client volume logs
           (/var/log/glusterfs/mnt-repos-volume1.log)
        [2016-03-21 16:53:17.897902] I
          [dht-rename.c:1344:dht_rename] 0-volume1-dht: renaming
          master.lock
          (hash=volume1-replicate-0/cache=volume1-replicate-0) =>
          master (hash=volume1-replicate-0/cache=<nul>)
        [2016-03-21 16:53:18.735090] W
          [client-rpc-fops.c:504:client3_3_stat_cbk] 0-volume1-client-0:
          remote operation failed: Stale file handle
      
    
    Hi Rama,

    
    ESTALE error in rename normally generated when either the source
    file is not resolvable (deleted or inaccessible) or when the parent
    of destination is not resolvable. It can happen when let's say file
    F' was present when your application did a lookup before rename, but
    if it is got renamed by Node M1 before M2 could rename it. Basically
    a race between two rename on the same file can result in ESTALE for
    either of one.

    
    To confirm this, Can you please paste the log message from brick
    "0-volume1-client-0". You can find out the brick name from the
    graph.

    
    Also if you can share the program or snippet that used to reproduce
    this issue, that would be great.

    
    Rafi KC

    
        We see no error when: have two processes of the above
          program running on the same machine (say on M1) accessing the
          file F on the gluster volume, for which we want to understand
          the expectations of atomicity in gluster specifically
          specifically for rename, and if the above is a bug.
        

        Also  glusterfs --version => glusterfs 3.6.9 built on
          Mar  2 2016 18:21:14
        

        We also would like to know if there any parameter in the
          one translators  that we can tweak to prevent this problem
        

        Any help or insights here is appreciated
        

        Thanks
        Rama
        

      _______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users
    
    
_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-users