Going ahead on my translator attempt, the file read at node "A" and write at node "B" is working OK already with the syncop calls since some days now. I am copying the same gfid to the temporary local cached file, but I am using a different name and path at this file at node B because it wasn't my plan to have all the dir paths replicated at all nodes.
So, at the first read of the file it gets the content from the node "A" where the original file is placed. Although, once this is done I insert some extended metadata there so that further calls coming from node B can know the file is cached there.
This implies on a redirection of the open and readv calls, but I am always getting a "file not found" error at the end because the posix layer is trying to get the original path at the posix level within resolve_gfid(). Instead of getting the hash for the file created at node B /48344277 which is stored at <node B brick>/.glusterfs/20/dd/20dd... it is looking for the <node B brick>/.glusterfs/bc/47/bc47... in this case, which is the hash created by the posix layer to /dir01/f01.txt.
[2012-09-17 15:45:23.447620] I [gbfs_t.c:448:gbfs_open] 0-gbfs-test: entering gbfs_open function
[2012-09-17 15:45:23.448253] W [client-rpc-fops.c:470:client3_3_open_cbk] 0-examplevol-client-1: remote operation failed: No such file or directory. Path: /48344277 (bc477228-42d9-4f90-8806-bd68a5590519)
[2012-09-17 15:45:23.448271] I [gbfs_t.c:435:gbfs_open_cbk] 0-gbfs-test: entering gbfs_open_cbk function
[2012-09-17 15:45:23.448283] W [fuse-bridge.c:808:fuse_fd_cbk] 0-glusterfs-fuse: 14: OPEN() /dir01/f01.txt => -1 (No such file or directory)
I know I am doing it wrong, but I'm not sure how to correctly proceed here. Currently I am trying to follow the lookup/open/readv calls without performing suboperations apart (like syncops I used to write the file), but I'm trying to redirect them to the correct subvolume with the modified loc for the target file.
Gustavo Bervian Brand
---------------------------------------------------------------------------------
On Mon, Sep 10, 2012 at 9:30 PM, Anand Avati <anand.avati@xxxxxxxxx> wrote:
On Mon, Sep 10, 2012 at 12:11 PM, Gustavo Bervian Brand <gugabrand@xxxxxxxxx> wrote:
Hello,I wasn't using the synctask_new worker, but now I implemented my code using it to trigger the create/write call. Btw, is the usage of syncop calls documented somewhere? I found it through the history of the dev-list.But going ahead on my issue, I was able to create the file avoiding the NFS stale handle error. The file is being created at the backend, but the syncop_create call it's still returning an error from the posix translator at the server side. I copied the error from the glusterd log below... I am checking the code trying to figure it out.To write the file I am generating a new gfid for the local file and setting the pargfid as 1 (root node) with the loc.parent pointing to the inode->table->root inode. Below I copied what is being copied the loc, fd, flag and mode_t structures being used at the create call.*** From the glusterd log:[2012-09-10 18:12:34.035940] W [posix-handle.c:590:posix_handle_hard] 0-examplevol1: mismatching ino/dev between file /bricks/exampleDir_local/1ccfca9f (146628/2049) and handle /bricks/exampleDir_local/.glusterfs/00/00/00000000-0000-0000-0000-000000000000 (146617/2049)[2012-09-10 18:12:34.035977] E [posix.c:1724:posix_create] 0-examplevol1: setting gfid on /bricks/exampleDir_local/1ccfca9f failed*** Relevant code before calling the create function107 inode_t *newinode = inode_new (old_loc.inode->table);108109 loc.path = strdup(hashed_path_str);110 loc.name = strdup(hashed_filename_str);111 loc.inode = newinode;113 loc.parent = loc.inode->table->root;115 uuid_copy(loc.gfid, gfid);116118 memset (loc.pargfid, 0, 16);119 loc.pargfid[15] = 1;120121 fd = fd_create (loc.inode, getpid());...127128 uint32_t flags = O_CREAT | O_RDWR | O_EXCL;129 mode_t mode = (mode_t)0700;130 fd->flags = flags;131 fd = fd_ref(fd);It is very likely you are just missing setting the "gfid-req" field in the dictionary. You can check how fuse-bridge, nfs, api/src/glfs.c are all setting gfid-req. In your case you would want to copy the GFID of the original file for the cached file as well.Avati
