On 02/28/2017 12:51 AM, Mohammed Rafi K C wrote:
Hi All,
We discussed the problem $subject in the mail thread [1]. Based on the
comments and suggestions I will summarize the design (Made as points for
simplicity.)
Some generic comment(s),
- I would like to see the design in such a form that, when server side
replication and disperse become a reality, parts of the
design/implementation remain the same. This way work need not be redone
when server side anything happens. (I would have extended this to adapt
to DHT2 as well, but will leave that to you)
- As this evolves we need to assign clear responsibilities to different
xlators that we are discussing, which will eventually happen anyway, but
just noting it ahead for clarity as we discuss this further.
1) As part of each fop, top layer will generate a time stamp and pass it
to the down along with other param.
1.1) This will bring a dependency for NTP synced clients along with
servers
As stated by others and even noted by you, this requirement is a bit of
a bother, but as the current DHT, AFR, EC architecture stands, this is
what we need to start with. Just noting my observation as well on this.
1.2) There can be a diff in time if the fop stuck in the xlator for
various reason, for ex: because of locks.
2) On the server posix layer stores the value in the memory (inode ctx)
and will sync the data periodically to the disk as an extended attr
I believe this should not be a part of the posix xlator. our posix store
definition is that it uses a local file-system underneath that is POSIX
compliant. This need, for storing the time information outside of POSIX
specification (as an xattr or otherwise), is something that is gluster
specific. As a result we should not fold this into the posix xlator.
For example, if we replace posix later with a db store, or a key-vlaue
store, we would need to code this cache management of time information
again for these stores, but if we abstract it out, then we do not need
to do the same.
I may need to chew on this further, but at the moment I think this
functionality should be stuffed into posix-xlator, and rather should
live on its own.
2.1) of course sync call also will force it. And fop comes for an
inode which is not linked, we do the sync immediately.
3) Each time when inodes are created or initialized it read the data
from disk and store it.
4) Before setting to inode_ctx we compare the timestamp stored and the
timestamp received, and only store if the stored value is lesser than
the current value.
5) So in best case data will be stored and retrieved from the memory. We
replace the values in iatt with the values in inode_ctx.
6) File ops that changes the parent directory attr time need to be
consistent across all the distributed directories across the subvolumes.
(for eg: a create call will change ctime and mtime of parent dir)
6.1) This has to handle separately because we only send the fop to
the hashed subvolume.
6.2) We can asynchronously send the timeupdate setattr fop to the
other subvoumes and change the values for parent directory if the file
fops is successful on hashed subvolume.
Am I right in understanding that this is not part of the solution, and
just a suggestion on what we may do in the future, or is it part of the
solution proposed?
If the latter (i.e part of the solution proposed), which layer has the
responsibility to asynchronously update other DHT subvolumes?
For example, posix-xlator does not have that knowledge, so it should not
be that xlator, which means it is some other xlator, now is that on the
client or on the server, and how is it crash consistent are some things
that come to mind when reading this, but will wait for the details
before thinking aloud.
6.3) This will have a window where the times are inconsistent
across dht subvolume (Please provide your suggestions)
7) Currently we have couple of mount options for time attributes like
noatime, relatime , nodiratime etc. But we are not explicitly handled
those options even if it is given as mount option when gluster mount. [2]
7.1) We always relay on back end storage layer behavior, if you
have given those mount options when you mount your disk, you will get
this behaviour
7.2) Now if we are taking effort to fix the consistency issue, do
we need to honour those options by our own ?
Please provide your comments and suggestions.
[1] :
http://lists.gluster.org/pipermail/gluster-devel/2016-January/048003.html
Regards
Rafi KC
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://lists.gluster.org/mailman/listinfo/gluster-devel
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://lists.gluster.org/mailman/listinfo/gluster-devel
