Gluster Tools infrastructure
============================
Are our Gluster log files sufficient for root causing the issues? Is
that error caused due to miss configuration? Geo-replication status is
showing faulty. Where to find the reason for Faulty?
Sac(surs AT redhat.com) mentioned that heis working on gdeploy and many
developers
are using their owntools. How about providing common infrastructure(say
gtool/glustertool) to host all these tools.
From my toolkit, following tools are available, planning to create
more such tools for Geo-replication and Gluster.
volinfo [<VOLNAME>] - Enhanced version of Gluster Volume info
command (http://aravindavk.in/blog/glusterfs-tools/ )
df - df for Gluster Volumes
(http://aravindavk.in/blog/glusterdf-df-for-gluster-volumes/ )
georepsetup - A tool to Create Geo-replication session
easily(http://aravindavk.in/blog/introducing-georepsetup/ )
gdash - A simple Dashboard for
Gluster(http://aravindavk.in/blog/introducing-gdash/ )
gfid <PATH> - To get GFID of a file, works both in Mount and
Backend(https://github.com/aravindavk/gluster_georep_scripts )
clparser <PATH> - Parse the backend Changelog and print in human
readable
format(https://github.com/aravindavk/gluster_georep_scripts )
xtime <PATH> - To get XTIME xattr from given
path(https://github.com/aravindavk/gluster_georep_scripts )
stime <PATH> - To get STIME xattr from given path(Used by
Geo-replication
https://github.com/aravindavk/gluster_georep_scripts )
volmark <VOLNAME> - To get Volume Mark of given Volume(Used by
Geo-replication
https://github.com/aravindavk/gluster_georep_scripts )
Geo-replication developers are already using some tools like Changelog
parser, `arequal-checksum` etc.
Initial idea for Tools Framework:
---------------------------------
A Shell/Python script which looks for the tool in plugins sub
directory, if
exists pass all the arguments and call that script.
`glustertool help` triggers a python Script plugins/help.py which reads
plugins.yml file to get the list of tools and help messages associated
with it.
No restrictions on the choice of programming language to create
tool. It can be bash, Python, Go, Rust, awk, sed etc.
Challenges:
- Each plugin may have different dependencies, installing all tools
may install all the dependencies.
- Multiple programming languages, may be difficult to maintain/build.
- Maintenance of Third party tools.
- Creating Plugins registry to discover tools created by other
developers.
Tool Ideas:
-----------
If you are interested in working on tools for Gluster, I am listing a
few ideas to start with, feel free to add your ideas to the list.
- A tool to analyze the log file and identify issues. For example,
glustertool georep_log_analize <LOG FILE PATH> --after-date
<TIMESTAMP>
Example output: (Illustrative purpose only)
Number of workers in this node: 2
Number of restarts: 5
Errors: 10
Python Tracebacks: 5
Last state: Active
Files Skipped: 0
Setup issue: No
- Extract skipped GFIDs from Geo-replication logs and re-trigger sync.
For example,
glustertool georep_extract_skipped <LOG_FILE> --after-date<TIMESTAMP>
This command will
1. extract Skipped GFIDs list,
2. Mounts Master Volume
3. converts GFID to Path
4. Set Virtual xattr to re-trigger the sync
- A tool to detect Split brain
- A tool to convert GFID to Path
Created a etherpad to record the available tools and ideas
https://public.pad.fsfe.org/p/gluster-tools
Will update once the I make some progress in creating infrastructure.
Comments and Suggestions welcome.
