On Fri, Nov 22, 2019 at 11:45 AM Barak Sason Rofman <bsasonro@xxxxxxxxxx> wrote:
Thank you for your input Atin and Xie Changlong.
Regarding log ordering - my initial thought was to do it offline using a
dedicated too. Should be straight forward, as the logs have time stamp
composed of seconds and microseconds, so ordering them using this value is
definitely possible.
This is actually one of the main reasons I wanted to bring this up for
discussion - will it be fine with the community to run a dedicated tool to
reorder the logs offline?
Reordering the logs offline will allow us to gain the most performance
improvement, as keeping the logs order online will have some cost (probably
through stronger synchronization).
Moreover, we can take log viewing one step further and maybe create some
GUI system (JAVA based?) to view and handle logs (e.g. one window to show
the combined order logs, other windows to show logs per thread etc').
If you need an external tool (please not Java - let's not add another language to the project), you might as well move to binary logging.
I believe we need to be able to do this sort using Linux command line tools only.
Regarding the test method - my initial testing was done by removing all
logging from regression. Modifying the method "skip_logging" to return
'true' in all cases seems to remove most of the logs (though not all, "to
be on the safe side", really removing all logging related methods is
probably even better).
This is not a fair comparison:
1. The regression tests are running with debug log
2. Not logging at all != replacing the logging framework - the new one will have its own overhead as well.
3. I believe there's a lot of code that is not real life scenarios there - such as dumping a lot of data there (which causes a lot of calls to inode_find() and friends - for example).
Y.
As regression tests use mostly single-node tests, some additional testing
was needed. I've written a couple of very basic scripts to create large
number of files / big files, read / write to / from them, move them around
and perform some other basic functionality.
I'd actually be glad to test this in some 'real world' use cases - if you
have specific use cases that you use frequently, we can model them and
benchmark against - this will likely offer an even more accurate benchmark.
On Fri, Nov 22, 2019 at 7:27 AM Xie Changlong <zgrep@xxxxxxx> wrote:
>
> 在 2019/11/21 21:04, Barak Sason Rofman 写道:
>
> I see two design / implementation problems with that mechanism:
>
> 1.
>
> The mutex that guards the log file is likely under constant contention
________ Community Meeting Calendar: APAC Schedule - Every 2nd and 4th Tuesday at 11:30 AM IST Bridge: https://bluejeans.com/441850968 NA/EMEA Schedule - Every 1st and 3rd Tuesday at 01:00 PM EDT Bridge: https://bluejeans.com/441850968 Gluster-users mailing list Gluster-users@xxxxxxxxxxx https://lists.gluster.org/mailman/listinfo/gluster-users
