Took the liberty of adding [Outreachy] to the subject, makes it easier
for others to search as well (probably). If people have objections, let
me know and I will post a single response here removing the same for the
future.
<rest inline>
On 10/11/2016 03:53 PM, Menaka Mohan wrote:
Hi,
I am Menaka M. I am interested in participating in this round of
Outreachy under Gluster. I am new to this open source world. Kindly help
me with the following query.
Welcome!
I have set up the Gluster development environment with two servers and
one client. I am trying to run the basic bench test on the Gluster
cluster from this GitHub repo
<https://github.com/gluster/gbench/tree/master/bench-tests/bt-0000-0001>. I
also have IOZone installed. While trying to run the provided script, i
get the following error. I was trying to identify the cause of the
error. Kindly help me with that.
It looks like you have made progress (from your IRC ping on clients.ioz
and its contents). I guess you got the CLIENTS, SERVERS and some other
prerequisites right. It also looks like you either figured out how to
setup rsh for iozone, or exported RSH env-var as 'ssh'. So some good
progress there.
It would be nice if you could take some time out to add to the README
these additional prerequisites. (when you get the time) (you would be
looking at github pull requests [1] for the same)
So, I learned more about IOZone and also the performance testing section
in the Gluster docs. With that knowledge and to learn more, I have gone
through the code and running iozone commands mentioned in the
GlusterBench.py script individually.
If I am asking a very basic thing, apologies. I will quickly learn things.
Nope! not necessarily basic, as I got some failures too when using
latest iozone binaries/sources. One such failure was in [2] where the
latest iozone results are changed to report "kB/...", rather than
"KB/..." and so the parsing failed. Anyway, I made a local fix to my
python script for the same (will push the change out soonish).
So coming to your test failure:
- Could you post your full log using something like fpaste, and share
the link (helps not making any assumptions on my part) [3]
- I hit a similar failure in my tests, see [4]. This run seems to have
hit some ssh connection issue, because of which a sample was left as
"None" (which is the default)
- In your case *all* samples are left as "None". So I suspect a more
generic parsing failure of the results as obtained from *every* iozone run
- To detect what failed and why, it would be better to take a look at
the entire log
- Additionally, you could also add a few prints within the
extract_iozone_result function to debug the root cause of failure
Regards,
Menaka M
----------------------------------------------------------------------------------------------------------------------------------
python GlusterBench.py -s 64 -f 10000 -n 5 -v
Number threads = 4
Client list = HadoopSlave4
Running IOZone with 64KB record size and 4 threads, Creating an 8 GB
file with every thread.
Running smallfile with 64KB files, creating 10000 files.
Running squential IOZone tests, starting with sequential writes.
About to gather sample --> 0
Iozone: Performance Test of File I/O
Version $Revision: 3.429 $
Compiled for 64 bit mode.
Build: linux-AMD64
Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
Were there any additional output lines here? I do understand that you
have kept the output terse later, but to understand parsing errors, the
output from iozone would be helpful.
Adding the current sample to the list: None
The above line is where the parsing has failed, it should retrieve a
number and not a default string "None" from the routine.
Dropping cache
/root/sync-drop-caches.sh: 13: /root/sync-drop-caches.sh: Bad substitution
I think the default shell in your environment is not bash, and that
leads to the above "Bad substitution". On a hunch I am providing a stack
overflow link for the same [5]. This should not have a repercussion on
the test result, but I could be mistaken.
Cleaning up files.
About to gather sample --> 1
/* I removed the Iozone and Contributors header */
Adding the current sample to the list: None
Dropping cache
/root/sync-drop-caches.sh: 13: /root/sync-drop-caches.sh: Bad substitution
Cleaning up files.
About to gather sample --> 2
Adding the current sample to the list: None
Dropping cache
/root/sync-drop-caches.sh: 13: /root/sync-drop-caches.sh: Bad substitution
Cleaning up files.
About to gather sample --> 3
Adding the current sample to the list: None
Dropping cache
/root/sync-drop-caches.sh: 13: /root/sync-drop-caches.sh: Bad substitution
Cleaning up files.
About to gather sample --> 4
Adding the current sample to the list: None
Dropping cache
/root/sync-drop-caches.sh: 13: /root/sync-drop-caches.sh: Bad substitution
No cleanup on 4 iteration.
The results for sequential writes are: [None, None, None, None, None]
And, it looks like every instance has had some problem causing the
entire array of result to just have "None" as printed above.
Traceback (most recent call last):
File "GlusterBench.py", line 255, in <module>
main()
File "GlusterBench.py", line 76, in main
average_seq_write = find_average(result1)
File "GlusterBench.py", line 250, in find_average
total = total + int(samples_in[x])
TypeError: int() argument must be a string or a number, not 'NoneType'
-----------------------------------------------------------------------------------------------------------------------------------------
Shyam
[1] github pul requests:
https://help.github.com/articles/about-pull-requests/
[2] Parsing assert in the script, when using latest iozone:
https://github.com/gluster/gbench/blob/master/bench-tests/bt-0000-0001/GlusterBench.py#L178
[3] Fedora paste: https://paste.fedoraproject.org/
[4] Similar error in my setup:
http://paste.fedoraproject.org/449085/89758147/
[5] bash "Bad Substitution" error possible help:
http://stackoverflow.com/questions/20615217/bash-bad-substitution
_______________________________________________
Gluster-devel mailing list
Gluster-devel@xxxxxxxxxxx
http://www.gluster.org/mailman/listinfo/gluster-devel
