Ok, so maybe just launch the container with /bin/bash or /bin/sh instead of having it run FIO. Then once you are inside the running container, cd to /fio and manually run the ./configure to see if libaiao is being found. It is possible that it got installed in a different directory than what ./configure is expecting.
As far as the host filesystem goes, when running a simple container as you showed below, it will be using AUFS as an overlay filesystem and not performing I/O against the host filesystem. I would recommend mounting a volume in a directory that is part of the mounted filesystem you wish to profile, and tell FIO to use that path instead. That way you are closer to profiling the host vs. profiling Docker AUFS.
If you really want to ensure you are going directly against the host filesystem, I would think that you would need to run privileged and talk directly to the desired path.
Regards,
Mark
Mark Beierl
SW System Sr Principal Developer
Dell EMC | Cloud & Communication Service Provider Solution
Mark.Beierl@xxxxxxxx
On 2018-11-25, 16:01, "Mohanraj B" <bmohanraj91@xxxxxxxxx> wrote:
[EXTERNAL EMAIL]
Hi,
1) How are you installing FIO – from the apt repos, or building from source?
I build fio from source, below is my dockerfile.
FROM centos:latest
RUN yum update -y
RUN yum install git gcc gcc-c++ make -y
RUN git clone https://github.com/axboe/fio
RUN yum install zlib-devel libaio -y
WORKDIR /fio
RUN ./configure
RUN make
RUN make install
VOLUME /tmp/fio-data
WORKDIR /tmp/fio-data
ENTRYPOINT ["fio"]
2) What storage are you using in the container – a mounted volume or
are you trying to use the host OS/raw device?
After reading this question, I getting more doubts, with the above
docker file and few examples given below.
docker run fio:6.0 --name:job1 --size=50m --rw=read
docker run fio:6.0 --name:job1 --size=100m --rw=readwrite
Am I really testing the host HDD or testing docker volume only. If I
am testing only the docker volume then its not my intended use case, I
need to test host machine HDD.
What is the best way to test host machine HDD ?
3) Is the container running as privileged if you are trying to access the host?
Only when I am trying to test a RAW device, I use --privileged=true
like below otherwise I dont supply --privileged=true .
docker run --privileged=true fio:6.0 --name=file --size=100m
--filename=/dev/sdc
Thanks and Regards,
Mohan
On 11/26/18, Beierl, Mark <Mark.Beierl@xxxxxxxx> wrote:
> Hello!
>
> I have a couple of questions, as I also build FIO in an Alpine container,
> but there I build it from source so that I have all required libraries. The
> source for the container is here [1] and it is part of the OPNFV StorPerf
> project, which is for wrapping FIO in a web front end and making it easier
> to test against various distributions or VMs [2].
>
> My questions are:
>
> 1) How are you installing FIO – from the apt repos, or building from
> source?
> 2) What storage are you using in the container – a mounted volume or are you
> trying to use the host OS/raw device?
> 3) Is the container running as privileged if you are trying to access the
> host?
>
> [1] https://git.opnfv.org/storperf/tree/docker/storperf-master/Dockerfile
> [2] https://wiki.opnfv.org/display/storperf
>
> Regards,
> Mark
>
> Mark Beierl
> SW System Sr Principal Developer
> Dell EMC | Cloud & Communication Service Provider Solution
> Mark.Beierl@xxxxxxxx
>
> On 2018-11-25, 15:28, "fio-owner@xxxxxxxxxxxxxxx on behalf of Mohanraj B"
> <fio-owner@xxxxxxxxxxxxxxx on behalf of bmohanraj91@xxxxxxxxx> wrote:
>
>
> [EXTERNAL EMAIL]
>
> typo
> running in docker container.
>
> On 11/26/18, Mohanraj B <bmohanraj91@xxxxxxxxx> wrote:
> > Hello,
> >
> > I am trying to use libaio engine for a job file in my docker
> > container, but it throws fio: engine libaio not loadable error.
> >
> > fio --name=hello1 --size=1m --ioengine=libaio
> >
> > fio: engine libaio not loadable
> > fio: engine libaio not loadable
> > fio: failed to load engine
> > fio: file:ioengines.c:89, func=dlopen, error=libaio: cannot open
> > shared object file: No such file or directory
> >
> > Available IO engines:
> > [root@210df130f2c7 fio-data]# fio --enghelp
> > Available IO engines:
> > cpuio
> > mmap
> > sync
> > psync
> > vsync
> > pvsync
> > pvsync2
> > null
> > net
> > netsplice
> > ftruncate
> > filecreate
> > posixaio
> > falloc
> > e4defrag
> > splice
> > mtd
> > sg
> >
> >
> > libaio is not listed above, but its installed in container.
> >
> > yum install libaio
> > Loaded plugins: fastestmirror, ovl
> > Loading mirror speeds from cached hostfile
> > * base: ftp.iitm.ac.in
> > * extras: ftp.iitm.ac.in
> > * updates: ftp.iitm.ac.in
> > Package libaio-0.3.109-13.el7.x86_64 already installed and latest
> version
> > Nothing to do
> >
> > Centos docker container : centos-release-7-5.1804.5.el7.centos.x86_64
> >
> >
> > Re pvsync2 operation not permitted error
> >
> > fio --name=hello --size=10m --ioengine=pvsync2
> >
> > hello: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T)
> > 4096B-4096B, ioengine=pvsync2, iodepth=1
> > fio-3.12-18-g92a1
> > Starting 1 process
> > hello: Laying out IO file (1 file / 10MiB)
> > fio: io_u error on file hello.0.0: Operation not permitted: read
> > offset=0, buflen=4096
> > fio: pid=59, err=1/file:io_u.c:1789, func=io_u error, error=Operation
> > not permitted
> >
> > hello: (groupid=0, jobs=1): err= 1 (file:io_u.c:1789, func=io_u
> error,
> > error=Operation not permitted): pid=59: Sun Nov 25 19:23:14 2018
> > cpu : usr=0.00%, sys=0.00%, ctx=2, majf=0, minf=19
> > IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%,
> >>=64=0.0%
> > submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
> >>=64=0.0%
> > complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%,
> >>=64=0.0%
> > issued rwts: total=1,0,0,0 short=0,0,0,0 dropped=0,0,0,0
> > latency : target=0, window=0, percentile=100.00%, depth=1
> >
> > Run status group 0 (all jobs):
> >
> > Disk stats (read/write):
> > sda: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%
> >
> > I dont have access to machine running centos, so running in centos
> > docker container. Are these issues occuring because of running in
> fio?
> >
> > host machine OS is ubuntu 18.04.
> >
> > Thanks and Regards,
> > Mohan
> >
>
>
>
