the logs I m getting when starting the server
```
ceph-mgr dashboard not built - disabling.
=== osd.2 ===
Stopping Ceph osd.2 on abhinav-Lenovo-ideapad-320-15IKB...kill 2489...kill 2489...done
=== osd.1 ===
Stopping Ceph osd.1 on abhinav-Lenovo-ideapad-320-15IKB...kill 748...kill 748...done
=== osd.0 ===
Stopping Ceph osd.0 on abhinav-Lenovo-ideapad-320-15IKB...kill 31736...kill 31736...done
=== mon.c ===
Stopping Ceph mon.c on abhinav-Lenovo-ideapad-320-15IKB...kill 30107...done
=== mon.b ===
Stopping Ceph mon.b on abhinav-Lenovo-ideapad-320-15IKB...kill 30064...done
=== mon.a ===
Stopping Ceph mon.a on abhinav-Lenovo-ideapad-320-15IKB...kill 30021...done
=== mgr.x ===
Stopping Ceph mgr.x on abhinav-Lenovo-ideapad-320-15IKB...kill 30295...done
=== mds.c ===
Stopping Ceph mds.c on abhinav-Lenovo-ideapad-320-15IKB...kill 3545...kill 3545...done
=== mds.b ===
Stopping Ceph mds.b on abhinav-Lenovo-ideapad-320-15IKB...kill 3489...kill 3489...done
=== mds.a ===
Stopping Ceph mds.a on abhinav-Lenovo-ideapad-320-15IKB...kill 3386...kill 3386...done
** going verbose **
rm -f core*
hostname abhinav-Lenovo-ideapad-320-15IKB
ip 192.168.0.108
port 40816
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-authtool --create-keyring --gen-key --name=mon. /home/abhinav/GSOC/PR/ceph/build/keyring --cap mon 'allow *'
creating /home/abhinav/GSOC/PR/ceph/build/keyring
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /home/abhinav/GSOC/PR/ceph/build/keyring
/home/abhinav/GSOC/PR/ceph/build/bin/monmaptool --create --clobber --addv a [v2:192.168.0.108:40816,v1:192.168.0.108:40817] --addv b [v2:192.168.0.108:40818,v1:192.168.0.108:40819] --addv c [v2:192.168.0.108:40820,v1:192.168.0.108:40821] --print /tmp/ceph_monmap.20439
/home/abhinav/GSOC/PR/ceph/build/bin/monmaptool: monmap file /tmp/ceph_monmap.20439
/home/abhinav/GSOC/PR/ceph/build/bin/monmaptool: generated fsid c07c2224-8dfa-4668-be3b-f78ef3ab7542
epoch 0
fsid c07c2224-8dfa-4668-be3b-f78ef3ab7542
last_changed 2020-09-11T14:13:07.057233+0530
created 2020-09-11T14:13:07.057233+0530
min_mon_release 0 (unknown)
0: [v2:192.168.0.108:40816/0,v1:192.168.0.108:40817/0] mon.a
1: [v2:192.168.0.108:40818/0,v1:192.168.0.108:40819/0] mon.b
2: [v2:192.168.0.108:40820/0,v1:192.168.0.108:40821/0] mon.c
/home/abhinav/GSOC/PR/ceph/build/bin/monmaptool: writing epoch 0 to /tmp/ceph_monmap.20439 (3 monitors)
rm -rf -- /home/abhinav/GSOC/PR/ceph/build/dev/mon.a
mkdir -p /home/abhinav/GSOC/PR/ceph/build/dev/mon.a
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-mon --mkfs -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -i a --monmap=/tmp/ceph_monmap.20439 --keyring=/home/abhinav/GSOC/PR/ceph/build/keyring
rm -rf -- /home/abhinav/GSOC/PR/ceph/build/dev/mon.b
mkdir -p /home/abhinav/GSOC/PR/ceph/build/dev/mon.b
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-mon --mkfs -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -i b --monmap=/tmp/ceph_monmap.20439 --keyring=/home/abhinav/GSOC/PR/ceph/build/keyring
rm -rf -- /home/abhinav/GSOC/PR/ceph/build/dev/mon.c
mkdir -p /home/abhinav/GSOC/PR/ceph/build/dev/mon.c
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-mon --mkfs -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -i c --monmap=/tmp/ceph_monmap.20439 --keyring=/home/abhinav/GSOC/PR/ceph/build/keyring
rm -- /tmp/ceph_monmap.20439
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-mon -i a -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-mon -i b -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-mon -i c -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf
Populating config ...
[mgr]
mgr/telemetry/enable = false
mgr/telemetry/nag = false
Setting debug configs ...
creating /home/abhinav/GSOC/PR/ceph/build/dev/mgr.x/keyring
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -i /home/abhinav/GSOC/PR/ceph/build/dev/mgr.x/keyring auth add mgr.x mon 'allow profile mgr' mds 'allow *' osd 'allow *'
added key for mgr.x
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring config set mgr mgr/prometheus/x/server_port 9283 --force
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring config set mgr mgr/restful/x/server_port 42816 --force
Starting mgr.x
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-mgr -i x -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -h
waiting for mgr restful module to start
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -h
waiting for mgr restful module to start
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -h
waiting for mgr restful module to start
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -h
waiting for mgr restful module to start
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -h
waiting for mgr restful module to start
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -h
waiting for mgr restful module to start
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -h
waiting for mgr restful module to start
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -h
waiting for mgr restful module to start
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -h
waiting for mgr restful module to start
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -h
waiting for mgr restful module to start
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -h
waiting for mgr restful module to start
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -h
waiting for mgr restful module to start
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -h
waiting for mgr restful module to start
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -h
waiting for mgr restful module to start
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -h
waiting for mgr restful module to start
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -h
waiting for mgr restful module to start
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -h
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring restful create-self-signed-cert
Restarting RESTful API server...
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring restful create-key admin -o /tmp/tmp.KMlJCQmJNJ
add osd0 06bcb0d2-dde9-480c-9376-d3981c558a6a
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring osd new 06bcb0d2-dde9-480c-9376-d3981c558a6a -i /home/abhinav/GSOC/PR/ceph/build/dev/osd0/new.json
0
2020-09-11T14:14:01.448+0530 7f4248b68d80 -1 auth: unable to find a keyring on /home/abhinav/GSOC/PR/ceph/build/dev/osd0/keyring: (2) No such file or directory
2020-09-11T14:14:01.448+0530 7f4248b68d80 -1 auth: unable to find a keyring on /home/abhinav/GSOC/PR/ceph/build/dev/osd0/keyring: (2) No such file or directory
2020-09-11T14:14:01.448+0530 7f4248b68d80 -1 auth: unable to find a keyring on /home/abhinav/GSOC/PR/ceph/build/dev/osd0/keyring: (2) No such file or directory
2020-09-11T14:14:01.608+0530 7f4248b68d80 -1 bluestore(/home/abhinav/GSOC/PR/ceph/build/dev/osd0/block) _read_bdev_label failed to open /home/abhinav/GSOC/PR/ceph/build/dev/osd0/block: (2) No such file or directory
2020-09-11T14:14:01.608+0530 7f4248b68d80 -1 bluestore(/home/abhinav/GSOC/PR/ceph/build/dev/osd0/block) _read_bdev_label failed to open /home/abhinav/GSOC/PR/ceph/build/dev/osd0/block: (2) No such file or directory
2020-09-11T14:14:01.608+0530 7f4248b68d80 -1 bluestore(/home/abhinav/GSOC/PR/ceph/build/dev/osd0/block) _read_bdev_label failed to open /home/abhinav/GSOC/PR/ceph/build/dev/osd0/block: (2) No such file or directory
2020-09-11T14:14:01.796+0530 7f4248b68d80 -1 bluestore(/home/abhinav/GSOC/PR/ceph/build/dev/osd0) _read_fsid unparsable uuid
start osd.0
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-osd -i 0 -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf
add osd1 10fa52bb-a52f-4e15-ac9a-f05dfe352e08
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring osd new 10fa52bb-a52f-4e15-ac9a-f05dfe352e08 -i /home/abhinav/GSOC/PR/ceph/build/dev/osd1/new.json
1
2020-09-11T14:14:24.848+0530 7efce203bd80 -1 Falling back to public interface
2020-09-11T14:14:24.928+0530 7faaa1f87d80 -1 auth: unable to find a keyring on /home/abhinav/GSOC/PR/ceph/build/dev/osd1/keyring: (2) No such file or directory
2020-09-11T14:14:24.928+0530 7faaa1f87d80 -1 auth: unable to find a keyring on /home/abhinav/GSOC/PR/ceph/build/dev/osd1/keyring: (2) No such file or directory
2020-09-11T14:14:24.928+0530 7faaa1f87d80 -1 auth: unable to find a keyring on /home/abhinav/GSOC/PR/ceph/build/dev/osd1/keyring: (2) No such file or directory
2020-09-11T14:14:24.932+0530 7faa9d046700 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2]
2020-09-11T14:14:24.940+0530 7faaa1f87d80 -1 bluestore(/home/abhinav/GSOC/PR/ceph/build/dev/osd1/block) _read_bdev_label failed to open /home/abhinav/GSOC/PR/ceph/build/dev/osd1/block: (2) No such file or directory
2020-09-11T14:14:24.940+0530 7faaa1f87d80 -1 bluestore(/home/abhinav/GSOC/PR/ceph/build/dev/osd1/block) _read_bdev_label failed to open /home/abhinav/GSOC/PR/ceph/build/dev/osd1/block: (2) No such file or directory
2020-09-11T14:14:24.940+0530 7faaa1f87d80 -1 bluestore(/home/abhinav/GSOC/PR/ceph/build/dev/osd1/block) _read_bdev_label failed to open /home/abhinav/GSOC/PR/ceph/build/dev/osd1/block: (2) No such file or directory
2020-09-11T14:14:25.184+0530 7faaa1f87d80 -1 bluestore(/home/abhinav/GSOC/PR/ceph/build/dev/osd1) _read_fsid unparsable uuid
2020-09-11T14:14:42.020+0530 7efce203bd80 -1 osd.0 0 log_to_monitors {default=true}
start osd.1
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-osd -i 1 -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf
add osd2 a5813f44-e770-485f-9f87-3368d16c354b
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring osd new a5813f44-e770-485f-9f87-3368d16c354b -i /home/abhinav/GSOC/PR/ceph/build/dev/osd2/new.json
2020-09-11T14:14:53.876+0530 7f0d01562d80 -1 Falling back to public interface
2
2020-09-11T14:14:54.676+0530 7fda89858d80 -1 auth: unable to find a keyring on /home/abhinav/GSOC/PR/ceph/build/dev/osd2/keyring: (2) No such file or directory
2020-09-11T14:14:54.676+0530 7fda89858d80 -1 auth: unable to find a keyring on /home/abhinav/GSOC/PR/ceph/build/dev/osd2/keyring: (2) No such file or directory
2020-09-11T14:14:54.676+0530 7fda89858d80 -1 auth: unable to find a keyring on /home/abhinav/GSOC/PR/ceph/build/dev/osd2/keyring: (2) No such file or directory
2020-09-11T14:14:54.796+0530 7fda89858d80 -1 bluestore(/home/abhinav/GSOC/PR/ceph/build/dev/osd2/block) _read_bdev_label failed to open /home/abhinav/GSOC/PR/ceph/build/dev/osd2/block: (2) No such file or directory
2020-09-11T14:14:54.796+0530 7fda89858d80 -1 bluestore(/home/abhinav/GSOC/PR/ceph/build/dev/osd2/block) _read_bdev_label failed to open /home/abhinav/GSOC/PR/ceph/build/dev/osd2/block: (2) No such file or directory
2020-09-11T14:14:54.796+0530 7fda89858d80 -1 bluestore(/home/abhinav/GSOC/PR/ceph/build/dev/osd2/block) _read_bdev_label failed to open /home/abhinav/GSOC/PR/ceph/build/dev/osd2/block: (2) No such file or directory
2020-09-11T14:14:55.092+0530 7fda89858d80 -1 bluestore(/home/abhinav/GSOC/PR/ceph/build/dev/osd2) _read_fsid unparsable uuid
2020-09-11T14:15:01.756+0530 7f0d01562d80 -1 osd.1 0 log_to_monitors {default=true}
start osd.2
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-osd -i 2 -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf
2020-09-11T14:15:24.512+0530 7fb803fa8d80 -1 Falling back to public interface
2020-09-11T14:15:31.672+0530 7fb803fa8d80 -1 osd.2 0 log_to_monitors {default=true}
OSDs started
mkdir -p /home/abhinav/GSOC/PR/ceph/build/dev/mds.a
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-authtool --create-keyring --gen-key --name=mds.a /home/abhinav/GSOC/PR/ceph/build/dev/mds.a/keyring
creating /home/abhinav/GSOC/PR/ceph/build/dev/mds.a/keyring
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -i /home/abhinav/GSOC/PR/ceph/build/dev/mds.a/keyring auth add mds.a mon 'allow profile mds' osd 'allow rw tag cephfs *=*' mds allow mgr 'allow profile mds'
added key for mds.a
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-mds -i a -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf
starting mds.a at
mkdir -p /home/abhinav/GSOC/PR/ceph/build/dev/mds.b
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-authtool --create-keyring --gen-key --name=mds.b /home/abhinav/GSOC/PR/ceph/build/dev/mds.b/keyring
creating /home/abhinav/GSOC/PR/ceph/build/dev/mds.b/keyring
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -i /home/abhinav/GSOC/PR/ceph/build/dev/mds.b/keyring auth add mds.b mon 'allow profile mds' osd 'allow rw tag cephfs *=*' mds allow mgr 'allow profile mds'
added key for mds.b
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-mds -i b -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf
starting mds.b at
mkdir -p /home/abhinav/GSOC/PR/ceph/build/dev/mds.c
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-authtool --create-keyring --gen-key --name=mds.c /home/abhinav/GSOC/PR/ceph/build/dev/mds.c/keyring
creating /home/abhinav/GSOC/PR/ceph/build/dev/mds.c/keyring
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -i /home/abhinav/GSOC/PR/ceph/build/dev/mds.c/keyring auth add mds.c mon 'allow profile mds' osd 'allow rw tag cephfs *=*' mds allow mgr 'allow profile mds'
added key for mds.c
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-mds -i c -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf
2020-09-11T14:15:49.724+0530 7ff20f790700 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2]
starting mds.c at
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring fs volume ls
[]
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring fs volume create a
new fs with metadata pool 2 and data pool 3
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring fs authorize a client.fs_a / rwp
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring fs authorize * client.fs / rwp
setting up user testid
2020-09-11T14:16:51.181+0530 7f503e09d0c0 -1 static int RGWDataChangesFIFO::exists(ceph::common::CephContext*, librados::v14_2_0::Rados*, const rgw_pool&, bool*, bool*): unable to open FIFO: default.rgw.log/data_log.0: (95) Operation not supported
2020-09-11T14:16:51.181+0530 7f503e09d0c0 -1 int RGWDataChangesLog::start(const RGWZone*, const RGWZoneParams&, RGWSI_Cls*, librados::v14_2_0::Rados*): Error when checking for existing FIFO datalog backend: (95) Operation not supported
2020-09-11T14:16:51.317+0530 7f503e09d0c0 -1 int RGWDataChangesLog::start(const RGWZone*, const RGWZoneParams&, RGWSI_Cls*, librados::v14_2_0::Rados*): Error when starting backend: Operation not supported
2020-09-11T14:16:51.317+0530 7f503e09d0c0 0 ERROR: failed to start datalog_rados service ((95) Operation not supported
2020-09-11T14:16:51.317+0530 7f503e09d0c0 0 ERROR: failed to init services (ret=(95) Operation not supported)
couldn't init storage provider
=== osd.2 ===
Stopping Ceph osd.2 on abhinav-Lenovo-ideapad-320-15IKB...kill 2489...kill 2489...done
=== osd.1 ===
Stopping Ceph osd.1 on abhinav-Lenovo-ideapad-320-15IKB...kill 748...kill 748...done
=== osd.0 ===
Stopping Ceph osd.0 on abhinav-Lenovo-ideapad-320-15IKB...kill 31736...kill 31736...done
=== mon.c ===
Stopping Ceph mon.c on abhinav-Lenovo-ideapad-320-15IKB...kill 30107...done
=== mon.b ===
Stopping Ceph mon.b on abhinav-Lenovo-ideapad-320-15IKB...kill 30064...done
=== mon.a ===
Stopping Ceph mon.a on abhinav-Lenovo-ideapad-320-15IKB...kill 30021...done
=== mgr.x ===
Stopping Ceph mgr.x on abhinav-Lenovo-ideapad-320-15IKB...kill 30295...done
=== mds.c ===
Stopping Ceph mds.c on abhinav-Lenovo-ideapad-320-15IKB...kill 3545...kill 3545...done
=== mds.b ===
Stopping Ceph mds.b on abhinav-Lenovo-ideapad-320-15IKB...kill 3489...kill 3489...done
=== mds.a ===
Stopping Ceph mds.a on abhinav-Lenovo-ideapad-320-15IKB...kill 3386...kill 3386...done
** going verbose **
rm -f core*
hostname abhinav-Lenovo-ideapad-320-15IKB
ip 192.168.0.108
port 40816
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-authtool --create-keyring --gen-key --name=mon. /home/abhinav/GSOC/PR/ceph/build/keyring --cap mon 'allow *'
creating /home/abhinav/GSOC/PR/ceph/build/keyring
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /home/abhinav/GSOC/PR/ceph/build/keyring
/home/abhinav/GSOC/PR/ceph/build/bin/monmaptool --create --clobber --addv a [v2:192.168.0.108:40816,v1:192.168.0.108:40817] --addv b [v2:192.168.0.108:40818,v1:192.168.0.108:40819] --addv c [v2:192.168.0.108:40820,v1:192.168.0.108:40821] --print /tmp/ceph_monmap.20439
/home/abhinav/GSOC/PR/ceph/build/bin/monmaptool: monmap file /tmp/ceph_monmap.20439
/home/abhinav/GSOC/PR/ceph/build/bin/monmaptool: generated fsid c07c2224-8dfa-4668-be3b-f78ef3ab7542
epoch 0
fsid c07c2224-8dfa-4668-be3b-f78ef3ab7542
last_changed 2020-09-11T14:13:07.057233+0530
created 2020-09-11T14:13:07.057233+0530
min_mon_release 0 (unknown)
0: [v2:192.168.0.108:40816/0,v1:192.168.0.108:40817/0] mon.a
1: [v2:192.168.0.108:40818/0,v1:192.168.0.108:40819/0] mon.b
2: [v2:192.168.0.108:40820/0,v1:192.168.0.108:40821/0] mon.c
/home/abhinav/GSOC/PR/ceph/build/bin/monmaptool: writing epoch 0 to /tmp/ceph_monmap.20439 (3 monitors)
rm -rf -- /home/abhinav/GSOC/PR/ceph/build/dev/mon.a
mkdir -p /home/abhinav/GSOC/PR/ceph/build/dev/mon.a
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-mon --mkfs -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -i a --monmap=/tmp/ceph_monmap.20439 --keyring=/home/abhinav/GSOC/PR/ceph/build/keyring
rm -rf -- /home/abhinav/GSOC/PR/ceph/build/dev/mon.b
mkdir -p /home/abhinav/GSOC/PR/ceph/build/dev/mon.b
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-mon --mkfs -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -i b --monmap=/tmp/ceph_monmap.20439 --keyring=/home/abhinav/GSOC/PR/ceph/build/keyring
rm -rf -- /home/abhinav/GSOC/PR/ceph/build/dev/mon.c
mkdir -p /home/abhinav/GSOC/PR/ceph/build/dev/mon.c
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-mon --mkfs -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -i c --monmap=/tmp/ceph_monmap.20439 --keyring=/home/abhinav/GSOC/PR/ceph/build/keyring
rm -- /tmp/ceph_monmap.20439
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-mon -i a -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-mon -i b -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-mon -i c -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf
Populating config ...
[mgr]
mgr/telemetry/enable = false
mgr/telemetry/nag = false
Setting debug configs ...
creating /home/abhinav/GSOC/PR/ceph/build/dev/mgr.x/keyring
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -i /home/abhinav/GSOC/PR/ceph/build/dev/mgr.x/keyring auth add mgr.x mon 'allow profile mgr' mds 'allow *' osd 'allow *'
added key for mgr.x
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring config set mgr mgr/prometheus/x/server_port 9283 --force
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring config set mgr mgr/restful/x/server_port 42816 --force
Starting mgr.x
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-mgr -i x -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -h
waiting for mgr restful module to start
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -h
waiting for mgr restful module to start
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -h
waiting for mgr restful module to start
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -h
waiting for mgr restful module to start
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -h
waiting for mgr restful module to start
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -h
waiting for mgr restful module to start
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -h
waiting for mgr restful module to start
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -h
waiting for mgr restful module to start
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -h
waiting for mgr restful module to start
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -h
waiting for mgr restful module to start
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -h
waiting for mgr restful module to start
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -h
waiting for mgr restful module to start
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -h
waiting for mgr restful module to start
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -h
waiting for mgr restful module to start
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -h
waiting for mgr restful module to start
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -h
waiting for mgr restful module to start
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -h
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring restful create-self-signed-cert
Restarting RESTful API server...
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring restful create-key admin -o /tmp/tmp.KMlJCQmJNJ
add osd0 06bcb0d2-dde9-480c-9376-d3981c558a6a
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring osd new 06bcb0d2-dde9-480c-9376-d3981c558a6a -i /home/abhinav/GSOC/PR/ceph/build/dev/osd0/new.json
0
2020-09-11T14:14:01.448+0530 7f4248b68d80 -1 auth: unable to find a keyring on /home/abhinav/GSOC/PR/ceph/build/dev/osd0/keyring: (2) No such file or directory
2020-09-11T14:14:01.448+0530 7f4248b68d80 -1 auth: unable to find a keyring on /home/abhinav/GSOC/PR/ceph/build/dev/osd0/keyring: (2) No such file or directory
2020-09-11T14:14:01.448+0530 7f4248b68d80 -1 auth: unable to find a keyring on /home/abhinav/GSOC/PR/ceph/build/dev/osd0/keyring: (2) No such file or directory
2020-09-11T14:14:01.608+0530 7f4248b68d80 -1 bluestore(/home/abhinav/GSOC/PR/ceph/build/dev/osd0/block) _read_bdev_label failed to open /home/abhinav/GSOC/PR/ceph/build/dev/osd0/block: (2) No such file or directory
2020-09-11T14:14:01.608+0530 7f4248b68d80 -1 bluestore(/home/abhinav/GSOC/PR/ceph/build/dev/osd0/block) _read_bdev_label failed to open /home/abhinav/GSOC/PR/ceph/build/dev/osd0/block: (2) No such file or directory
2020-09-11T14:14:01.608+0530 7f4248b68d80 -1 bluestore(/home/abhinav/GSOC/PR/ceph/build/dev/osd0/block) _read_bdev_label failed to open /home/abhinav/GSOC/PR/ceph/build/dev/osd0/block: (2) No such file or directory
2020-09-11T14:14:01.796+0530 7f4248b68d80 -1 bluestore(/home/abhinav/GSOC/PR/ceph/build/dev/osd0) _read_fsid unparsable uuid
start osd.0
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-osd -i 0 -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf
add osd1 10fa52bb-a52f-4e15-ac9a-f05dfe352e08
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring osd new 10fa52bb-a52f-4e15-ac9a-f05dfe352e08 -i /home/abhinav/GSOC/PR/ceph/build/dev/osd1/new.json
1
2020-09-11T14:14:24.848+0530 7efce203bd80 -1 Falling back to public interface
2020-09-11T14:14:24.928+0530 7faaa1f87d80 -1 auth: unable to find a keyring on /home/abhinav/GSOC/PR/ceph/build/dev/osd1/keyring: (2) No such file or directory
2020-09-11T14:14:24.928+0530 7faaa1f87d80 -1 auth: unable to find a keyring on /home/abhinav/GSOC/PR/ceph/build/dev/osd1/keyring: (2) No such file or directory
2020-09-11T14:14:24.928+0530 7faaa1f87d80 -1 auth: unable to find a keyring on /home/abhinav/GSOC/PR/ceph/build/dev/osd1/keyring: (2) No such file or directory
2020-09-11T14:14:24.932+0530 7faa9d046700 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2]
2020-09-11T14:14:24.940+0530 7faaa1f87d80 -1 bluestore(/home/abhinav/GSOC/PR/ceph/build/dev/osd1/block) _read_bdev_label failed to open /home/abhinav/GSOC/PR/ceph/build/dev/osd1/block: (2) No such file or directory
2020-09-11T14:14:24.940+0530 7faaa1f87d80 -1 bluestore(/home/abhinav/GSOC/PR/ceph/build/dev/osd1/block) _read_bdev_label failed to open /home/abhinav/GSOC/PR/ceph/build/dev/osd1/block: (2) No such file or directory
2020-09-11T14:14:24.940+0530 7faaa1f87d80 -1 bluestore(/home/abhinav/GSOC/PR/ceph/build/dev/osd1/block) _read_bdev_label failed to open /home/abhinav/GSOC/PR/ceph/build/dev/osd1/block: (2) No such file or directory
2020-09-11T14:14:25.184+0530 7faaa1f87d80 -1 bluestore(/home/abhinav/GSOC/PR/ceph/build/dev/osd1) _read_fsid unparsable uuid
2020-09-11T14:14:42.020+0530 7efce203bd80 -1 osd.0 0 log_to_monitors {default=true}
start osd.1
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-osd -i 1 -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf
add osd2 a5813f44-e770-485f-9f87-3368d16c354b
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring osd new a5813f44-e770-485f-9f87-3368d16c354b -i /home/abhinav/GSOC/PR/ceph/build/dev/osd2/new.json
2020-09-11T14:14:53.876+0530 7f0d01562d80 -1 Falling back to public interface
2
2020-09-11T14:14:54.676+0530 7fda89858d80 -1 auth: unable to find a keyring on /home/abhinav/GSOC/PR/ceph/build/dev/osd2/keyring: (2) No such file or directory
2020-09-11T14:14:54.676+0530 7fda89858d80 -1 auth: unable to find a keyring on /home/abhinav/GSOC/PR/ceph/build/dev/osd2/keyring: (2) No such file or directory
2020-09-11T14:14:54.676+0530 7fda89858d80 -1 auth: unable to find a keyring on /home/abhinav/GSOC/PR/ceph/build/dev/osd2/keyring: (2) No such file or directory
2020-09-11T14:14:54.796+0530 7fda89858d80 -1 bluestore(/home/abhinav/GSOC/PR/ceph/build/dev/osd2/block) _read_bdev_label failed to open /home/abhinav/GSOC/PR/ceph/build/dev/osd2/block: (2) No such file or directory
2020-09-11T14:14:54.796+0530 7fda89858d80 -1 bluestore(/home/abhinav/GSOC/PR/ceph/build/dev/osd2/block) _read_bdev_label failed to open /home/abhinav/GSOC/PR/ceph/build/dev/osd2/block: (2) No such file or directory
2020-09-11T14:14:54.796+0530 7fda89858d80 -1 bluestore(/home/abhinav/GSOC/PR/ceph/build/dev/osd2/block) _read_bdev_label failed to open /home/abhinav/GSOC/PR/ceph/build/dev/osd2/block: (2) No such file or directory
2020-09-11T14:14:55.092+0530 7fda89858d80 -1 bluestore(/home/abhinav/GSOC/PR/ceph/build/dev/osd2) _read_fsid unparsable uuid
2020-09-11T14:15:01.756+0530 7f0d01562d80 -1 osd.1 0 log_to_monitors {default=true}
start osd.2
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-osd -i 2 -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf
2020-09-11T14:15:24.512+0530 7fb803fa8d80 -1 Falling back to public interface
2020-09-11T14:15:31.672+0530 7fb803fa8d80 -1 osd.2 0 log_to_monitors {default=true}
OSDs started
mkdir -p /home/abhinav/GSOC/PR/ceph/build/dev/mds.a
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-authtool --create-keyring --gen-key --name=mds.a /home/abhinav/GSOC/PR/ceph/build/dev/mds.a/keyring
creating /home/abhinav/GSOC/PR/ceph/build/dev/mds.a/keyring
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -i /home/abhinav/GSOC/PR/ceph/build/dev/mds.a/keyring auth add mds.a mon 'allow profile mds' osd 'allow rw tag cephfs *=*' mds allow mgr 'allow profile mds'
added key for mds.a
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-mds -i a -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf
starting mds.a at
mkdir -p /home/abhinav/GSOC/PR/ceph/build/dev/mds.b
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-authtool --create-keyring --gen-key --name=mds.b /home/abhinav/GSOC/PR/ceph/build/dev/mds.b/keyring
creating /home/abhinav/GSOC/PR/ceph/build/dev/mds.b/keyring
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -i /home/abhinav/GSOC/PR/ceph/build/dev/mds.b/keyring auth add mds.b mon 'allow profile mds' osd 'allow rw tag cephfs *=*' mds allow mgr 'allow profile mds'
added key for mds.b
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-mds -i b -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf
starting mds.b at
mkdir -p /home/abhinav/GSOC/PR/ceph/build/dev/mds.c
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-authtool --create-keyring --gen-key --name=mds.c /home/abhinav/GSOC/PR/ceph/build/dev/mds.c/keyring
creating /home/abhinav/GSOC/PR/ceph/build/dev/mds.c/keyring
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring -i /home/abhinav/GSOC/PR/ceph/build/dev/mds.c/keyring auth add mds.c mon 'allow profile mds' osd 'allow rw tag cephfs *=*' mds allow mgr 'allow profile mds'
added key for mds.c
/home/abhinav/GSOC/PR/ceph/build/bin/ceph-mds -i c -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf
2020-09-11T14:15:49.724+0530 7ff20f790700 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2]
starting mds.c at
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring fs volume ls
[]
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring fs volume create a
new fs with metadata pool 2 and data pool 3
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring fs authorize a client.fs_a / rwp
/home/abhinav/GSOC/PR/ceph/build/bin/ceph -c /home/abhinav/GSOC/PR/ceph/build/ceph.conf -k /home/abhinav/GSOC/PR/ceph/build/keyring fs authorize * client.fs / rwp
setting up user testid
2020-09-11T14:16:51.181+0530 7f503e09d0c0 -1 static int RGWDataChangesFIFO::exists(ceph::common::CephContext*, librados::v14_2_0::Rados*, const rgw_pool&, bool*, bool*): unable to open FIFO: default.rgw.log/data_log.0: (95) Operation not supported
2020-09-11T14:16:51.181+0530 7f503e09d0c0 -1 int RGWDataChangesLog::start(const RGWZone*, const RGWZoneParams&, RGWSI_Cls*, librados::v14_2_0::Rados*): Error when checking for existing FIFO datalog backend: (95) Operation not supported
2020-09-11T14:16:51.317+0530 7f503e09d0c0 -1 int RGWDataChangesLog::start(const RGWZone*, const RGWZoneParams&, RGWSI_Cls*, librados::v14_2_0::Rados*): Error when starting backend: Operation not supported
2020-09-11T14:16:51.317+0530 7f503e09d0c0 0 ERROR: failed to start datalog_rados service ((95) Operation not supported
2020-09-11T14:16:51.317+0530 7f503e09d0c0 0 ERROR: failed to init services (ret=(95) Operation not supported)
couldn't init storage provider
```
On Fri, Sep 11, 2020 at 11:29 AM Abhinav Singh <singhabhinav0796@xxxxxxxxx> wrote:
one update I m getting this message after this "setting up user testid"On Fri, Sep 11, 2020 at 3:40 AM Abhinav Singh <singhabhinav0796@xxxxxxxxx> wrote:Hello everyone,When I m starting this my rgw server I m getting this error "couldn't init storage provider"Command I used : "RGW=1 ../src/vstart.sh -d -n -x"OS:ubuntu : 18.04Ceph version : ceph version 15.1.0-1866-g053fd8f816 (053fd8f816ec0583fddfc63918dda521a3cf821e) octopus (rc)I see that error is happening inside "rgw_sal_rados.cc" in function `rgw::sal::RadosStore::init_storage_provider(...)` but I dont understand why it is happeningI tried to restart my system and do it is still not working,I dont understand which file should I share with you all to find my issue so you can ask me for specific file :)Thank You,Abhinav Singh
_______________________________________________ Dev mailing list -- dev@xxxxxxx To unsubscribe send an email to dev-leave@xxxxxxx