Hi All,
We are facing also following issue on openshift origin while we are creating pvc for pods. (atlease provide workaround to move further)
Failed to provision volume with StorageClass "glusterfs-storage": glusterfs: create volume err: error creating volume
Failed to provision volume with StorageClass "glusterfs-storage": glusterfs: create volume err: error creating volume Server busy. Retry operation later..
Please find heketidb dump and log
[negroni] Completed 429 Too Many Requests in 250.763µs
[negroni] Started POST /volumes
[heketi] WARNING 2019/01/23 12:07:49 operations in-flight (8) exceeds limit (8)
[negroni] Completed 429 Too Many Requests in 169.08µs
[negroni] Started DELETE /volumes/520bc5f4e1bfd029855a72f9ca7ebf6c
[negroni] Completed 404 Not Found in 148.125µs
[negroni] Started POST /volumes
[heketi] WARNING 2019/01/23 12:08:04 operations in-flight (8) exceeds limit (8)
[negroni] Completed 429 Too Many Requests in 496.624µs
[negroni] Started POST /volumes
[heketi] WARNING 2019/01/23 12:08:04 operations in-flight (8) exceeds limit (8)
[negroni] Completed 429 Too Many Requests in 101.673µs
[negroni] Started POST /volumes
[heketi] WARNING 2019/01/23 12:08:19 operations in-flight (8) exceeds limit (8)
[negroni] Completed 429 Too Many Requests in 209.681µs
[negroni] Started POST /volumes
[heketi] WARNING 2019/01/23 12:08:19 operations in-flight (8) exceeds limit (8)
[negroni] Completed 429 Too Many Requests in 103.595µs
[negroni] Started POST /volumes
[heketi] WARNING 2019/01/23 12:08:34 operations in-flight (8) exceeds limit (8)
[negroni] Completed 429 Too Many Requests in 297.594µs
[negroni] Started POST /volumes
[heketi] WARNING 2019/01/23 12:08:34 operations in-flight (8) exceeds limit (8)
[negroni] Completed 429 Too Many Requests in 96.75µs
[negroni] Started POST /volumes
[negroni] Started POST /volumes
[heketi] WARNING 2019/01/23 12:08:49 operations in-flight (8) exceeds limit (8)
[negroni] Completed 429 Too Many Requests in 477.007µs
[heketi] WARNING 2019/01/23 12:08:49 operations in-flight (8) exceeds limit (8)
[negroni] Completed 429 Too Many Requests in 165.38µs
[negroni] Started POST /volumes
[heketi] WARNING 2019/01/23 12:09:04 operations in-flight (8) exceeds limit (8)
[negroni] Completed 429 Too Many Requests in 488.253µs
[negroni] Started POST /volumes
[heketi] WARNING 2019/01/23 12:09:04 operations in-flight (8) exceeds limit (8)
[negroni] Completed 429 Too Many Requests in 171.836µs
[negroni] Started POST /volumes
[heketi] WARNING 2019/01/23 12:09:19 operations in-flight (8) exceeds limit (8)
[negroni] Completed 429 Too Many Requests in 208.59µs
[negroni] Started POST /volumes
[heketi] WARNING 2019/01/23 12:09:19 operations in-flight (8) exceeds limit (8)
[negroni] Completed 429 Too Many Requests in 125.141µs
[negroni] Started DELETE /volumes/99e87ecd0a816ac34ae5a04eabc1d606
[negroni] Completed 404 Not Found in 138.687µs
[negroni] Started POST /volumes
BR
Salam
=====-----=====-----=====
Notice: The information contained in this e-mail
message and/or attachments to it may contain
confidential or privileged information. If you are
not the intended recipient, any dissemination, use,
review, distribution, printing or copying of the
information contained in this e-mail message
and/or attachments to it are strictly prohibited. If
you have received this communication in error,
please notify us by reply e-mail or telephone and
immediately and permanently delete the message
and any attachments. Thank you
[root@master ~]# oc rsh heketi-storage-6-2bgwr
//localhost:8080 ; export HEKETI_CLI_USER=admin ; export HEKETI_CLI_SERVER=http:/
sh-4.4# heketi-cli db dump
{"clusterentries":{"e66ce2684350cb023c36d63b3d2bbf6f":{"Info":{"id":"e66ce2684350cb023c36d63b3d2bbf6f","nodes":["5f090e0aa757345d22ebdb15897bd68a","b72660bc8e28725f612d33bd675f8341","f829fc134bc9f983a2865cc0f46a31a6"],"volumes":["2e2e6b9cf174901b370ea79a266c651b","2e6ad6aa03f7fe219807cb135ca1c766","47c2af6849ac9ad7d7fbc897ae3ae80c","5101331e9d33a5d04adab92837b9d5ad","62905ab0e959e7663f4576501c0c9b69","63428c902b5d577a7739eb66a050a420","78d426889b7bd661f3d6c6f7814b6d4f","7e73769a754d7f9f6213ef3b4551af0e","83dcb62de26517790f155b51526cace5","88fbf36bf87106e549dca765f171cf69","937aa673e0d60126082bbd8c1589e383","9f04d82be19e2ea8ee80deb9098cd390","bc5012fd39c5b3ab958b9da4b0256d3a","c3b5f711128418ce9a7c68b514e34618","c5fa076446479cf414397591c0af1c7f","dbc601f7162df326784aea34c4ebe8f2","dcee4f7f020d47e58fb48612bbba19d1"],"block":true,"file":true,"blockvolumes":[]}}},"volumeentries":{"2e2e6b9cf174901b370ea79a266c651b":{"Info":{"size":10,"name":"vol_2e2e6b9cf174901b370ea79a266c651b","durability":{"type":"replicate","replicate":{"replica":3},"disperse":{}},"gid":2004,"snapshot":{"enable":false,"factor":1},"id":"2e2e6b9cf174901b370ea79a266c651b","cluster":"e66ce2684350cb023c36d63b3d2bbf6f","mount":{"glusterfs":{"hosts":["192.168.89.220","192.168.89.218","192.168.89.219"],"device":"192.168.89.220:vol_2e2e6b9cf174901b370ea79a266c651b","options":{"backup-volfile-servers":"192.168.89.218,192.168.89.219"}}},"blockinfo":{}},"Bricks":["62c4432fc7a6e4aeb11573e22eaf53d8","6f738c7cc82f66dbda8bc0b6d74883c0","c607b4aed1593917939ba85df5eefaae"],"GlusterVolumeOptions":null,"Pending":{"Id":""}},"2e6ad6aa03f7fe219807cb135ca1c766":{"Info":{"size":70,"name":"vol_2e6ad6aa03f7fe219807cb135ca1c766","durability":{"type":"replicate","replicate":{"replica":3},"disperse":{}},"gid":2007,"snapshot":{"enable":false,"factor":1},"id":"2e6ad6aa03f7fe219807cb135ca1c766","cluster":"e66ce2684350cb023c36d63b3d2bbf6f","mount":{"glusterfs":{"hosts":["192.168.89.220","192.168.89.218","192.168.89.219"],"device":"192.168.89.220:vol_2e6ad6aa03f7fe219807cb135ca1c766","options":{"backup-volfile-servers":"192.168.89.218,192.168.89.219"}}},"blockinfo":{}},"Bricks":["2fd555bc1cbd1b49ef04e7b88d34297b","980bd302213bbdd2d9aa09dbe8abf62b","fb2e1c7f95f9926166c585ca16ac2402"],"GlusterVolumeOptions":null,"Pending":{"Id":""}},"47c2af6849ac9ad7d7fbc897ae3ae80c":{"Info":{"size":50,"name":"vol_47c2af6849ac9ad7d7fbc897ae3ae80c","durability":{"type":"replicate","replicate":{"replica":3},"disperse":{}},"gid":2001,"snapshot":{"enable":false,"factor":1},"id":"47c2af6849ac9ad7d7fbc897ae3ae80c","cluster":"e66ce2684350cb023c36d63b3d2bbf6f","mount":{"glusterfs":{"hosts":["192.168.89.220","192.168.89.218","192.168.89.219"],"device":"192.168.89.220:vol_47c2af6849ac9ad7d7fbc897ae3ae80c","options":{"backup-volfile-servers":"192.168.89.218,192.168.89.219"}}},"blockinfo":{}},"Bricks":["025b1bf4bbabea839ef139c4453f9bcd","1f52101af46329a1d5e758156c8267e0","1f69ed2975c20811515f054a74669be6"],"GlusterVolumeOptions":null,"Pending":{"Id":""}},"5101331e9d33a5d04adab92837b9d5ad":{"Info":{"size":70,"name":"vol_5101331e9d33a5d04adab92837b9d5ad","durability":{"type":"replicate","replicate":{"replica":3},"disperse":{}},"gid":2006,"snapshot":{"enable":false,"factor":1},"id":"5101331e9d33a5d04adab92837b9d5ad","cluster":"e66ce2684350cb023c36d63b3d2bbf6f","mount":{"glusterfs":{"hosts":["192.168.89.220","192.168.89.218","192.168.89.219"],"device":"192.168.89.220:vol_5101331e9d33a5d04adab92837b9d5ad","options":{"backup-volfile-servers":"192.168.89.218,192.168.89.219"}}},"blockinfo":{}},"Bricks":["079acb5b4a0b0f3c45ed87f9c7def14b","17f8162940314babf2b7d75976890130","413e08918a40ce6806dd6c2711d923f3"],"GlusterVolumeOptions":null,"Pending":{"Id":""}},"62905ab0e959e7663f4576501c0c9b69":{"Info":{"size":1,"name":"vol_62905ab0e959e7663f4576501c0c9b69","durability":{"type":"replicate","replicate":{"replica":3},"disperse":{}},"gid":2002,"snapshot":{"enable":false,"factor":1},"id":"62905ab0e959e7663f4576501c0c9b69","cluster":"e66ce2684350cb023c36d63b3d2bbf6f","mount":{"glusterfs":{"hosts":["192.168.89.220","192.168.89.218","192.168.89.219"],"device":"192.168.89.220:vol_62905ab0e959e7663f4576501c0c9b69","options":{"backup-volfile-servers":"192.168.89.218,192.168.89.219"}}},"blockinfo":{}},"Bricks":["31ed93bf445cf1fd00d34660a80e1078","9684d9e56109e011585af33ecd8039cc","d343f2a8b3c19bda521f93d6327451f2"],"GlusterVolumeOptions":null,"Pending":{"Id":""}},"63428c902b5d577a7739eb66a050a420":{"Info":{"size":25,"name":"vol_63428c902b5d577a7739eb66a050a420","durability":{"type":"replicate","replicate":{"replica":3},"disperse":{}},"gid":2013,"snapshot":{"enable":false,"factor":1},"id":"63428c902b5d577a7739eb66a050a420","cluster":"e66ce2684350cb023c36d63b3d2bbf6f","mount":{"glusterfs":{"hosts":["192.168.89.220","192.168.89.218","192.168.89.219"],"device":"192.168.89.220:vol_63428c902b5d577a7739eb66a050a420","options":{"backup-volfile-servers":"192.168.89.218,192.168.89.219"}}},"blockinfo":{}},"Bricks":["3c16802735df66a463468ce8262ef0a6","586c05c5ce5bd7474cfb9a9cd190225c","85c1670a17f956abd0f3dd99cad53a7e"],"GlusterVolumeOptions":null,"Pending":{"Id":""}},"78d426889b7bd661f3d6c6f7814b6d4f":{"Info":{"size":70,"name":"vol_78d426889b7bd661f3d6c6f7814b6d4f","durability":{"type":"replicate","replicate":{"replica":3},"disperse":{}},"gid":2008,"snapshot":{"enable":false,"factor":1},"id":"78d426889b7bd661f3d6c6f7814b6d4f","cluster":"e66ce2684350cb023c36d63b3d2bbf6f","mount":{"glusterfs":{"hosts":["192.168.89.220","192.168.89.218","192.168.89.219"],"device":"192.168.89.220:vol_78d426889b7bd661f3d6c6f7814b6d4f","options":{"backup-volfile-servers":"192.168.89.218,192.168.89.219"}}},"blockinfo":{}},"Bricks":["28b3599a579db6e987f46ab0120f9d8c","8dba95329e797beb06eca079b9ae6a4b","d3230ee5de8fe8300f47eb4bcea6f2e2"],"GlusterVolumeOptions":null,"Pending":{"Id":""}},"7e73769a754d7f9f6213ef3b4551af0e":{"Info":{"size":5,"name":"vol_7e73769a754d7f9f6213ef3b4551af0e","durability":{"type":"replicate","replicate":{"replica":3},"disperse":{"data":4,"redundancy":2}},"snapshot":{"enable":false,"factor":1},"id":"7e73769a754d7f9f6213ef3b4551af0e","cluster":"e66ce2684350cb023c36d63b3d2bbf6f","mount":{"glusterfs":{"hosts":["192.168.89.220","192.168.89.218","192.168.89.219"],"device":"192.168.89.220:vol_7e73769a754d7f9f6213ef3b4551af0e","options":{"backup-volfile-servers":"192.168.89.218,192.168.89.219"}}},"blockinfo":{}},"Bricks":["774b52d13a2cf996784181cf6d7db93c","84e395f5464cd25dce66b8e60f395982","c2fe3a63636192e85d8adeefff18c92b"],"GlusterVolumeOptions":null,"Pending":{"Id":""}},"83dcb62de26517790f155b51526cace5":{"Info":{"size":200,"name":"glusterfs-registry-volume","durability":{"type":"replicate","replicate":{"replica":3},"disperse":{"data":4,"redundancy":2}},"snapshot":{"enable":false,"factor":1},"id":"83dcb62de26517790f155b51526cace5","cluster":"e66ce2684350cb023c36d63b3d2bbf6f","mount":{"glusterfs":{"hosts":["192.168.89.220","192.168.89.218","192.168.89.219"],"device":"192.168.89.220:glusterfs-registry-volume","options":{"backup-volfile-servers":"192.168.89.218,192.168.89.219"}}},"blockinfo":{}},"Bricks":["7ef53adf2bce0e240229941ac7f43661","b22c3ab4d7bbc420e05340e8e187bcf5","b85460c3ba53bb7db5ab712f7b684c51"],"GlusterVolumeOptions":null,"Pending":{"Id":""}},"88fbf36bf87106e549dca765f171cf69":{"Info":{"size":5,"name":"vol_88fbf36bf87106e549dca765f171cf69","durability":{"type":"replicate","replicate":{"replica":3},"disperse":{"data":4,"redundancy":2}},"snapshot":{"enable":false,"factor":1},"id":"88fbf36bf87106e549dca765f171cf69","cluster":"e66ce2684350cb023c36d63b3d2bbf6f","mount":{"glusterfs":{"hosts":["192.168.89.220","192.168.89.218","192.168.89.219"],"device":"192.168.89.220:vol_88fbf36bf87106e549dca765f171cf69","options":{"backup-volfile-servers":"192.168.89.218,192.168.89.219"}}},"blockinfo":{}},"Bricks":["22982c432dd089512ab2bde1e0663f9a","e26c7a93f2a7b6d6455ab5fa7615dfb1","ef253aa4b2756fc293aa101c2eb743cb"],"GlusterVolumeOptions":null,"Pending":{"Id":""}},"937aa673e0d60126082bbd8c1589e383":{"Info":{"size":20,"name":"vol_937aa673e0d60126082bbd8c1589e383","durability":{"type":"replicate","replicate":{"replica":3},"disperse":{}},"gid":2003,"snapshot":{"enable":false,"factor":1},"id":"937aa673e0d60126082bbd8c1589e383","cluster":"e66ce2684350cb023c36d63b3d2bbf6f","mount":{"glusterfs":{"hosts":["192.168.89.220","192.168.89.218","192.168.89.219"],"device":"192.168.89.220:vol_937aa673e0d60126082bbd8c1589e383","options":{"backup-volfile-servers":"192.168.89.218,192.168.89.219"}}},"blockinfo":{}},"Bricks":["275d68b56cfc42c8b46aa90eb227eb25","5f22e9906d3566116ec474bb618213e5","bb29a705b3f39522b131277d64761033"],"GlusterVolumeOptions":null,"Pending":{"Id":""}},"9f04d82be19e2ea8ee80deb9098cd390":{"Info":{"size":20,"name":"vol_9f04d82be19e2ea8ee80deb9098cd390","durability":{"type":"replicate","replicate":{"replica":3},"disperse":{}},"gid":2014,"snapshot":{"enable":false,"factor":1},"id":"9f04d82be19e2ea8ee80deb9098cd390","cluster":"e66ce2684350cb023c36d63b3d2bbf6f","mount":{"glusterfs":{"hosts":["192.168.89.220","192.168.89.218","192.168.89.219"],"device":"192.168.89.220:vol_9f04d82be19e2ea8ee80deb9098cd390","options":{"backup-volfile-servers":"192.168.89.218,192.168.89.219"}}},"blockinfo":{}},"Bricks":["0bd819f4969e7adbba32e2c77afc202c","281606b3024ac98416fedee94528e3c7","9603a0a95036d6362ae2b38bdbb40428"],"GlusterVolumeOptions":null,"Pending":{"Id":""}},"bc5012fd39c5b3ab958b9da4b0256d3a":{"Info":{"size":4,"name":"vol_bc5012fd39c5b3ab958b9da4b0256d3a","durability":{"type":"replicate","replicate":{"replica":3},"disperse":{}},"gid":2009,"snapshot":{"enable":false,"factor":1},"id":"bc5012fd39c5b3ab958b9da4b0256d3a","cluster":"e66ce2684350cb023c36d63b3d2bbf6f","mount":{"glusterfs":{"hosts":["192.168.89.220","192.168.89.218","192.168.89.219"],"device":"192.168.89.220:vol_bc5012fd39c5b3ab958b9da4b0256d3a","options":{"backup-volfile-servers":"192.168.89.218,192.168.89.219"}}},"blockinfo":{}},"Bricks":["08599e162e105063a8e2efec868d2a76","c4b582ef0712e8c2d750c97ff443e47c","fded2411a02e03c7e0219eb9a2b08c9a"],"GlusterVolumeOptions":null,"Pending":{"Id":""}},"c3b5f711128418ce9a7c68b514e34618":{"Info":{"size":2,"name":"heketidbstorage","durability":{"type":"replicate","replicate":{"replica":3},"disperse":{}},"snapshot":{"enable":false,"factor":1},"id":"c3b5f711128418ce9a7c68b514e34618","cluster":"e66ce2684350cb023c36d63b3d2bbf6f","mount":{"glusterfs":{"hosts":["192.168.89.220","192.168.89.218","192.168.89.219"],"device":"192.168.89.220:heketidbstorage","options":{"backup-volfile-servers":"192.168.89.218,192.168.89.219"}}},"blockinfo":{}},"Bricks":["2b6ed29607d007a75f7a7c1bf0da7bae","79e211c7bee7d61bfdabb0add8e42b33","ac26b8e8306405623200599d05e87fe2"],"GlusterVolumeOptions":null,"Pending":{"Id":""}},"c5fa076446479cf414397591c0af1c7f":{"Info":{"size":10,"name":"vol_c5fa076446479cf414397591c0af1c7f","durability":{"type":"replicate","replicate":{"replica":3},"disperse":{}},"gid":2005,"snapshot":{"enable":false,"factor":1},"id":"c5fa076446479cf414397591c0af1c7f","cluster":"e66ce2684350cb023c36d63b3d2bbf6f","mount":{"glusterfs":{"hosts":["192.168.89.220","192.168.89.218","192.168.89.219"],"device":"192.168.89.220:vol_c5fa076446479cf414397591c0af1c7f","options":{"backup-volfile-servers":"192.168.89.218,192.168.89.219"}}},"blockinfo":{}},"Bricks":["93eb3caff9f179d522b5498b515bf991","ac7b22198b6424785edd32d6f0a078d2","ae2fa0410160c9fb489af8e5ad063ac3"],"GlusterVolumeOptions":null,"Pending":{"Id":""}},"dbc601f7162df326784aea34c4ebe8f2":{"Info":{"size":20,"name":"vol_dbc601f7162df326784aea34c4ebe8f2","durability":{"type":"replicate","replicate":{"replica":3},"disperse":{}},"gid":2000,"snapshot":{"enable":false,"factor":1},"id":"dbc601f7162df326784aea34c4ebe8f2","cluster":"e66ce2684350cb023c36d63b3d2bbf6f","mount":{"glusterfs":{"hosts":["192.168.89.220","192.168.89.218","192.168.89.219"],"device":"192.168.89.220:vol_dbc601f7162df326784aea34c4ebe8f2","options":{"backup-volfile-servers":"192.168.89.218,192.168.89.219"}}},"blockinfo":{}},"Bricks":["4fa8a73223d69de183d8e58618722084","c5d298199cfbc91ebf013707709bdbb6","ff62a0d3e0fe5c53c3725235d2832893"],"GlusterVolumeOptions":null,"Pending":{"Id":""}},"dcee4f7f020d47e58fb48612bbba19d1":{"Info":{"size":25,"name":"vol_dcee4f7f020d47e58fb48612bbba19d1","durability":{"type":"replicate","replicate":{"replica":3},"disperse":{}},"gid":2010,"snapshot":{"enable":false,"factor":1},"id":"dcee4f7f020d47e58fb48612bbba19d1","cluster":"e66ce2684350cb023c36d63b3d2bbf6f","mount":{"glusterfs":{"hosts":["192.168.89.220","192.168.89.218","192.168.89.219"],"device":"192.168.89.220:vol_dcee4f7f020d47e58fb48612bbba19d1","options":{"backup-volfile-servers":"192.168.89.218,192.168.89.219"}}},"blockinfo":{}},"Bricks":["16de636f8337f47a87996fca50313260","800e2a99110af93ca69bd950c9ea7ef8","fd6a1a83004a11c0007e9392913aea8f"],"GlusterVolumeOptions":null,"Pending":{"Id":""}}},"brickentries":{"025b1bf4bbabea839ef139c4453f9bcd":{"Info":{"id":"025b1bf4bbabea839ef139c4453f9bcd","path":"/var/lib/heketi/mounts/vg_5054cf122b4f54021cb9d67752d23c1d/brick_025b1bf4bbabea839ef139c4453f9bcd/brick","device":"5054cf122b4f54021cb9d67752d23c1d","node":"5f090e0aa757345d22ebdb15897bd68a","volume":"47c2af6849ac9ad7d7fbc897ae3ae80c","size":52428800},"TpSize":52428800,"PoolMetadataSize":262144,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"079acb5b4a0b0f3c45ed87f9c7def14b":{"Info":{"id":"079acb5b4a0b0f3c45ed87f9c7def14b","path":"/var/lib/heketi/mounts/vg_9c007bfbe66a2522ad5dd851bb18d03a/brick_079acb5b4a0b0f3c45ed87f9c7def14b/brick","device":"9c007bfbe66a2522ad5dd851bb18d03a","node":"5f090e0aa757345d22ebdb15897bd68a","volume":"5101331e9d33a5d04adab92837b9d5ad","size":73400320},"TpSize":73400320,"PoolMetadataSize":368640,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"08599e162e105063a8e2efec868d2a76":{"Info":{"id":"08599e162e105063a8e2efec868d2a76","path":"/var/lib/heketi/mounts/vg_cea64ccf8a96a13dacc1cb3ac3ea13de/brick_08599e162e105063a8e2efec868d2a76/brick","device":"cea64ccf8a96a13dacc1cb3ac3ea13de","node":"b72660bc8e28725f612d33bd675f8341","volume":"bc5012fd39c5b3ab958b9da4b0256d3a","size":4194304},"TpSize":4194304,"PoolMetadataSize":24576,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"0bd819f4969e7adbba32e2c77afc202c":{"Info":{"id":"0bd819f4969e7adbba32e2c77afc202c","path":"/var/lib/heketi/mounts/vg_018e77166eb3b38a0c250eb8faede2f7/brick_0bd819f4969e7adbba32e2c77afc202c/brick","device":"018e77166eb3b38a0c250eb8faede2f7","node":"5f090e0aa757345d22ebdb15897bd68a","volume":"9f04d82be19e2ea8ee80deb9098cd390","size":20971520},"TpSize":20971520,"PoolMetadataSize":106496,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"16de636f8337f47a87996fca50313260":{"Info":{"id":"16de636f8337f47a87996fca50313260","path":"/var/lib/heketi/mounts/vg_018e77166eb3b38a0c250eb8faede2f7/brick_16de636f8337f47a87996fca50313260/brick","device":"018e77166eb3b38a0c250eb8faede2f7","node":"5f090e0aa757345d22ebdb15897bd68a","volume":"dcee4f7f020d47e58fb48612bbba19d1","size":26214400},"TpSize":26214400,"PoolMetadataSize":131072,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"17f8162940314babf2b7d75976890130":{"Info":{"id":"17f8162940314babf2b7d75976890130","path":"/var/lib/heketi/mounts/vg_127a3d697690febddfa2a41b3a820bde/brick_17f8162940314babf2b7d75976890130/brick","device":"127a3d697690febddfa2a41b3a820bde","node":"b72660bc8e28725f612d33bd675f8341","volume":"5101331e9d33a5d04adab92837b9d5ad","size":73400320},"TpSize":73400320,"PoolMetadataSize":368640,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"1f52101af46329a1d5e758156c8267e0":{"Info":{"id":"1f52101af46329a1d5e758156c8267e0","path":"/var/lib/heketi/mounts/vg_cea64ccf8a96a13dacc1cb3ac3ea13de/brick_1f52101af46329a1d5e758156c8267e0/brick","device":"cea64ccf8a96a13dacc1cb3ac3ea13de","node":"b72660bc8e28725f612d33bd675f8341","volume":"47c2af6849ac9ad7d7fbc897ae3ae80c","size":52428800},"TpSize":52428800,"PoolMetadataSize":262144,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"1f69ed2975c20811515f054a74669be6":{"Info":{"id":"1f69ed2975c20811515f054a74669be6","path":"/var/lib/heketi/mounts/vg_339b50a1c21d42f39d5568dcfbbf8844/brick_1f69ed2975c20811515f054a74669be6/brick","device":"339b50a1c21d42f39d5568dcfbbf8844","node":"f829fc134bc9f983a2865cc0f46a31a6","volume":"47c2af6849ac9ad7d7fbc897ae3ae80c","size":52428800},"TpSize":52428800,"PoolMetadataSize":262144,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"22982c432dd089512ab2bde1e0663f9a":{"Info":{"id":"22982c432dd089512ab2bde1e0663f9a","path":"/var/lib/heketi/mounts/vg_9c007bfbe66a2522ad5dd851bb18d03a/brick_22982c432dd089512ab2bde1e0663f9a/brick","device":"9c007bfbe66a2522ad5dd851bb18d03a","node":"5f090e0aa757345d22ebdb15897bd68a","volume":"88fbf36bf87106e549dca765f171cf69","size":5242880},"TpSize":5242880,"PoolMetadataSize":28672,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"275d68b56cfc42c8b46aa90eb227eb25":{"Info":{"id":"275d68b56cfc42c8b46aa90eb227eb25","path":"/var/lib/heketi/mounts/vg_018e77166eb3b38a0c250eb8faede2f7/brick_275d68b56cfc42c8b46aa90eb227eb25/brick","device":"018e77166eb3b38a0c250eb8faede2f7","node":"5f090e0aa757345d22ebdb15897bd68a","volume":"937aa673e0d60126082bbd8c1589e383","size":20971520},"TpSize":20971520,"PoolMetadataSize":106496,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"281606b3024ac98416fedee94528e3c7":{"Info":{"id":"281606b3024ac98416fedee94528e3c7","path":"/var/lib/heketi/mounts/vg_88c314d3366471a0dd89ca4d5a3c8ef8/brick_281606b3024ac98416fedee94528e3c7/brick","device":"88c314d3366471a0dd89ca4d5a3c8ef8","node":"b72660bc8e28725f612d33bd675f8341","volume":"9f04d82be19e2ea8ee80deb9098cd390","size":20971520},"TpSize":20971520,"PoolMetadataSize":106496,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"28b3599a579db6e987f46ab0120f9d8c":{"Info":{"id":"28b3599a579db6e987f46ab0120f9d8c","path":"/var/lib/heketi/mounts/vg_06f12f2c23a257117083a6d86d6f4087/brick_28b3599a579db6e987f46ab0120f9d8c/brick","device":"06f12f2c23a257117083a6d86d6f4087","node":"f829fc134bc9f983a2865cc0f46a31a6","volume":"78d426889b7bd661f3d6c6f7814b6d4f","size":73400320},"TpSize":73400320,"PoolMetadataSize":368640,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"2b6ed29607d007a75f7a7c1bf0da7bae":{"Info":{"id":"2b6ed29607d007a75f7a7c1bf0da7bae","path":"/var/lib/heketi/mounts/vg_018e77166eb3b38a0c250eb8faede2f7/brick_2b6ed29607d007a75f7a7c1bf0da7bae/brick","device":"018e77166eb3b38a0c250eb8faede2f7","node":"5f090e0aa757345d22ebdb15897bd68a","volume":"c3b5f711128418ce9a7c68b514e34618","size":2097152},"TpSize":2097152,"PoolMetadataSize":12288,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"2fd555bc1cbd1b49ef04e7b88d34297b":{"Info":{"id":"2fd555bc1cbd1b49ef04e7b88d34297b","path":"/var/lib/heketi/mounts/vg_9c007bfbe66a2522ad5dd851bb18d03a/brick_2fd555bc1cbd1b49ef04e7b88d34297b/brick","device":"9c007bfbe66a2522ad5dd851bb18d03a","node":"5f090e0aa757345d22ebdb15897bd68a","volume":"2e6ad6aa03f7fe219807cb135ca1c766","size":73400320},"TpSize":73400320,"PoolMetadataSize":368640,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"31ed93bf445cf1fd00d34660a80e1078":{"Info":{"id":"31ed93bf445cf1fd00d34660a80e1078","path":"/var/lib/heketi/mounts/vg_cea64ccf8a96a13dacc1cb3ac3ea13de/brick_31ed93bf445cf1fd00d34660a80e1078/brick","device":"cea64ccf8a96a13dacc1cb3ac3ea13de","node":"b72660bc8e28725f612d33bd675f8341","volume":"62905ab0e959e7663f4576501c0c9b69","size":1048576},"TpSize":1048576,"PoolMetadataSize":8192,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"3c16802735df66a463468ce8262ef0a6":{"Info":{"id":"3c16802735df66a463468ce8262ef0a6","path":"/var/lib/heketi/mounts/vg_ae7c5467f294a953bfb274c1e6afc26d/brick_3c16802735df66a463468ce8262ef0a6/brick","device":"ae7c5467f294a953bfb274c1e6afc26d","node":"f829fc134bc9f983a2865cc0f46a31a6","volume":"63428c902b5d577a7739eb66a050a420","size":26214400},"TpSize":26214400,"PoolMetadataSize":131072,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"413e08918a40ce6806dd6c2711d923f3":{"Info":{"id":"413e08918a40ce6806dd6c2711d923f3","path":"/var/lib/heketi/mounts/vg_06f12f2c23a257117083a6d86d6f4087/brick_413e08918a40ce6806dd6c2711d923f3/brick","device":"06f12f2c23a257117083a6d86d6f4087","node":"f829fc134bc9f983a2865cc0f46a31a6","volume":"5101331e9d33a5d04adab92837b9d5ad","size":73400320},"TpSize":73400320,"PoolMetadataSize":368640,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"4fa8a73223d69de183d8e58618722084":{"Info":{"id":"4fa8a73223d69de183d8e58618722084","path":"/var/lib/heketi/mounts/vg_5054cf122b4f54021cb9d67752d23c1d/brick_4fa8a73223d69de183d8e58618722084/brick","device":"5054cf122b4f54021cb9d67752d23c1d","node":"5f090e0aa757345d22ebdb15897bd68a","volume":"dbc601f7162df326784aea34c4ebe8f2","size":20971520},"TpSize":20971520,"PoolMetadataSize":106496,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"586c05c5ce5bd7474cfb9a9cd190225c":{"Info":{"id":"586c05c5ce5bd7474cfb9a9cd190225c","path":"/var/lib/heketi/mounts/vg_88c314d3366471a0dd89ca4d5a3c8ef8/brick_586c05c5ce5bd7474cfb9a9cd190225c/brick","device":"88c314d3366471a0dd89ca4d5a3c8ef8","node":"b72660bc8e28725f612d33bd675f8341","volume":"63428c902b5d577a7739eb66a050a420","size":26214400},"TpSize":26214400,"PoolMetadataSize":131072,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"5f22e9906d3566116ec474bb618213e5":{"Info":{"id":"5f22e9906d3566116ec474bb618213e5","path":"/var/lib/heketi/mounts/vg_127a3d697690febddfa2a41b3a820bde/brick_5f22e9906d3566116ec474bb618213e5/brick","device":"127a3d697690febddfa2a41b3a820bde","node":"b72660bc8e28725f612d33bd675f8341","volume":"937aa673e0d60126082bbd8c1589e383","size":20971520},"TpSize":20971520,"PoolMetadataSize":106496,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"62c4432fc7a6e4aeb11573e22eaf53d8":{"Info":{"id":"62c4432fc7a6e4aeb11573e22eaf53d8","path":"/var/lib/heketi/mounts/vg_25eb42cb007aab028afd0ee42b72bae9/brick_62c4432fc7a6e4aeb11573e22eaf53d8/brick","device":"25eb42cb007aab028afd0ee42b72bae9","node":"5f090e0aa757345d22ebdb15897bd68a","volume":"2e2e6b9cf174901b370ea79a266c651b","size":10485760},"TpSize":10485760,"PoolMetadataSize":53248,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"6f738c7cc82f66dbda8bc0b6d74883c0":{"Info":{"id":"6f738c7cc82f66dbda8bc0b6d74883c0","path":"/var/lib/heketi/mounts/vg_88c314d3366471a0dd89ca4d5a3c8ef8/brick_6f738c7cc82f66dbda8bc0b6d74883c0/brick","device":"88c314d3366471a0dd89ca4d5a3c8ef8","node":"b72660bc8e28725f612d33bd675f8341","volume":"2e2e6b9cf174901b370ea79a266c651b","size":10485760},"TpSize":10485760,"PoolMetadataSize":53248,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"774b52d13a2cf996784181cf6d7db93c":{"Info":{"id":"774b52d13a2cf996784181cf6d7db93c","path":"/var/lib/heketi/mounts/vg_3343a86e75865dd02b054fb268781815/brick_774b52d13a2cf996784181cf6d7db93c/brick","device":"3343a86e75865dd02b054fb268781815","node":"f829fc134bc9f983a2865cc0f46a31a6","volume":"7e73769a754d7f9f6213ef3b4551af0e","size":5242880},"TpSize":5242880,"PoolMetadataSize":28672,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"79e211c7bee7d61bfdabb0add8e42b33":{"Info":{"id":"79e211c7bee7d61bfdabb0add8e42b33","path":"/var/lib/heketi/mounts/vg_127a3d697690febddfa2a41b3a820bde/brick_79e211c7bee7d61bfdabb0add8e42b33/brick","device":"127a3d697690febddfa2a41b3a820bde","node":"b72660bc8e28725f612d33bd675f8341","volume":"c3b5f711128418ce9a7c68b514e34618","size":2097152},"TpSize":2097152,"PoolMetadataSize":12288,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"7ef53adf2bce0e240229941ac7f43661":{"Info":{"id":"7ef53adf2bce0e240229941ac7f43661","path":"/var/lib/heketi/mounts/vg_cea64ccf8a96a13dacc1cb3ac3ea13de/brick_7ef53adf2bce0e240229941ac7f43661/brick","device":"cea64ccf8a96a13dacc1cb3ac3ea13de","node":"b72660bc8e28725f612d33bd675f8341","volume":"83dcb62de26517790f155b51526cace5","size":209715200},"TpSize":209715200,"PoolMetadataSize":1048576,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"800e2a99110af93ca69bd950c9ea7ef8":{"Info":{"id":"800e2a99110af93ca69bd950c9ea7ef8","path":"/var/lib/heketi/mounts/vg_e7e08adf3c8cd3dee08dd006b1596074/brick_800e2a99110af93ca69bd950c9ea7ef8/brick","device":"e7e08adf3c8cd3dee08dd006b1596074","node":"b72660bc8e28725f612d33bd675f8341","volume":"dcee4f7f020d47e58fb48612bbba19d1","size":26214400},"TpSize":26214400,"PoolMetadataSize":131072,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"84e395f5464cd25dce66b8e60f395982":{"Info":{"id":"84e395f5464cd25dce66b8e60f395982","path":"/var/lib/heketi/mounts/vg_88c314d3366471a0dd89ca4d5a3c8ef8/brick_84e395f5464cd25dce66b8e60f395982/brick","device":"88c314d3366471a0dd89ca4d5a3c8ef8","node":"b72660bc8e28725f612d33bd675f8341","volume":"7e73769a754d7f9f6213ef3b4551af0e","size":5242880},"TpSize":5242880,"PoolMetadataSize":28672,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"85c1670a17f956abd0f3dd99cad53a7e":{"Info":{"id":"85c1670a17f956abd0f3dd99cad53a7e","path":"/var/lib/heketi/mounts/vg_25eb42cb007aab028afd0ee42b72bae9/brick_85c1670a17f956abd0f3dd99cad53a7e/brick","device":"25eb42cb007aab028afd0ee42b72bae9","node":"5f090e0aa757345d22ebdb15897bd68a","volume":"63428c902b5d577a7739eb66a050a420","size":26214400},"TpSize":26214400,"PoolMetadataSize":131072,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"8dba95329e797beb06eca079b9ae6a4b":{"Info":{"id":"8dba95329e797beb06eca079b9ae6a4b","path":"/var/lib/heketi/mounts/vg_9c007bfbe66a2522ad5dd851bb18d03a/brick_8dba95329e797beb06eca079b9ae6a4b/brick","device":"9c007bfbe66a2522ad5dd851bb18d03a","node":"5f090e0aa757345d22ebdb15897bd68a","volume":"78d426889b7bd661f3d6c6f7814b6d4f","size":73400320},"TpSize":73400320,"PoolMetadataSize":368640,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"93eb3caff9f179d522b5498b515bf991":{"Info":{"id":"93eb3caff9f179d522b5498b515bf991","path":"/var/lib/heketi/mounts/vg_ae7c5467f294a953bfb274c1e6afc26d/brick_93eb3caff9f179d522b5498b515bf991/brick","device":"ae7c5467f294a953bfb274c1e6afc26d","node":"f829fc134bc9f983a2865cc0f46a31a6","volume":"c5fa076446479cf414397591c0af1c7f","size":10485760},"TpSize":10485760,"PoolMetadataSize":53248,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"9603a0a95036d6362ae2b38bdbb40428":{"Info":{"id":"9603a0a95036d6362ae2b38bdbb40428","path":"/var/lib/heketi/mounts/vg_3343a86e75865dd02b054fb268781815/brick_9603a0a95036d6362ae2b38bdbb40428/brick","device":"3343a86e75865dd02b054fb268781815","node":"f829fc134bc9f983a2865cc0f46a31a6","volume":"9f04d82be19e2ea8ee80deb9098cd390","size":20971520},"TpSize":20971520,"PoolMetadataSize":106496,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"9684d9e56109e011585af33ecd8039cc":{"Info":{"id":"9684d9e56109e011585af33ecd8039cc","path":"/var/lib/heketi/mounts/vg_5054cf122b4f54021cb9d67752d23c1d/brick_9684d9e56109e011585af33ecd8039cc/brick","device":"5054cf122b4f54021cb9d67752d23c1d","node":"5f090e0aa757345d22ebdb15897bd68a","volume":"62905ab0e959e7663f4576501c0c9b69","size":1048576},"TpSize":1048576,"PoolMetadataSize":8192,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"980bd302213bbdd2d9aa09dbe8abf62b":{"Info":{"id":"980bd302213bbdd2d9aa09dbe8abf62b","path":"/var/lib/heketi/mounts/vg_e7e08adf3c8cd3dee08dd006b1596074/brick_980bd302213bbdd2d9aa09dbe8abf62b/brick","device":"e7e08adf3c8cd3dee08dd006b1596074","node":"b72660bc8e28725f612d33bd675f8341","volume":"2e6ad6aa03f7fe219807cb135ca1c766","size":73400320},"TpSize":73400320,"PoolMetadataSize":368640,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"ac26b8e8306405623200599d05e87fe2":{"Info":{"id":"ac26b8e8306405623200599d05e87fe2","path":"/var/lib/heketi/mounts/vg_06f12f2c23a257117083a6d86d6f4087/brick_ac26b8e8306405623200599d05e87fe2/brick","device":"06f12f2c23a257117083a6d86d6f4087","node":"f829fc134bc9f983a2865cc0f46a31a6","volume":"c3b5f711128418ce9a7c68b514e34618","size":2097152},"TpSize":2097152,"PoolMetadataSize":12288,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"ac7b22198b6424785edd32d6f0a078d2":{"Info":{"id":"ac7b22198b6424785edd32d6f0a078d2","path":"/var/lib/heketi/mounts/vg_25eb42cb007aab028afd0ee42b72bae9/brick_ac7b22198b6424785edd32d6f0a078d2/brick","device":"25eb42cb007aab028afd0ee42b72bae9","node":"5f090e0aa757345d22ebdb15897bd68a","volume":"c5fa076446479cf414397591c0af1c7f","size":10485760},"TpSize":10485760,"PoolMetadataSize":53248,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"ae2fa0410160c9fb489af8e5ad063ac3":{"Info":{"id":"ae2fa0410160c9fb489af8e5ad063ac3","path":"/var/lib/heketi/mounts/vg_cea64ccf8a96a13dacc1cb3ac3ea13de/brick_ae2fa0410160c9fb489af8e5ad063ac3/brick","device":"cea64ccf8a96a13dacc1cb3ac3ea13de","node":"b72660bc8e28725f612d33bd675f8341","volume":"c5fa076446479cf414397591c0af1c7f","size":10485760},"TpSize":10485760,"PoolMetadataSize":53248,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"b22c3ab4d7bbc420e05340e8e187bcf5":{"Info":{"id":"b22c3ab4d7bbc420e05340e8e187bcf5","path":"/var/lib/heketi/mounts/vg_5054cf122b4f54021cb9d67752d23c1d/brick_b22c3ab4d7bbc420e05340e8e187bcf5/brick","device":"5054cf122b4f54021cb9d67752d23c1d","node":"5f090e0aa757345d22ebdb15897bd68a","volume":"83dcb62de26517790f155b51526cace5","size":209715200},"TpSize":209715200,"PoolMetadataSize":1048576,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"b85460c3ba53bb7db5ab712f7b684c51":{"Info":{"id":"b85460c3ba53bb7db5ab712f7b684c51","path":"/var/lib/heketi/mounts/vg_339b50a1c21d42f39d5568dcfbbf8844/brick_b85460c3ba53bb7db5ab712f7b684c51/brick","device":"339b50a1c21d42f39d5568dcfbbf8844","node":"f829fc134bc9f983a2865cc0f46a31a6","volume":"83dcb62de26517790f155b51526cace5","size":209715200},"TpSize":209715200,"PoolMetadataSize":1048576,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"bb29a705b3f39522b131277d64761033":{"Info":{"id":"bb29a705b3f39522b131277d64761033","path":"/var/lib/heketi/mounts/vg_3343a86e75865dd02b054fb268781815/brick_bb29a705b3f39522b131277d64761033/brick","device":"3343a86e75865dd02b054fb268781815","node":"f829fc134bc9f983a2865cc0f46a31a6","volume":"937aa673e0d60126082bbd8c1589e383","size":20971520},"TpSize":20971520,"PoolMetadataSize":106496,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"c2fe3a63636192e85d8adeefff18c92b":{"Info":{"id":"c2fe3a63636192e85d8adeefff18c92b","path":"/var/lib/heketi/mounts/vg_018e77166eb3b38a0c250eb8faede2f7/brick_c2fe3a63636192e85d8adeefff18c92b/brick","device":"018e77166eb3b38a0c250eb8faede2f7","node":"5f090e0aa757345d22ebdb15897bd68a","volume":"7e73769a754d7f9f6213ef3b4551af0e","size":5242880},"TpSize":5242880,"PoolMetadataSize":28672,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"c4b582ef0712e8c2d750c97ff443e47c":{"Info":{"id":"c4b582ef0712e8c2d750c97ff443e47c","path":"/var/lib/heketi/mounts/vg_ae7c5467f294a953bfb274c1e6afc26d/brick_c4b582ef0712e8c2d750c97ff443e47c/brick","device":"ae7c5467f294a953bfb274c1e6afc26d","node":"f829fc134bc9f983a2865cc0f46a31a6","volume":"bc5012fd39c5b3ab958b9da4b0256d3a","size":4194304},"TpSize":4194304,"PoolMetadataSize":24576,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"c5d298199cfbc91ebf013707709bdbb6":{"Info":{"id":"c5d298199cfbc91ebf013707709bdbb6","path":"/var/lib/heketi/mounts/vg_ae7c5467f294a953bfb274c1e6afc26d/brick_c5d298199cfbc91ebf013707709bdbb6/brick","device":"ae7c5467f294a953bfb274c1e6afc26d","node":"f829fc134bc9f983a2865cc0f46a31a6","volume":"dbc601f7162df326784aea34c4ebe8f2","size":20971520},"TpSize":20971520,"PoolMetadataSize":106496,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"c607b4aed1593917939ba85df5eefaae":{"Info":{"id":"c607b4aed1593917939ba85df5eefaae","path":"/var/lib/heketi/mounts/vg_3343a86e75865dd02b054fb268781815/brick_c607b4aed1593917939ba85df5eefaae/brick","device":"3343a86e75865dd02b054fb268781815","node":"f829fc134bc9f983a2865cc0f46a31a6","volume":"2e2e6b9cf174901b370ea79a266c651b","size":10485760},"TpSize":10485760,"PoolMetadataSize":53248,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"d3230ee5de8fe8300f47eb4bcea6f2e2":{"Info":{"id":"d3230ee5de8fe8300f47eb4bcea6f2e2","path":"/var/lib/heketi/mounts/vg_127a3d697690febddfa2a41b3a820bde/brick_d3230ee5de8fe8300f47eb4bcea6f2e2/brick","device":"127a3d697690febddfa2a41b3a820bde","node":"b72660bc8e28725f612d33bd675f8341","volume":"78d426889b7bd661f3d6c6f7814b6d4f","size":73400320},"TpSize":73400320,"PoolMetadataSize":368640,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"d343f2a8b3c19bda521f93d6327451f2":{"Info":{"id":"d343f2a8b3c19bda521f93d6327451f2","path":"/var/lib/heketi/mounts/vg_339b50a1c21d42f39d5568dcfbbf8844/brick_d343f2a8b3c19bda521f93d6327451f2/brick","device":"339b50a1c21d42f39d5568dcfbbf8844","node":"f829fc134bc9f983a2865cc0f46a31a6","volume":"62905ab0e959e7663f4576501c0c9b69","size":1048576},"TpSize":1048576,"PoolMetadataSize":8192,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"e26c7a93f2a7b6d6455ab5fa7615dfb1":{"Info":{"id":"e26c7a93f2a7b6d6455ab5fa7615dfb1","path":"/var/lib/heketi/mounts/vg_ae7c5467f294a953bfb274c1e6afc26d/brick_e26c7a93f2a7b6d6455ab5fa7615dfb1/brick","device":"ae7c5467f294a953bfb274c1e6afc26d","node":"f829fc134bc9f983a2865cc0f46a31a6","volume":"88fbf36bf87106e549dca765f171cf69","size":5242880},"TpSize":5242880,"PoolMetadataSize":28672,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"ef253aa4b2756fc293aa101c2eb743cb":{"Info":{"id":"ef253aa4b2756fc293aa101c2eb743cb","path":"/var/lib/heketi/mounts/vg_e7e08adf3c8cd3dee08dd006b1596074/brick_ef253aa4b2756fc293aa101c2eb743cb/brick","device":"e7e08adf3c8cd3dee08dd006b1596074","node":"b72660bc8e28725f612d33bd675f8341","volume":"88fbf36bf87106e549dca765f171cf69","size":5242880},"TpSize":5242880,"PoolMetadataSize":28672,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"fb2e1c7f95f9926166c585ca16ac2402":{"Info":{"id":"fb2e1c7f95f9926166c585ca16ac2402","path":"/var/lib/heketi/mounts/vg_06f12f2c23a257117083a6d86d6f4087/brick_fb2e1c7f95f9926166c585ca16ac2402/brick","device":"06f12f2c23a257117083a6d86d6f4087","node":"f829fc134bc9f983a2865cc0f46a31a6","volume":"2e6ad6aa03f7fe219807cb135ca1c766","size":73400320},"TpSize":73400320,"PoolMetadataSize":368640,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"fd6a1a83004a11c0007e9392913aea8f":{"Info":{"id":"fd6a1a83004a11c0007e9392913aea8f","path":"/var/lib/heketi/mounts/vg_ae7c5467f294a953bfb274c1e6afc26d/brick_fd6a1a83004a11c0007e9392913aea8f/brick","device":"ae7c5467f294a953bfb274c1e6afc26d","node":"f829fc134bc9f983a2865cc0f46a31a6","volume":"dcee4f7f020d47e58fb48612bbba19d1","size":26214400},"TpSize":26214400,"PoolMetadataSize":131072,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"fded2411a02e03c7e0219eb9a2b08c9a":{"Info":{"id":"fded2411a02e03c7e0219eb9a2b08c9a","path":"/var/lib/heketi/mounts/vg_5054cf122b4f54021cb9d67752d23c1d/brick_fded2411a02e03c7e0219eb9a2b08c9a/brick","device":"5054cf122b4f54021cb9d67752d23c1d","node":"5f090e0aa757345d22ebdb15897bd68a","volume":"bc5012fd39c5b3ab958b9da4b0256d3a","size":4194304},"TpSize":4194304,"PoolMetadataSize":24576,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0},"ff62a0d3e0fe5c53c3725235d2832893":{"Info":{"id":"ff62a0d3e0fe5c53c3725235d2832893","path":"/var/lib/heketi/mounts/vg_e7e08adf3c8cd3dee08dd006b1596074/brick_ff62a0d3e0fe5c53c3725235d2832893/brick","device":"e7e08adf3c8cd3dee08dd006b1596074","node":"b72660bc8e28725f612d33bd675f8341","volume":"dbc601f7162df326784aea34c4ebe8f2","size":20971520},"TpSize":20971520,"PoolMetadataSize":106496,"Pending":{"Id":""},"LvmThinPool":"","LvmLv":"","SubType":0}},"nodeentries":{"5f090e0aa757345d22ebdb15897bd68a":{"State":"online","Info":{"zone":1,"hostnames":{"manage":["storage.matrix.nokia.com"],"storage":["192.168.89.220"]},"cluster":"e66ce2684350cb023c36d63b3d2bbf6f","id":"5f090e0aa757345d22ebdb15897bd68a"},"Devices":["018e77166eb3b38a0c250eb8faede2f7","25eb42cb007aab028afd0ee42b72bae9","5054cf122b4f54021cb9d67752d23c1d","9c007bfbe66a2522ad5dd851bb18d03a"]},"b72660bc8e28725f612d33bd675f8341":{"State":"online","Info":{"zone":1,"hostnames":{"manage":["app1.matrix.nokia.com"],"storage":["192.168.89.218"]},"cluster":"e66ce2684350cb023c36d63b3d2bbf6f","id":"b72660bc8e28725f612d33bd675f8341"},"Devices":["127a3d697690febddfa2a41b3a820bde","88c314d3366471a0dd89ca4d5a3c8ef8","cea64ccf8a96a13dacc1cb3ac3ea13de","e7e08adf3c8cd3dee08dd006b1596074"]},"f829fc134bc9f983a2865cc0f46a31a6":{"State":"online","Info":{"zone":1,"hostnames":{"manage":["app2.matrix.nokia.com"],"storage":["192.168.89.219"]},"cluster":"e66ce2684350cb023c36d63b3d2bbf6f","id":"f829fc134bc9f983a2865cc0f46a31a6"},"Devices":["06f12f2c23a257117083a6d86d6f4087","3343a86e75865dd02b054fb268781815","339b50a1c21d42f39d5568dcfbbf8844","ae7c5467f294a953bfb274c1e6afc26d"]}},"deviceentries":{"018e77166eb3b38a0c250eb8faede2f7":{"State":"online","Info":{"name":"/dev/sda","storage":{"total":292835328,"free":199589888,"used":93245440},"id":"018e77166eb3b38a0c250eb8faede2f7"},"Bricks":["0bd819f4969e7adbba32e2c77afc202c","16de636f8337f47a87996fca50313260","275d68b56cfc42c8b46aa90eb227eb25","2b6ed29607d007a75f7a7c1bf0da7bae","c2fe3a63636192e85d8adeefff18c92b"],"NodeId":"5f090e0aa757345d22ebdb15897bd68a","ExtentSize":4096},"06f12f2c23a257117083a6d86d6f4087":{"State":"online","Info":{"name":"/dev/sdf","storage":{"total":292835328,"free":48340992,"used":244494336},"id":"06f12f2c23a257117083a6d86d6f4087"},"Bricks":["28b3599a579db6e987f46ab0120f9d8c","413e08918a40ce6806dd6c2711d923f3","ac26b8e8306405623200599d05e87fe2","fb2e1c7f95f9926166c585ca16ac2402"],"NodeId":"f829fc134bc9f983a2865cc0f46a31a6","ExtentSize":4096},"127a3d697690febddfa2a41b3a820bde":{"State":"online","Info":{"name":"/dev/sde","storage":{"total":292835328,"free":120684544,"used":172150784},"id":"127a3d697690febddfa2a41b3a820bde"},"Bricks":["17f8162940314babf2b7d75976890130","5f22e9906d3566116ec474bb618213e5","79e211c7bee7d61bfdabb0add8e42b33","d3230ee5de8fe8300f47eb4bcea6f2e2"],"NodeId":"b72660bc8e28725f612d33bd675f8341","ExtentSize":4096},"25eb42cb007aab028afd0ee42b72bae9":{"State":"online","Info":{"name":"/dev/sde","storage":{"total":292835328,"free":244355072,"used":48480256},"id":"25eb42cb007aab028afd0ee42b72bae9"},"Bricks":["62c4432fc7a6e4aeb11573e22eaf53d8","85c1670a17f956abd0f3dd99cad53a7e","ac7b22198b6424785edd32d6f0a078d2"],"NodeId":"5f090e0aa757345d22ebdb15897bd68a","ExtentSize":4096},"3343a86e75865dd02b054fb268781815":{"State":"online","Info":{"name":"/dev/sde","storage":{"total":292835328,"free":218001408,"used":74833920},"id":"3343a86e75865dd02b054fb268781815"},"Bricks":["774b52d13a2cf996784181cf6d7db93c","9603a0a95036d6362ae2b38bdbb40428","bb29a705b3f39522b131277d64761033","c607b4aed1593917939ba85df5eefaae"],"NodeId":"f829fc134bc9f983a2865cc0f46a31a6","ExtentSize":4096},"339b50a1c21d42f39d5568dcfbbf8844":{"State":"online","Info":{"name":"/dev/sdb","storage":{"total":292835328,"free":27267072,"used":265568256},"id":"339b50a1c21d42f39d5568dcfbbf8844"},"Bricks":["1f69ed2975c20811515f054a74669be6","b85460c3ba53bb7db5ab712f7b684c51","d343f2a8b3c19bda521f93d6327451f2"],"NodeId":"f829fc134bc9f983a2865cc0f46a31a6","ExtentSize":4096},"5054cf122b4f54021cb9d67752d23c1d":{"State":"online","Info":{"name":"/dev/sdd","storage":{"total":292835328,"free":1970176,"used":290865152},"id":"5054cf122b4f54021cb9d67752d23c1d"},"Bricks":["025b1bf4bbabea839ef139c4453f9bcd","4fa8a73223d69de183d8e58618722084","9684d9e56109e011585af33ecd8039cc","b22c3ab4d7bbc420e05340e8e187bcf5","fded2411a02e03c7e0219eb9a2b08c9a"],"NodeId":"5f090e0aa757345d22ebdb15897bd68a","ExtentSize":4096},"88c314d3366471a0dd89ca4d5a3c8ef8":{"State":"online","Info":{"name":"/dev/sdf","storage":{"total":292835328,"free":212733952,"used":80101376},"id":"88c314d3366471a0dd89ca4d5a3c8ef8"},"Bricks":["281606b3024ac98416fedee94528e3c7","586c05c5ce5bd7474cfb9a9cd190225c","6f738c7cc82f66dbda8bc0b6d74883c0","84e395f5464cd25dce66b8e60f395982"],"NodeId":"b72660bc8e28725f612d33bd675f8341","ExtentSize":4096},"9c007bfbe66a2522ad5dd851bb18d03a":{"State":"online","Info":{"name":"/dev/sdf","storage":{"total":292835328,"free":45178880,"used":247656448},"id":"9c007bfbe66a2522ad5dd851bb18d03a"},"Bricks":["079acb5b4a0b0f3c45ed87f9c7def14b","22982c432dd089512ab2bde1e0663f9a","2fd555bc1cbd1b49ef04e7b88d34297b","8dba95329e797beb06eca079b9ae6a4b"],"NodeId":"5f090e0aa757345d22ebdb15897bd68a","ExtentSize":4096},"ae7c5467f294a953bfb274c1e6afc26d":{"State":"online","Info":{"name":"/dev/sdd","storage":{"total":292835328,"free":197980160,"used":94855168},"id":"ae7c5467f294a953bfb274c1e6afc26d"},"Bricks":["3c16802735df66a463468ce8262ef0a6","93eb3caff9f179d522b5498b515bf991","c4b582ef0712e8c2d750c97ff443e47c","c5d298199cfbc91ebf013707709bdbb6","e26c7a93f2a7b6d6455ab5fa7615dfb1","fd6a1a83004a11c0007e9392913aea8f"],"NodeId":"f829fc134bc9f983a2865cc0f46a31a6","ExtentSize":4096},"cea64ccf8a96a13dacc1cb3ac3ea13de":{"State":"online","Info":{"name":"/dev/sdd","storage":{"total":292835328,"free":12509184,"used":280326144},"id":"cea64ccf8a96a13dacc1cb3ac3ea13de"},"Bricks":["08599e162e105063a8e2efec868d2a76","1f52101af46329a1d5e758156c8267e0","31ed93bf445cf1fd00d34660a80e1078","7ef53adf2bce0e240229941ac7f43661","ae2fa0410160c9fb489af8e5ad063ac3"],"NodeId":"b72660bc8e28725f612d33bd675f8341","ExtentSize":4096},"e7e08adf3c8cd3dee08dd006b1596074":{"State":"online","Info":{"name":"/dev/sdc","storage":{"total":292835328,"free":145293312,"used":147542016},"id":"e7e08adf3c8cd3dee08dd006b1596074"},"Bricks":["800e2a99110af93ca69bd950c9ea7ef8","980bd302213bbdd2d9aa09dbe8abf62b","ef253aa4b2756fc293aa101c2eb743cb","ff62a0d3e0fe5c53c3725235d2832893"],"NodeId":"b72660bc8e28725f612d33bd675f8341","ExtentSize":4096}},"blockvolumeentries":{},"dbattributeentries":{"DB_BRICK_HAS_SUBTYPE_FIELD":{"Key":"DB_BRICK_HAS_SUBTYPE_FIELD","Value":"yes"},"DB_CLUSTER_HAS_FILE_BLOCK_FLAG":{"Key":"DB_CLUSTER_HAS_FILE_BLOCK_FLAG","Value":"yes"},"DB_GENERATION_ID":{"Key":"DB_GENERATION_ID","Value":"d90958f6a026bc80efe34bea84fa074f"},"DB_HAS_PENDING_OPS_BUCKET":{"Key":"DB_HAS_PENDING_OPS_BUCKET","Value":"yes"}},"pendingoperations":{}}heketi] WARNING 2019/01/23 12:07:49 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 250.763µs [negroni] Started POST /volumes [heketi] WARNING 2019/01/23 12:07:49 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 169.08µs [negroni] Started DELETE /volumes/520bc5f4e1bfd029855a72f9ca7ebf6c [negroni] Completed 404 Not Found in 148.125µs [negroni] Started POST /volumes [heketi] WARNING 2019/01/23 12:08:04 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 496.624µs [negroni] Started POST /volumes [heketi] WARNING 2019/01/23 12:08:04 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 101.673µs [negroni] Started POST /volumes [heketi] WARNING 2019/01/23 12:08:19 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 209.681µs [negroni] Started POST /volumes [heketi] WARNING 2019/01/23 12:08:19 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 103.595µs [negroni] Started POST /volumes [heketi] WARNING 2019/01/23 12:08:34 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 297.594µs [negroni] Started POST /volumes [heketi] WARNING 2019/01/23 12:08:34 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 96.75µs [negroni] Started POST /volumes [negroni] Started POST /volumes [heketi] WARNING 2019/01/23 12:08:49 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 477.007µs [heketi] WARNING 2019/01/23 12:08:49 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 165.38µs [negroni] Started POST /volumes [heketi] WARNING 2019/01/23 12:09:04 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 488.253µs [negroni] Started POST /volumes [heketi] WARNING 2019/01/23 12:09:04 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 171.836µs [negroni] Started POST /volumes [heketi] WARNING 2019/01/23 12:09:19 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 208.59µs [negroni] Started POST /volumes [heketi] WARNING 2019/01/23 12:09:19 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 125.141µs [negroni] Started DELETE /volumes/99e87ecd0a816ac34ae5a04eabc1d606 [negroni] Completed 404 Not Found in 138.687µs [negroni] Started POST /volumes [heketi] WARNING 2019/01/23 12:09:34 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 392.814µs [negroni] Started POST /volumes [heketi] WARNING 2019/01/23 12:09:34 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 208.28µs [negroni] Started POST /volumes [heketi] WARNING 2019/01/23 12:09:49 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 360.289µs [negroni] Started POST /volumes [heketi] WARNING 2019/01/23 12:09:49 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 169.465µs [negroni] Started DELETE /volumes/520bc5f4e1bfd029855a72f9ca7ebf6c [negroni] Completed 404 Not Found in 197.691µs [negroni] Started POST /volumes [heketi] WARNING 2019/01/23 12:10:19 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 349.65µs [negroni] Started POST /volumes [heketi] WARNING 2019/01/23 12:10:19 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 80.175µs [negroni] Started POST /volumes [heketi] WARNING 2019/01/23 12:10:34 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 191.618µs [negroni] Started POST /volumes [heketi] WARNING 2019/01/23 12:10:34 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 104.703µs [negroni] Started POST /volumes [negroni] Started POST /volumes [heketi] WARNING 2019/01/23 12:10:49 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 139.424µs [heketi] WARNING 2019/01/23 12:10:49 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 356.587µs [negroni] Started POST /volumes [negroni] Started POST /volumes [heketi] WARNING 2019/01/23 12:11:04 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 603.166µs [heketi] WARNING 2019/01/23 12:11:04 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 409.484µs [negroni] Started POST /volumes [negroni] Started POST /volumes [heketi] WARNING 2019/01/23 12:11:19 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 224.426µs [heketi] WARNING 2019/01/23 12:11:19 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 244.469µs [negroni] Started POST /volumes [heketi] WARNING 2019/01/23 12:11:34 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 198.034µs [negroni] Started POST /volumes [heketi] WARNING 2019/01/23 12:11:34 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 91.59µs [negroni] Started DELETE /volumes/99e87ecd0a816ac34ae5a04eabc1d606 [negroni] Completed 404 Not Found in 128.194µs [negroni] Started POST /volumes [negroni] Started POST /volumes [heketi] WARNING 2019/01/23 12:11:49 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 227.904µs [heketi] WARNING 2019/01/23 12:11:49 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 129.223µs [negroni] Started POST /volumes [heketi] WARNING 2019/01/23 12:12:04 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 203.491µs [negroni] Started POST /volumes [heketi] WARNING 2019/01/23 12:12:04 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 109.563µs [negroni] Started POST /volumes [negroni] Started POST /volumes [heketi] WARNING 2019/01/23 12:12:19 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 262.199µs [heketi] WARNING 2019/01/23 12:12:19 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 168.901µs [negroni] Started DELETE /volumes/520bc5f4e1bfd029855a72f9ca7ebf6c [negroni] Completed 404 Not Found in 111.988µs [negroni] Started POST /volumes [heketi] WARNING 2019/01/23 12:12:34 operations in-flight (8) exceeds limit (8) [negroni] Started POST /volumes [negroni] Completed 429 Too Many Requests in 339.567µs [heketi] WARNING 2019/01/23 12:12:34 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 304.878µs [negroni] Started POST /volumes [negroni] Started POST /volumes [heketi] WARNING 2019/01/23 12:12:49 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 357.406µs [heketi] WARNING 2019/01/23 12:12:49 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 400.669µs [negroni] Started POST /volumes [negroni] Started POST /volumes [heketi] WARNING 2019/01/23 12:13:04 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 240.076µs [heketi] WARNING 2019/01/23 12:13:04 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 118.418µs [negroni] Started POST /volumes [negroni] Started POST /volumes [heketi] WARNING 2019/01/23 12:13:19 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 230.416µs [heketi] WARNING 2019/01/23 12:13:19 operations in-flight (8) exceeds limit (8) [negroni] Completed 429 Too Many Requests in 178.939µs
_______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx https://lists.gluster.org/mailman/listinfo/gluster-users
