Hello, we see same problems. Deleting all the pools and redeploy rgw solved it on that test cluster, however that is no solution for production ;) systemd[1]: Started Ceph rados gateway. radosgw[7171]: 2021-04-04T14:37:51.508+0000 7fc6641efc00 0 deferred set uid:gid to 167:167 (ceph:ceph) radosgw[7171]: failed to chown /dev/null: (30) Read-only file system radosgw[7171]: 2021-04-04T14:37:51.508+0000 7fc6641efc00 0 ceph version 16.2.0-31-g5922b2b9c1 (5922b2b9c17f0877f84b0b3f2557ab72a628cbfe) pacific (stable), process radosgw, pid 7171 radosgw[7171]: 2021-04-04T14:37:51.508+0000 7fc6641efc00 0 framework: beast radosgw[7171]: 2021-04-04T14:37:51.508+0000 7fc6641efc00 0 framework conf key: ssl_port, val: 443 radosgw[7171]: 2021-04-04T14:37:51.508+0000 7fc6641efc00 0 framework conf key: port, val: 80 radosgw[7171]: 2021-04-04T14:37:51.508+0000 7fc6641efc00 0 framework conf key: ssl_certificate, val: /etc/ceph/rgwcert.pem radosgw[7171]: 2021-04-04T14:37:51.508+0000 7fc6641efc00 1 radosgw_Main not setting numa affinity radosgw[7171]: 2021-04-04T14:37:51.680+0000 7fc6641efc00 -1 static int rgw::cls::fifo::FIFO::create(librados::v14_2_0::IoCtx, std::__cxx11::string, std::unique_ptr<rgw::cls::fifo::FIFO>*, optional_yield, std::optional<rados::cls::fifo::objv>, std::optional<std::basic_string_view<char> >, bool, uint64_t, uint64_t):925 create_meta failed: r=-5 radosgw[7171]: 2021-04-04T14:37:51.680+0000 7fc6641efc00 -1 static int rgw::cls::fifo::FIFO::create(librados::v14_2_0::IoCtx, std::__cxx11::string, std::unique_ptr<rgw::cls::fifo::FIFO>*, optional_yield, std::optional<rados::cls::fifo::objv>, std::optional<std::basic_string_view<char> >, bool, uint64_t, uint64_t):925 create_meta failed: r=-5 radosgw[7171]: 2021-04-04T14:37:51.680+0000 7fc6641efc00 -1 int RGWDataChangesLog::start(const RGWZone*, const RGWZoneParams&, RGWSI_Cls*, librados::v14_2_0::Rados*): Error when starting backend: Input/output error radosgw[7171]: 2021-04-04T14:37:51.680+0000 7fc6641efc00 0 ERROR: failed to start datalog_rados service ((5) Input/output error radosgw[7171]: 2021-04-04T14:37:51.680+0000 7fc6641efc00 -1 int RGWDataChangesLog::start(const RGWZone*, const RGWZoneParams&, RGWSI_Cls*, librados::v14_2_0::Rados*): Error when starting backend: Input/output error radosgw[7171]: 2021-04-04T14:37:51.680+0000 7fc6641efc00 0 ERROR: failed to init services (ret=(5) Input/output error) radosgw[7171]: 2021-04-04T14:37:51.700+0000 7fc6641efc00 -1 Couldn't init storage provider (RADOS) radosgw[7171]: 2021-04-04T14:37:51.700+0000 7fc6641efc00 -1 Couldn't init storage provider (RADOS) systemd[1]: ceph-radosgw@xxxxxxx-croit-host-C0DE01.service: Main process exited, code=exited, status=5/NOTINSTALLED systemd[1]: ceph-radosgw@xxxxxxx-croit-host-C0DE01.service: Unit entered failed state. systemd[1]: ceph-radosgw@xxxxxxx-croit-host-C0DE01.service: Failed with result 'exit-code'. -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.verges@xxxxxxxx Chat: https://t.me/MartinVerges croit GmbH, Freseniusstr. 31h, 81247 Munich CEO: Martin Verges - VAT-ID: DE310638492 Com. register: Amtsgericht Munich HRB 231263 Web: https://croit.io YouTube: https://goo.gl/PGE1Bx -- Martin Verges Managing director Mobile: +49 174 9335695 E-Mail: martin.verges@xxxxxxxx Chat: https://t.me/MartinVerges croit GmbH, Freseniusstr. 31h, 81247 Munich CEO: Martin Verges - VAT-ID: DE310638492 Com. register: Amtsgericht Munich HRB 231263 Web: https://croit.io YouTube: https://goo.gl/PGE1Bx On Mon, 5 Apr 2021 at 19:59, Robert Sander <r.sander@xxxxxxxxxxxxxxxxxxx> wrote: > Hi, > > Am 04.04.21 um 15:22 schrieb 胡 玮文: > > > bash[9823]: debug 2021-04-04T13:01:04.995+0000 7ff80f172440 -1 static > int rgw::cls::fifo::FIFO::create(librados::v14_2_0::IoCtx, > std::__cxx11::string, std::unique_ptr<rgw::cls::fifo::FIFO>*, > optional_yield, std::optional<rados::cls::fifo::objv>, > std::optional<std::basic_string_view<char> >, bool, uint64_t, uint64_t):925 > create_meta failed: r=-5 > > bash[9823]: debug 2021-04-04T13:01:04.995+0000 7ff80f172440 -1 int > RGWDataChangesLog::start(const RGWZone*, const RGWZoneParams&, RGWSI_Cls*, > librados::v14_2_0::Rados*): Error when starting backend: Input/output error > > bash[9823]: debug 2021-04-04T13:01:04.995+0000 7ff80f172440 0 ERROR: > failed to start datalog_rados service ((5) Input/output error > > bash[9823]: debug 2021-04-04T13:01:04.995+0000 7ff80f172440 0 ERROR: > failed to init services (ret=(5) Input/output error) > > I see the same issues on an upgraded cluster. > > Regards > -- > Robert Sander > Heinlein Consulting GmbH > Schwedter Str. 8/9b, 10119 Berlin > > http://www.heinlein-support.de > > Tel: 030 / 405051-43 > Fax: 030 / 405051-19 > > Zwangsangaben lt. §35a GmbHG: > HRB 93818 B / Amtsgericht Berlin-Charlottenburg, > Geschäftsführer: Peer Heinlein -- Sitz: Berlin > > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx > _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx