Hi Kyle, On Thu, May 6, 2021 at 7:56 AM Kyle Dean <k.s-dean@xxxxxxxxxxx> wrote: > > Hi, hoping someone could help me get to the bottom of this particular issue I'm having. > > I have ceph octopus installed using ceph-ansible. > > Currently, I have 3 MDS servers running, and one client connected to the active MDS. I'm currently storing a very large encrypted container on the CephFS file system, 8TB worth, and I'm writing data into it from the client host. > > recently I have noticed a severe impact on performance, and the time take to do processing on file within the container has increased from 1 minute to 11 minutes. > > in the ceph dashboard, when I take a look at the performance tab on the file system page, the Write Ops are increasing exponentially over time. > > At the end of April around the 22nd I had 49 write Ops on the performance page for the MDS deamons. This is now at 266467 Write Ops and increasing. > > Also the client requests have gone from 14 to 67 to 117 and is now at 283 > > would someone be able to help me make sense of why the performance has decreased and what is going on with the client requests and write operations. I suggest you look at the "perf dump" statistics from the MDS (via ceph tell or admin socket) over a period of time to get an idea what operations it's performing. It's probable your workload changed somehow and that is the cause. -- Patrick Donnelly, Ph.D. He / Him / His Principal Software Engineer Red Hat Sunnyvale, CA GPG: 19F28A586F808C2402351B93C3301A3E258DD79D _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |