Re: Ceph OSDs advice

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



You should subtract buffers and cache from the used memory to get a more accurate representation of how much memory is actually available to processes. In this case that puts you around 22G of used - or a better term might be unavailable memory. Buffers and cache can be reallocated when needed - it's just Linux taking advantage of memory under the theory why not use it if it's there? Memory is fast so Linux will take advantage of it.

With 72 OSD's 22G of memory puts you below the 500MB/daemon that you've mentioned so I don't think you have anything to be concerned about.

___

John Petrini

The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission,  dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer.


On Tue, Feb 14, 2017 at 11:24 PM, Khang Nguyễn Nhật <nguyennhatkhang2704@xxxxxxxxx> wrote:
Hi Sam,
Thank for your reply. I use BTRFS file system on OSDs.
Here is result of "free -hw":

                   total         used          free        shared     buffers       cache     available
Mem:           125G         58G         31G        1.2M        3.7M         36G         60G

and "ceph df":

GLOBAL:
    SIZE     AVAIL     RAW USED     %RAW USED 
    523T      522T        1539G          0.29 
POOLS:
    NAME                           ID     USED     %USED     MAX AVAIL     OBJECTS 
    ............
    default.rgw.buckets.data  92     597G      0.15              391T            84392 
    ............

I was reviced this a few minutes ago.

2017-02-15 10:50 GMT+07:00 Sam Huracan <nowitzki.sammy@xxxxxxxxx>:
Hi Khang,

What file system do you use in OSD node?
XFS always use Memory for caching data before writing to disk.

So, don't worry, it always holds memory in your system as much as possible.



2017-02-15 10:35 GMT+07:00 Khang Nguyễn Nhật <nguyennhatkhang2704@xxxxxxxxx>:
Hi all,
My ceph OSDs is running on Fedora-server24 with config are:
128GB RAM DDR3, CPU Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz, 72 OSDs (8TB per OSD). My cluster was use ceph object gateway with S3 API. Now, it had contained 500GB data but it was used > 50GB RAM. I'm worry my OSD will dead if i continue put file to it. I had read "OSDs do not require as much RAM for regular operations (e.g., 500MB of RAM per daemon instance); however, during recovery they need significantly more RAM (e.g., ~1GB per 1TB of storage per daemon)." in Ceph Hardware Recommendations. Someone can give me advice on this issue? Thank

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com




_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com


_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux