We recently discussed briefly the Seagate Ethernet drives, which were
basically dismissed as too limited. But what about moving an ARM SBC to
the drive tray, complete with an mSATA SSD slot?
A proper SBC could implement full Ubuntu single-drive failure domains
that also solve the journal issue.
Using Seagate's dual-ethernet-over-sata-plug thingie, presumably any
interested party could make a drive chassis to accept Ethernet hot-plug
drives (i.e. basically a switch with a couple of 10GbE ports on the
back).
There are a couple of low-cost SBCs with GbE and SATA - anyone know of
one around $100 that provides 2x GbE and 2x SATA?
Apologise for the O/T - but you did raise the SBC idea :)
On 2013-11-05 15:21, Yu Changyuan wrote:
Finally, my tiny ceph cluster get 3 monitors, newly added mon.b and
mon.c both running on cubieboard2, which is cheap but still with
enough cpu power(dual-core arm A7 cpu, 1.2G) and memory(1G).
But compare to mon.a which running on an amd64 cpu, both mon.b and
mon.c easily consume too much memory, so I want to know whether this
is caused by memory leak. Below is the output of 'ceph tell mon.a
heap
stats' and 'ceph tell mon.c heap stats'(mon.c only start 12hr ago,
while mon.a already running for more than 10 days)
mon.atcmalloc heap
stats:------------------------------------------------
MALLOC: 5480160 ( 5.2 MiB) Bytes in use by application
MALLOC: + 28065792 ( 26.8 MiB) Bytes in page heap freelist
MALLOC: + 15242312 ( 14.5 MiB) Bytes in central cache freelist
MALLOC: + 10116608 ( 9.6 MiB) Bytes in transfer cache
freelist
MALLOC: + 10432216 ( 9.9 MiB) Bytes in thread cache freelists
MALLOC: + 1667224 ( 1.6 MiB) Bytes in malloc metadata
MALLOC: ------------
MALLOC: = 71004312 ( 67.7 MiB) Actual memory used (physical +
swap)
MALLOC: + 57540608 ( 54.9 MiB) Bytes released to OS (aka
unmapped)
MALLOC: ------------
MALLOC: = 128544920 ( 122.6 MiB) Virtual address space used
MALLOC:
MALLOC: 4655 Spans in use
MALLOC: 34 Thread heaps in use
MALLOC: 8192 Tcmalloc page size
------------------------------------------------
Call ReleaseFreeMemory() to release freelist memory to the OS (via
madvise()).
Bytes released to the
mon.ctcmalloc heap
stats:------------------------------------------------
MALLOC: 175861640 ( 167.7 MiB) Bytes in use by application
MALLOC: + 2220032 ( 2.1 MiB) Bytes in page heap freelist
MALLOC: + 1007560 ( 1.0 MiB) Bytes in central cache freelist
MALLOC: + 2871296 ( 2.7 MiB) Bytes in transfer cache freelist
MALLOC: + 4686000 ( 4.5 MiB) Bytes in thread cache freelists
MALLOC: + 2758880 ( 2.6 MiB) Bytes in malloc metadata
MALLOC: ------------
MALLOC: = 189405408 ( 180.6 MiB) Actual memory used (physical +
swap)
MALLOC: + 0 ( 0.0 MiB) Bytes released to OS (aka
unmapped)
MALLOC: ------------
MALLOC: = 189405408 ( 180.6 MiB) Virtual address space used
MALLOC:
MALLOC: 3445 Spans in use
MALLOC: 14 Thread heaps in use
MALLOC: 8192 Tcmalloc page size
------------------------------------------------
Call ReleaseFreeMemory() to release freelist memory to the OS (via
madvise()).
Bytes released to the
The ceph versin is 0.67.4, compiled with tcmalloc enabled,
gcc(armv7a-hardfloat-linux-gnueabi-gcc) version 4.7.3 and I also try
to dump heap, but I can not find anything useful, below is a recent
dump, output by command "pprof --text /usr/bin/ceph-mon
mon.c.profile.0021.heap". What extra step should I take to make the
dump more meaningful?
Using local file /usr/bin/ceph-mon.
Using local file mon.c.profile.0021.heap.
Total: 149.3 MB
146.2 97.9% 97.9% 146.2 97.9% 00000000b6a7ce7c
1.4 0.9% 98.9% 1.4 0.9%
std::basic_string::_Rep::_S_create ??:0
1.4 0.9% 99.8% 1.4 0.9% 00000000002dd794
0.1 0.1% 99.9% 0.1 0.1% 00000000b6a81170
0.1 0.1% 99.9% 0.1 0.1% 00000000b6a80894
0.0 0.0% 100.0% 0.0 0.0% 00000000b6a7e2ac
0.0 0.0% 100.0% 0.0 0.0% 00000000b6a81410
0.0 0.0% 100.0% 0.0 0.0% 0000000000367450
0.0 0.0% 100.0% 0.0 0.0% 00000000001d4474
0.0 0.0% 100.0% 0.0 0.0% 000000000028847c
0.0 0.0% 100.0% 0.0 0.0% 00000000b6a7e8d8
0.0 0.0% 100.0% 0.0 0.0% 000000000020c80c
0.0 0.0% 100.0% 0.0 0.0% 000000000028bd20
0.0 0.0% 100.0% 0.0 0.0% 00000000b6a63248
0.0 0.0% 100.0% 0.0 0.0% 00000000b6a83478
0.0 0.0% 100.0% 0.0 0.0% 00000000b6a806f0
0.0 0.0% 100.0% 0.0 0.0% 00000000002eb8b8
0.0 0.0% 100.0% 0.0 0.0% 000000000024efb4
0.0 0.0% 100.0% 0.0 0.0% 000000000027e550
0.0 0.0% 100.0% 0.0 0.0% 00000000b6a77104
0.0 0.0% 100.0% 0.0 0.0% _dl_mcount ??:0
0.0 0.0% 100.0% 0.0 0.0% 00000000003673ec
0.0 0.0% 100.0% 0.0 0.0% 00000000b6a7a91c
0.0 0.0% 100.0% 0.0 0.0% 0000000000295e44
0.0 0.0% 100.0% 0.0 0.0% 00000000b6a7ee38
0.0 0.0% 100.0% 0.0 0.0% 0000000000283948
0.0 0.0% 100.0% 0.0 0.0% 00000000002a53c4
0.0 0.0% 100.0% 0.0 0.0% 00000000b6a7665c
0.0 0.0% 100.0% 0.0 0.0% 00000000002c4590
0.0 0.0% 100.0% 0.0 0.0% 00000000b6a7e88c
0.0 0.0% 100.0% 0.0 0.0% 00000000b6a8456c
0.0 0.0% 100.0% 0.0 0.0% 00000000b6a76ed4
0.0 0.0% 100.0% 0.0 0.0% 00000000b6a842f0
0.0 0.0% 100.0% 0.0 0.0% 00000000b6a72bd0
0.0 0.0% 100.0% 0.0 0.0% 00000000b6a73cf8
0.0 0.0% 100.0% 0.0 0.0% 00000000b6a7100c
0.0 0.0% 100.0% 0.0 0.0% 00000000b6a7dec4
0.0 0.0% 100.0% 0.0 0.0% 000000000035e6e8
0.0 0.0% 100.0% 0.0 0.0% 00000000b6a78f68
0.0 0.0% 100.0% 0.0 0.0% 00000000b6a7de9c
0.0 0.0% 100.0% 0.0 0.0% 0000000000220528
0.0 0.0% 100.0% 0.0 0.0% 000000000035e7c0
0.0 0.0% 100.0% 0.0 0.0% 00000000b6a6b2f8
0.0 0.0% 100.0% 0.0 0.0% 00000000b6a80a04
0.0 0.0% 100.0% 0.0 0.0% 00000000b6a62e7c
0.0 0.0% 100.0% 0.0 0.0% 00000000b6a66f50
0.0 0.0% 100.0% 0.0 0.0% 00000000b6a7e958
0.0 0.0% 100.0% 0.0 0.0% 00000000b6a6cfb8
0.0 0.0% 100.0% 0.0 0.0%
leveldb::DBImpl::MakeRoomForWrite (inline) ??:0
0.0 0.0% 100.0% 0.0 0.0% 000000000020797c
0.0 0.0% 100.0% 0.0 0.0% 00000000b6a69de0
0.0 0.0% 100.0% 0.0 0.0% 00000000001d0af0
0.0 0.0% 100.0% 0.0 0.0% 00000000001d0ebc
0.0 0.0% 100.0% 0.0 0.0% 00000000002a0cd4
0.0 0.0% 100.0% 0.0 0.0% 000000000036909c
0.0 0.0% 100.0% 0.0 0.0% 000000000040b02c
0.0 0.0% 100.0% 0.0 0.0% 00000000001d0b68
0.0 0.0% 100.0% 0.0 0.0% 0000000000392fa0
0.0 0.0% 100.0% 0.0 0.0% 00000000b6a64404
0.0 0.0% 100.0% 0.0 0.0% 00000000b6a791b4
0.0 0.0% 100.0% 0.0 0.0% 00000000001d9824
0.0 0.0% 100.0% 0.0 0.0% 0000000000213928
0.0 0.0% 100.0% 0.0 0.0% 00000000002a0cb8
0.0 0.0% 100.0% 0.0 0.0% 00000000002a4fcc
0.0 0.0% 100.0% 0.0 0.0% 00000000b6a725ac
0.0 0.0% 100.0% 0.0 0.0% 00000000b6a66308
0.0 0.0% 100.0% 0.0 0.0% 00000000b6a79068
0.0 0.0% 100.0% 0.0 0.0% 00000000000013b2
0.0 0.0% 100.0% 0.0 0.0% 000000000040b000
0.0 0.0% 100.0% 0.1 0.1% 00000000004d839b
0.0 0.0% 100.0% 0.0 0.0% 0000000000f29887
0.0 0.0% 100.0% 0.0 0.0% 0000000000f3eb6b
0.0 0.0% 100.0% 0.0 0.0% 0000000000f6e1cb
0.0 0.0% 100.0% 0.0 0.0% 0000000000f6edab
0.0 0.0% 100.0% 0.0 0.0% 0000000000f873ab
0.0 0.0% 100.0% 0.0 0.0% 0000000000f8a26b
0.0 0.0% 100.0% 0.0 0.0% 0000000000f8b0cb
0.0 0.0% 100.0% 0.0 0.0% 0000000000f92dab
0.0 0.0% 100.0% 0.0 0.0% 0000000000f9c96b
0.0 0.0% 100.0% 0.0 0.0% 0000000000fa24bf
0.0 0.0% 100.0% 0.0 0.0% 0000000000fadd8b
0.0 0.0% 100.0% 0.0 0.0% 0000000000fb06ab
0.0 0.0% 100.0% 0.0 0.0% 0000000000fb0d0b
0.0 0.0% 100.0% 0.0 0.0% 0000000000fb494b
0.0 0.0% 100.0% 0.0 0.0% 0000000000fbad6b
0.0 0.0% 100.0% 0.0 0.0% 0000000000fbb2cb
0.0 0.0% 100.0% 0.0 0.0% 0000000000fbea6b
0.0 0.0% 100.0% 0.0 0.0% 0000000000fed0eb
0.0 0.0% 100.0% 0.0 0.0% 0000000000fed69b
0.0 0.0% 100.0% 0.0 0.0% 000000000129920b
0.0 0.0% 100.0% 0.0 0.0% 00000000014250eb
0.0 0.0% 100.0% 0.0 0.0% 000000000166cfc5
0.0 0.0% 100.0% 0.1 0.1% 000000000166d711
0.0 0.0% 100.0% 0.0 0.0% 0000000003531d2b
0.0 0.0% 100.0% 0.0 0.0% 000000000379adbb
0.0 0.0% 100.0% 0.0 0.0% 0000000004e888fb
0.0 0.0% 100.0% 0.0 0.0% 0000000004e894ab
0.0 0.0% 100.0% 0.0 0.0% 0000000004e8951b
0.0 0.0% 100.0% 0.0 0.0% 00000000060146d3
0.0 0.0% 100.0% 0.0 0.0% 000000000601482f
0.0 0.0% 100.0% 0.0 0.0% 00000000060fcd2b
0.0 0.0% 100.0% 0.0 0.0% 00000000060fd33b
0.0 0.0% 100.0% 0.0 0.0% 00000000060fdfbb
0.0 0.0% 100.0% 0.0 0.0% 000000000a820749
0.0 0.0% 100.0% 0.0 0.0% 000000000bfb1950
0.0 0.0% 100.0% 0.0 0.0% 00000000b6a43f23
0.0 0.0% 100.0% 0.0 0.0% __clone ??:0
0.0 0.0% 100.0% 0.0 0.0%
leveldb::DBImpl::MakeRoomForWrite ??:0
0.0 0.0% 100.0% 0.2 0.1% std::num_put::do_put@806e4
??:0
0.0 0.0% 100.0% 0.4 0.2% std::num_put::do_put@80b44
??:0
0.0 0.0% 100.0% 0.1 0.1% std::num_put::do_put@80e00
??:0
PS: there's a cubietruck
board(http://docs.cubieboard.org/products/start#cubietruck_cubieboard3
[1]) released recently, which features a dual-core arm A7 cpu, 2G
RAM,
1Gbit eth port, and a sata 2.0 port, for $89, maybe suitable for
cheap
dedicate osd server with single disk.
--
Best regards,
Changyuan
Links:
------
[1] http://docs.cubieboard.org/products/start#cubietruck_cubieboard3
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
