OK zstd.h needs: #define ZSTD_STATIC_LINKING_ONLY to export that one. Stefan Am 04.11.2017 um 21:23 schrieb Stefan Priebe - Profihost AG: > Thanks - not a C++ guy what's wrong with it? > > /build/ceph/src/compressor/zstd/ZstdCompressor.h: In member function > 'virtual int ZstdCompressor::compress(const bufferlist&, > ceph::bufferlist&)': > /build/ceph/src/compressor/zstd/ZstdCompressor.h:48:28: error: > 'ZSTD_resetCStream' was not declared in this scope > ZSTD_resetCStream(cs, 0); > ^ > /build/ceph/src/compressor/zstd/ZstdCompressor.h: In member function > 'virtual int ZstdCompressor::decompress(ceph::buffer::list::iterator&, > size_t, ceph::bufferlist&)': > /build/ceph/src/compressor/zstd/ZstdCompressor.h:89:28: error: > 'ZSTD_resetDStream' was not declared in this scope > ZSTD_resetDStream(ds, 0); > > > zstd.h which is included has the function: > > src/zstd/lib/zstd.h:ZSTDLIB_API size_t ZSTD_resetCStream(ZSTD_CStream* > zcs, unsigned long long pledgedSrcSize); /**< re-use compression > parameters from previous init; skip dictionary loading stage; zcs must > be init at least once before */ > > Stefan > > Am 04.11.2017 um 21:10 schrieb Sage Weil: >> On Sat, 4 Nov 2017, Stefan Priebe - Profihost AG wrote: >>> Hi Sage, >>> >>> Am 26.10.2017 um 13:58 schrieb Sage Weil: >>>> On Thu, 26 Oct 2017, Stefan Priebe - Profihost AG wrote: >>>>> Hi Sage, >>>>> >>>>> Am 25.10.2017 um 21:54 schrieb Sage Weil: >>>>>> On Wed, 25 Oct 2017, Stefan Priebe - Profihost AG wrote: >>>>>>> Hello, >>>>>>> >>>>>>> in the lumious release notes is stated that zstd is not supported by >>>>>>> bluestor due to performance reason. I'm wondering why btrfs instead >>>>>>> states that zstd is as fast as lz4 but compresses as good as zlib. >>>>>>> >>>>>>> Why is zlib than supported by bluestor? And why does btrfs / facebook >>>>>>> behave different? >>>>>>> >>>>>>> "BlueStore supports inline compression using zlib, snappy, or LZ4. (Ceph >>>>>>> also supports zstd for RGW compression but zstd is not recommended for >>>>>>> BlueStore for performance reasons.)" >>>>>> >>>>>> zstd will work but in our testing the performance wasn't great for >>>>>> bluestore in particular. The problem was that for each compression run >>>>>> there is a relatively high start-up cost initializing the zstd >>>>>> context/state (IIRC a memset of a huge memory buffer) that dominated the >>>>>> execution time... primarily because bluestore is generally compressing >>>>>> pretty small chunks of data at a time, not big buffers or streams. >>>>>> >>>>>> Take a look at unittest_compression timings on compressing 16KB buffers >>>>>> (smaller than bluestore needs usually, but illustrated of the problem): >>>>>> >>>>>> [ RUN ] Compressor/CompressorTest.compress_16384/0 >>>>>> [plugin zlib (zlib/isal)] >>>>>> [ OK ] Compressor/CompressorTest.compress_16384/0 (294 ms) >>>>>> [ RUN ] Compressor/CompressorTest.compress_16384/1 >>>>>> [plugin zlib (zlib/noisal)] >>>>>> [ OK ] Compressor/CompressorTest.compress_16384/1 (1755 ms) >>>>>> [ RUN ] Compressor/CompressorTest.compress_16384/2 >>>>>> [plugin snappy (snappy)] >>>>>> [ OK ] Compressor/CompressorTest.compress_16384/2 (169 ms) >>>>>> [ RUN ] Compressor/CompressorTest.compress_16384/3 >>>>>> [plugin zstd (zstd)] >>>>>> [ OK ] Compressor/CompressorTest.compress_16384/3 (4528 ms) >>>>>> >>>>>> It's an order of magnitude slower than zlib or snappy, which probably >>>>>> isn't acceptable--even if it is a bit smaller. >>> >>> i've fixed the zstd compression plugin to use reset stream instead of >>> initializing new objects. >>> >>> What's needed to run only / just the unittest_compression test? >> >> make unittest_compression && bin/unittest_compression >> >> should do it! >> >> sage >> _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |