On Monday 29 December 2008 14:04:29 ext Marcel Holtmann wrote: > Hi Siarhei, > > > > I wrote this script to make some tests on the SBC decoder and encoder using > > > the recommended testing procedure with the reference bitstreams, the > > > reference codec and PEAQ. > > > http://net.cs.uni-tuebingen.de/html/nexgenvoip/html/ > > > I got little bit confused with all the latest patches and whether they are > > > included or not. Just send me an email on which patch you like to have > > > tested. Running the tests just takes half an hour; me to answer my emails > > > maybe a bit longer. > > > > Please try the following patch (apply it to the latest git): > > http://marc.info/?l=linux-bluetooth&m=123054787830678&w=2 > > > > You can try the patch "as is", and also with SBC_HIGH_PRECISION define > > uncommented. High precision mode is naturally more likely to pass the > > conformance tests. > > > > I used my own script for testing with 'tiny_psnr' tool for comparing original file > > before encoding and the final decoded result (it measures standard deviation > > and PSNR). It would be interesting to see how our results correlate. > > can you open source tiny_psnr an we merge it into the BlueZ source? It is already open source. This very simple tool is used in ffmpeg project regression tests (and is part of ffmpeg distribution). It does not do anything extraordinary, but just analyzes the difference between several audio files and gets standard deviation and PSNR statistics. I just did not feel like reinventing the wheel and used it for estimating quality :) I only use the following patch on top of it (it can do automatic shift detection): http://article.gmane.org/gmane.comp.video.ffmpeg.devel/74597 I find this patch quite useful (especially as it turns out for SBC), but ffmpeg developers do not think so... A crude ruby script (sorry, I don't speak shell scripting language) is attached. I have been using it for testing audio quality. For example, here is the test of a high precision sbcenc build (bitpool=255): ./sbc_encode_test.rb BigBuckBunny-stereo.flac [2, 48000] ["-j -S -s8 -B16 -b255", "-j -l16 -n8 -r1569000"] --- comparing original / sbcenc + sbcdec --- stddev: 108.67 PSNR: 55.60 bytes:114519261/114520000 --- comparing original / sbcenc + sbc_decoder.exe --- stddev: 1.09 PSNR: 95.56 bytes:114519260/114520000 --- comparing original / sbc_encoder.exe + sbc_decoder.exe --- stddev: 1.09 PSNR: 95.56 bytes:114519260/114520000 --- comparing sbcenc + sbc_decoder.exe / sbc_encoder.exe + sbc_decoder.exe stddev: 0.01 PSNR:130.99 bytes:114519552/114519552 Test of a standard sbcenc build (bitpool=255): ./sbc_encode_test.rb BigBuckBunny-stereo.flac [2, 48000] ["-j -S -s8 -B16 -b255", "-j -l16 -n8 -r1569000"] --- comparing original / sbcenc + sbcdec --- stddev: 108.71 PSNR: 55.59 bytes:114519261/114520000 --- comparing original / sbcenc + sbc_decoder.exe --- stddev: 2.07 PSNR: 89.98 bytes:114519260/114520000 --- comparing original / sbc_encoder.exe + sbc_decoder.exe --- stddev: 1.09 PSNR: 95.56 bytes:114519260/114520000 --- comparing sbcenc + sbc_decoder.exe / sbc_encoder.exe + sbc_decoder.exe stddev: 1.77 PSNR: 91.34 bytes:114519552/114519552 Having high bitrate is useful for detecting bugs in analysis filter because its contribution to quality loss is more visible in this configuration and other factors play lesser role. I tested a lot of various files and settings configurations. According to this script, the quality of high precision sbcenc build matches the quality of reference encoder pretty well. Standard 16-bit fixed point sbcenc build introduces a very minor quality loss, noticeable only at extremely high bitrates. By the way, 'sbcdec' seems to introduce quite a noticeable distortion and is orders of magnitude less precise than the encoder. Of course it would be intertesting to see if the quality estimation done by using tiny_psnr is adequate and if it can replace a 6000 EUR tool for this particular purpose. Best regards, Siarhei Siamashka
Attachment:
sbc_encode_test.rb
Description: application/ruby
