I am trying to set up an audio pipeline, going over udp over wifi, with as consistent ( and hopefully minimal ) latency as possible. The goal is a constant 40ms latency. To that end I have specified a buffer time in alsasink, and added rtpjitterbuffer to limit the maximum latency, preferring short bits of silence over latency jitter. My audio source is over i2s, playng a WAV file with 1ms bursts of sine wave every 100ms, so I can measure the total latency on a scope.
Here is the pipeline:
gst-launch-1.0 -v alsasrc ! \
audio/x-raw,format=S16LE,layout=interleaved,rate=8000,channels=1 ! \
audioconvert ! \
rtpL16pay ! \
udpsink host=192.168.43.1 port=5000
gst-launch-1.0 -v udpsrc port=5000 buffer-size=1000 ! \
"application/x-rtp, media=(string)audio, clock-rate=(int)16000, encoding-name=(string)L16" ! \
rtpjitterbuffer mode=0 do-lost=true latency=28 ! rtpL16depay ! \
audio/x-raw,format=S16BE,layout=interleaved,rate=16000,channels=1 ! \
audioconvert ! \
audio/x-raw,format=S32LE,layout=interleaved,rate=16000,channels=1 ! \
alsasink sync=true buffer-time=40000
Without the rtpjitterbuffer, I get audio through, with latency varying from 18ms to 60ms or more. With rtpjitterbuffer, I get no output.
I added a debug option to the second pipeline, along with a few custom calls to GST_DEBUG_OBJECT, to isolate the problem, and I see that the buffers are getting dropped in alsasink:
GST_CLOCK gstclock.c:1187:gst_clock_get_calibration:<GstAudioSinkClock>^[[00m CUSTOM rate_num 0:00:00.000000001, rate_denom 0:00:00.000000001external_calibration 0:05:40.367852706internal_calibration 0:00:00.000000000
GST_CLOCK gstclock.c:1049:gst_clock_get_internal_time:<GstSystemClock>^[[00m internal time 0:05:40.403407831
GST_CLOCK gstclock.c:1094:gst_clock_get_time:<GstSystemClock>^[[00m adjusted time 0:05:40.403407831
audiobasesink gstaudiobasesink.c:545:gst_audio_base_sink_get_time:<alsasink0>^[[00m processed samples: raw 1120, delay 544, real 576, time 0:00:00.036000000
audioclock gstaudioclock.c:200:gst_audio_clock_get_time:<GstAudioSinkClock>^[[00m result 0:00:00.036000000, last_time 0:00:00.000000000
audiobasesink gstaudiobasesink.c:1439:gst_audio_base_sink_skew_slaving:<alsasink0>^[[00m internal 0:00:00.036000000 external 0:05:40.403407831 cinternal 0:00:00.000000000 cexternal 0:05:40.367852706
audiobasesink gstaudiobasesink.c:1460:gst_audio_base_sink_skew_slaving:<alsasink0>^[[00m internal 0:00:00.036000000 external 0:00:00.035555125 skew +0:00:00.000444875 avg +0:00:00.001856214
audiobasesink gstaudiobasesink.c:1237:clock_convert_external:<alsasink0>^[[00m CUSTOM external: 0:05:40.265906539 cexternal 0:05:40.367852706
audiobasesink gstaudiobasesink.c:1246:clock_convert_external:<alsasink0>^[[00m CUSTOM external: 0:00:00.101946167 cexternal 0:05:40.367852706
audiobasesink gstaudiobasesink.c:1252:clock_convert_external:<alsasink0>^[[00m CUSTOM external: 0:00:00.000000000
audiobasesink gstaudiobasesink.c:1237:clock_convert_external:<alsasink0>^[[00m CUSTOM external: 0:05:40.272156539 cexternal 0:05:40.367852706
audiobasesink gstaudiobasesink.c:1246:clock_convert_external:<alsasink0>^[[00m CUSTOM external: 0:00:00.095696167 cexternal 0:05:40.367852706
audiobasesink gstaudiobasesink.c:1252:clock_convert_external:<alsasink0>^[[00m CUSTOM external: 0:00:00.000000000
audiobasesink gstaudiobasesink.c:2049:gst_audio_base_sink_render:<alsasink0>^[[00m final timestamps: start 0:00:00.000000000 - stop 0:00:00.000000000
audiobasesink gstaudiobasesink.c:2219:gst_audio_base_sink_render:<alsasink0>^[[00m dropping late sample
Without rtpjitterbuffer, I do not follow the same path through clock_convert_external, so the start and stop below are not 0, and the render function does not drop the buffer.
I suspect I am doing something wrong with latency and/or buffers, but I have tried a wide range of experiments that have not led me to a better understanding of the problem. I set the latency to 28 in rtpjitterbuffer because the reported latency of the pipeline is 12ms, and I am trying to get it to a steady 40. Setting it to 28000000 nanoseconds does not work either.
This is gstreamer 1.12.2, built in yocto and running on a beaglebone black with a wifi cape.
Here is the pipeline:
gst-launch-1.0 -v alsasrc ! \
audio/x-raw,format=S16LE,layout=interleaved,rate=8000,channels=1 ! \
audioconvert ! \
rtpL16pay ! \
udpsink host=192.168.43.1 port=5000
gst-launch-1.0 -v udpsrc port=5000 buffer-size=1000 ! \
"application/x-rtp, media=(string)audio, clock-rate=(int)16000, encoding-name=(string)L16" ! \
rtpjitterbuffer mode=0 do-lost=true latency=28 ! rtpL16depay ! \
audio/x-raw,format=S16BE,layout=interleaved,rate=16000,channels=1 ! \
audioconvert ! \
audio/x-raw,format=S32LE,layout=interleaved,rate=16000,channels=1 ! \
alsasink sync=true buffer-time=40000
Without the rtpjitterbuffer, I get audio through, with latency varying from 18ms to 60ms or more. With rtpjitterbuffer, I get no output.
I added a debug option to the second pipeline, along with a few custom calls to GST_DEBUG_OBJECT, to isolate the problem, and I see that the buffers are getting dropped in alsasink:
GST_CLOCK gstclock.c:1187:gst_clock_get_calibration:<GstAudioSinkClock>^[[00m CUSTOM rate_num 0:00:00.000000001, rate_denom 0:00:00.000000001external_calibration 0:05:40.367852706internal_calibration 0:00:00.000000000
GST_CLOCK gstclock.c:1049:gst_clock_get_internal_time:<GstSystemClock>^[[00m internal time 0:05:40.403407831
GST_CLOCK gstclock.c:1094:gst_clock_get_time:<GstSystemClock>^[[00m adjusted time 0:05:40.403407831
audiobasesink gstaudiobasesink.c:545:gst_audio_base_sink_get_time:<alsasink0>^[[00m processed samples: raw 1120, delay 544, real 576, time 0:00:00.036000000
audioclock gstaudioclock.c:200:gst_audio_clock_get_time:<GstAudioSinkClock>^[[00m result 0:00:00.036000000, last_time 0:00:00.000000000
audiobasesink gstaudiobasesink.c:1439:gst_audio_base_sink_skew_slaving:<alsasink0>^[[00m internal 0:00:00.036000000 external 0:05:40.403407831 cinternal 0:00:00.000000000 cexternal 0:05:40.367852706
audiobasesink gstaudiobasesink.c:1460:gst_audio_base_sink_skew_slaving:<alsasink0>^[[00m internal 0:00:00.036000000 external 0:00:00.035555125 skew +0:00:00.000444875 avg +0:00:00.001856214
audiobasesink gstaudiobasesink.c:1237:clock_convert_external:<alsasink0>^[[00m CUSTOM external: 0:05:40.265906539 cexternal 0:05:40.367852706
audiobasesink gstaudiobasesink.c:1246:clock_convert_external:<alsasink0>^[[00m CUSTOM external: 0:00:00.101946167 cexternal 0:05:40.367852706
audiobasesink gstaudiobasesink.c:1252:clock_convert_external:<alsasink0>^[[00m CUSTOM external: 0:00:00.000000000
audiobasesink gstaudiobasesink.c:1237:clock_convert_external:<alsasink0>^[[00m CUSTOM external: 0:05:40.272156539 cexternal 0:05:40.367852706
audiobasesink gstaudiobasesink.c:1246:clock_convert_external:<alsasink0>^[[00m CUSTOM external: 0:00:00.095696167 cexternal 0:05:40.367852706
audiobasesink gstaudiobasesink.c:1252:clock_convert_external:<alsasink0>^[[00m CUSTOM external: 0:00:00.000000000
audiobasesink gstaudiobasesink.c:2049:gst_audio_base_sink_render:<alsasink0>^[[00m final timestamps: start 0:00:00.000000000 - stop 0:00:00.000000000
audiobasesink gstaudiobasesink.c:2219:gst_audio_base_sink_render:<alsasink0>^[[00m dropping late sample
Without rtpjitterbuffer, I do not follow the same path through clock_convert_external, so the start and stop below are not 0, and the render function does not drop the buffer.
I suspect I am doing something wrong with latency and/or buffers, but I have tried a wide range of experiments that have not led me to a better understanding of the problem. I set the latency to 28 in rtpjitterbuffer because the reported latency of the pipeline is 12ms, and I am trying to get it to a steady 40. Setting it to 28000000 nanoseconds does not work either.
This is gstreamer 1.12.2, built in yocto and running on a beaglebone black with a wifi cape.
_______________________________________________ gstreamer-embedded mailing list gstreamer-embedded@xxxxxxxxxxxxxxxxxxxxx https://lists.freedesktop.org/mailman/listinfo/gstreamer-embedded
