Accessing specific frames in order to use FEC (Forward Error Connection) feature of SILK codec to recover lost frames

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi Oleg,

No, I did a very quick search after you posted and realized it was 
required, but I didn't find an elegant way to access jitter buffer.
For now, in my mind, I think that two possible ways :
  * patch pjmedia to add as parameter of the recover interface of 
pjmedia codecs, the jitter buffer. It would break all other codecs, and 
will require a patch in all other codecs so.
  * assume somebody using this codec also uses pjsua and from pjsua 
retrieve pjmedia instances to finally be able to retrieve the jitter 
buffer. Or maybe there is a static in pjmedia that we could take 
advantage of?

About the note 1, it's still necessary (I'll do that right now BTW).
In fact, there is two places where it's managed :
* the encoder : this one is managed on our side. It's the code you 
mention. In fact it's configured by what we get from the SDP. As you can 
see in the code you replaced by "...", there is something that try to 
find in the remote SDP the "useinbandfec" parameter. If this one is set 
to 1, it will configure turn the "useInBandFEC" value to 1. As per the 
silk rtp doc, if not specified this value is set to 0, if I don't 
mistake. That's why the first line set this to 0.
Then it configures the encoder so that it send in band FEC info. But 
that's just for encoder way. If the remote side doesn't announce that it 
supports FEC, the encoder will not include this information in packets 
and it will save bandwidth.
  * the decoder : this is the point that is not clean for now. Here we 
should announce in SDP what we are capable of, so that remote side 
decide what to send in encoded packet, and so that on our side we can 
take advantage or not of FEC information.
This is managed in silk_default_attr. And it's just about declare in SDP 
what we will use decoding packets. If for now we don't use FEC to 
recover frames, we should send to remote side that we don't use it (or 
don't send anything about useinbandfec, since default is 0). In this 
case, the remote side, if implemented correctly will do the same thing I 
do for encoder => detect in SDP this value, and if it's 1, will 
configure silk encoder so that it includes FEC information.
If I don't do that it will still work perfectly, but since the other 
side will think FEC is used on our side, it will send FEC info in 
packets which made some useless extra usage of bandwidth.
On the decoder side, silk announce that they are safe on the fact no FEC 
data is sent by remote side, and we should get graceful failures when 
using the decoding FEC api I think if this data is not included by 
remote side encoder.

Regards,
R?gis

On 30/05/2012 10:41, Oleg Lutsenko wrote:
> Regis,
>
> The problem is that I need to work with SILK, so I can not move to 
> Opus. Do you have any idea how to get access to the frames from jbframes?
>
> Regarding note 1, I think that your implemenatation doesn't need any 
> changes, because it looks like it already notifies remote side that 
> you do not have support of FEC:
>
> /*SKP_int32 INBandFEC_enabled = 0 ...
> ...*/
> /* silk->enc.useInBandFEC         = INBandFEC_enabled;
> */
> Regards,
> Oleg Lutsenko
>
> On Friday, May 25, 2012 at 9:20 PM, R?gis Montoya wrote:
>
>> Ok,
>>
>> You're right, in fact what I use only enable the PLC feature of SILK
>> that does extrapolation with SKP_Silk_PLC. That's why I observed that
>> PLC is done.
>>
>> So the PLC feature is enabled, but you're right the FEC data is not
>> used. I just had a closer look to opus source code and they slightly
>> modified silk so that SILK is clever and does the FEC decode by itself
>> managing buffers.
>>
>> Maybe we could get inspired of what is done in opus, but would mean
>> adding extra complexity to SILK decode process by buffering things.
>> Another solution would be to get the next frames from pjmedia jitter
>> buffer; but AFAIK, there is no clean way to get the jb frames from a
>> codec implementation, but I may be wrong.
>>
>> Side note 1 : with your remark in mind, I should fix my current
>> implementation to announce remote side that I don't use FEC info for
>> SILK codec. It will reduce packet size, since for now it's not
>> implemented, not necessary to ask remote side to send FEC info :).
>>
>> Side note 2 : opus is definitely better and easier to use. If you have
>> no constraint on the codec choice, you should maybe have a look on the
>> opus project (it includes silk and use it for low bandwidth usage
>> cases). And even if still not finalized it sounds really promising :).
>>
>> Regards,
>> R?gis
>>
>> On 25/05/2012 17:06, Oleg Lutsenko wrote:
>>> Regis,
>>>
>>> Correct, I am using your implementation, good job! Thank you!
>>>
>>> But as I understand from SILK documentation (SILK SDK API pdf 
>>> document), you should call this function
>>>
>>> SKP_Silk_SDK_search_for_LBRR
>>>
>>> to retrive lost frame from next (n+1 and n+2) frames. And in your 
>>> implementation you never call this function.
>>>
>>> Regards,
>>> Oleg Lutsenko
>>>
>>>
>>> On May 25, 2012, at 4:51 PM, R?gis Montoya wrote:
>>>
>>>> Oleg,
>>>>
>>>> I guess that you use my implementation 
>>>> (http://code.google.com/p/csipsimple/source/browse/trunk/CSipSimple/jni/silk/pj_sources/silk.c) 
>>>> of silk codec.
>>>>
>>>> As far as I understood from silk codec api, the storage of previous 
>>>> frames is done by SILK itself. So the implementation I did is 
>>>> already valid to use FEC feature of SILK.
>>>> And for the few tests I did with SILK with my implementation it's 
>>>> already capable to retrieve lost frames.
>>>>
>>>> I could mistake however since I didn't spent a lot of time on the 
>>>> SILK implementation. But my guess are pretty confirmed by the Opus 
>>>> implementation I did later (opus is based on SILK + CELT).
>>>>
>>>> So normally, you have just nothing to do if you want to use FEC 
>>>> with pjsip + my codec implementation.
>>>>
>>>> Regards,
>>>> R?gis
>>>>
>>>>
>>>> Le 25 mai 2012 16:31, "Oleg Lutsenko" <oleg.lutsenko at googlemail.com 
>>>> <mailto:oleg.lutsenko at googlemail.com>> a ?crit :
>>>> Hi guys,
>>>>
>>>> I am going to use nice FEC (Forward Error Correction) feature of 
>>>> SILK codec, to recover lost frames. To use this feature I need to 
>>>> have access to the next two frames after the lost one. This two 
>>>> frames include redundant information about previous frame, so, 
>>>> potentially, its possible to recover the lost frame.
>>>>
>>>> pjmedia_codec_op struct has recover function which I need to implement:
>>>>
>>>> pj_status_t(* pjmedia_codec_op::recover)(pjmedia_codec *codec, 
>>>> unsigned out_size, struct pjmedia_frame *output)
>>>>
>>>> The problem I have right now, is that I have no idea how to get any 
>>>> frame around the lost one. Is there any way how to get specific 
>>>> frame in a stream?
>>>>
>>>> Regards,
>>>> Oleg Lutsenko
>>>> _______________________________________________
>>>> Visit our blog: http://blog.pjsip.org
>>>>
>>>> pjsip mailing list
>>>> pjsip at lists.pjsip.org <mailto:pjsip at lists.pjsip.org>
>>>> http://lists.pjsip.org/mailman/listinfo/pjsip_lists.pjsip.org
>>>>
>>>> _______________________________________________
>>>> Visit our blog: http://blog.pjsip.org
>>>>
>>>> pjsip mailing list
>>>> pjsip at lists.pjsip.org <mailto:pjsip at lists.pjsip.org>
>>>> http://lists.pjsip.org/mailman/listinfo/pjsip_lists.pjsip.org
>>>
>>> _______________________________________________
>>> Visit our blog: http://blog.pjsip.org
>>>
>>> pjsip mailing list
>>> pjsip at lists.pjsip.org <mailto:pjsip at lists.pjsip.org>
>>> http://lists.pjsip.org/mailman/listinfo/pjsip_lists.pjsip.org
>
>
>
> _______________________________________________
> Visit our blog: http://blog.pjsip.org
>
> pjsip mailing list
> pjsip at lists.pjsip.org
> http://lists.pjsip.org/mailman/listinfo/pjsip_lists.pjsip.org


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pjsip.org/pipermail/pjsip_lists.pjsip.org/attachments/20120530/2cbce0ab/attachment-0001.html>


[Index of Archives]     [Asterisk Users]     [Asterisk App Development]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Security]     [Bugtraq]     [Linux]     [Linux OMAP]     [Linux MIPS]     [Linux API]
  Powered by Linux