On Wed, May 26, 2010 at 10:36 PM, Thomas Falk <thfalk at gmail.com> wrote:
> Hi,
>
> thanks for the answer. I have to following situation: I want to stream
> coded speex data over rtp. I'm using a file_player, which is adapted
> from the wav_player to play speex frames out of a file. These frames
> are going through a switch board and to the stream. The stream uses
> the passthrough codec which I have extended to pass speex frames. The
> file_player serves frames of type PJMEDIA_FRAME_TYPE_EXTENDED.
>
Ok, that sounds good.
>> From: Benny Prijono <bennylp@xxxxxxxxx>
>>> A problem arises, if a processing block supplies a frame with type
>>> PJMEDIA_TYPE_NONE and the receiver has to cast this to EXTENDED. The
>>> extend frames must have its payload directly after the
>>> pjmedia_frame_ext structure. From pjmedia/types.h:
>>>
>>
>> Well, isn't the rule clear about this? You can only cast a frame to
>> pjmedia_frame_ext *if and only if* the frame type is
>> PJMEDIA_FRAME_TYPE_EXTENDED. If not, then you simply don't cast it.
>
> This rule was not clear to me. But: In the first call of the get_frame
> function of the file_player the switch board is using a frame with the
> type PJMEDIA_FRAME_TYPE_NONE. The file_player has to assume that this
> frame is actually an extended frame. It sets the type to
> PJMEDIA_FRAME_TYPE_EXTENDED and fills the fields and data. In the
> successive calls to the get_frame function the frames keeps this type.
>
> This first call using a NONE frame seems strange to me after your
> explanation. I will look into the switch board, where this comes from.
>
get_frame is probably an exception to the rule above, since with
get_frame(), basically the called port is responsible to fill in the
frame. In the switchboard case (see conf_switch.c:1254), the
switchboard gives a frame buffer which can be initialized as AUDIO,
EXTENDED, or NONE type by the port.
But you have a point there, the rule/convention about who is
responsible for what is not clearly specified at the moment. The frame
type initialization above is an example of this confusion. Another
example is with timestamp maintenance of the stream; in get_frame(),
some ports generate timestamp in the frames that they return, while
others expect the timestamp to have been initialized by caller.
>> But FYI, pjmedia_port_info has a "format" field, which tells you what
>> type of frames it expects/returns. If the format is not PCM, it
>> expects/will return extended frame.
>
> Ok. This is quite difficult to understand because there are lot of
> fields in the ports. One thing I noted in relation to this:
> There are the fields samples_per_frame and bytes_per_frame in the
> pjmedia_port_info. What is their relation if using encoded audio
> frames? The stream using a passthrough codec with speex claims
> samples_per_frame=320 and bytes_per_frame=106. Using PCMU leads to
> samples_per_frame=160 and bytes_per_frame=160. Therefore the
> bytes_per_frame seem to be the bytes per encoded frame, while the
> samples_per_frame are samples of the decoded frame.
>
That's exactly it.
> The switch board evalutes only samples_per_frame if checking when to
> deliver frames to a port. This is even if frames with
> PJMEDIA_FRAME_TYPE_EXTENDED are used to stream encoded data. The
> corresponding code is in conf_switch.c, line 1049:
>
> if (f_dst->samples_cnt >= cport_dst->info->samples_per_frame)
> {
> ? ? ? ?if (cport_dst->slot) {
> ? ? ? ? ? ? ? ?pjmedia_port_put_frame(cport_dst->port, ?(pjmedia_frame*)f_dst);
>
> In my case this leads to more than one speex frame in one rtp packet
> and to timing problems in the receiver. After changing to
>
> if (f_dst->samples_cnt >= cport_dst->info->bytes_per_frame)
>
> everything was ok.
>
> It seems to me, that if a frame of type PJMEDIA_FRAME_TYPE_EXTENED
> contains encoded data, the bytes_per_frame field of the corresponding
> port should be used.
>
I disagree. There are several reasons why the bitrate (the bytes per
second) can't be used for the above checking, e.g.: VAD, DTX, and VBR.
On the other hand, the clock rate of a port always stays the same. So
I think there's no problem here.
In your case, you receive more than one extended frames, because the
samples_per_frame in the port info says so. If you're using
narrowband/8KHz Speex codec, 320 samples is exactly 40ms, which equals
to two Speex frames.
>>> My suggestion would be to change the requirement that the data is
>>> directly behind the pjmedia_frame_ext structure to using the buffer
>>> pointer in the base structure of the pjmedia_frame_ext structure. It
>>> would then point to the sub frames in the same the way it is handeled
>>> by the simple frames. ?This would require slight changes in the
>>> functions in pjmedia/types.h, e.g. pjmedia_frame_ext_append_subframe.
>>>
>>
>> Even if the buffer requirement is changed according to above, you
>> still can't freely typecast any frames to pjmedia_frame_ext, because
>> pjmedia_frame_ext also has additional fields besides the buffer, such
>> as samples_cnt and subframe_cnt. So I don't think this would solve it.
>
> That's true. I wasn't aware of the rule above. But there seem to be
> cases where you get not an extended frame when you expect (and need)
> it. I will observe this further and give feedback if appropriate.
>
Great.
Cheers
Benny
