Hi,
thanks for the answer. I have to following situation: I want to stream
coded speex data over rtp. I'm using a file_player, which is adapted
from the wav_player to play speex frames out of a file. These frames
are going through a switch board and to the stream. The stream uses
the passthrough codec which I have extended to pass speex frames. The
file_player serves frames of type PJMEDIA_FRAME_TYPE_EXTENDED.
> From: Benny Prijono <bennylp@xxxxxxxxx>
>> A problem arises, if a processing block supplies a frame with type
>> PJMEDIA_TYPE_NONE and the receiver has to cast this to EXTENDED. The
>> extend frames must have its payload directly after the
>> pjmedia_frame_ext structure. From pjmedia/types.h:
>>
>
> Well, isn't the rule clear about this? You can only cast a frame to
> pjmedia_frame_ext *if and only if* the frame type is
> PJMEDIA_FRAME_TYPE_EXTENDED. If not, then you simply don't cast it.
This rule was not clear to me. But: In the first call of the get_frame
function of the file_player the switch board is using a frame with the
type PJMEDIA_FRAME_TYPE_NONE. The file_player has to assume that this
frame is actually an extended frame. It sets the type to
PJMEDIA_FRAME_TYPE_EXTENDED and fills the fields and data. In the
successive calls to the get_frame function the frames keeps this type.
This first call using a NONE frame seems strange to me after your
explanation. I will look into the switch board, where this comes from.
> But FYI, pjmedia_port_info has a "format" field, which tells you what
> type of frames it expects/returns. If the format is not PCM, it
> expects/will return extended frame.
Ok. This is quite difficult to understand because there are lot of
fields in the ports. One thing I noted in relation to this:
There are the fields samples_per_frame and bytes_per_frame in the
pjmedia_port_info. What is their relation if using encoded audio
frames? The stream using a passthrough codec with speex claims
samples_per_frame=320 and bytes_per_frame=106. Using PCMU leads to
samples_per_frame=160 and bytes_per_frame=160. Therefore the
bytes_per_frame seem to be the bytes per encoded frame, while the
samples_per_frame are samples of the decoded frame.
The switch board evalutes only samples_per_frame if checking when to
deliver frames to a port. This is even if frames with
PJMEDIA_FRAME_TYPE_EXTENDED are used to stream encoded data. The
corresponding code is in conf_switch.c, line 1049:
if (f_dst->samples_cnt >= cport_dst->info->samples_per_frame)
{
if (cport_dst->slot) {
pjmedia_port_put_frame(cport_dst->port, (pjmedia_frame*)f_dst);
In my case this leads to more than one speex frame in one rtp packet
and to timing problems in the receiver. After changing to
if (f_dst->samples_cnt >= cport_dst->info->bytes_per_frame)
everything was ok.
It seems to me, that if a frame of type PJMEDIA_FRAME_TYPE_EXTENED
contains encoded data, the bytes_per_frame field of the corresponding
port should be used.
>> My suggestion would be to change the requirement that the data is
>> directly behind the pjmedia_frame_ext structure to using the buffer
>> pointer in the base structure of the pjmedia_frame_ext structure. It
>> would then point to the sub frames in the same the way it is handeled
>> by the simple frames. ?This would require slight changes in the
>> functions in pjmedia/types.h, e.g. pjmedia_frame_ext_append_subframe.
>>
>
> Even if the buffer requirement is changed according to above, you
> still can't freely typecast any frames to pjmedia_frame_ext, because
> pjmedia_frame_ext also has additional fields besides the buffer, such
> as samples_cnt and subframe_cnt. So I don't think this would solve it.
That's true. I wasn't aware of the rule above. But there seem to be
cases where you get not an extended frame when you expect (and need)
it. I will observe this further and give feedback if appropriate.
>
> Cheers
> ?Benny
Thanks,
Thomas
