Hi Branko,
The problem of test case 1 (not working on 15fps, working on 30fps) is quite strange indeed. Tried to reproduce here, three party video conf, all endpoints using profile-level-id=428028, 352x288 @15Hz, but still no luck (i.e: user 3 sees the video mix properly, initiator on left & user 2 on right).
Regarding test case 2, profile-level-id difference should only be an issue in call setup, once the video call is established properly (i.e: video nego successful in the SDP offer/answer), it should not cause problems in video conference.
As the symptoms seem to be around '"user 3 cannot see user 2 video", perhaps it was somehow related to call hold. So could you try again, but
never hold the first call. Note that when a call is on hold, the streams (audio or video) may be deactivated/removed and connecting the call stream ports in conference bridge can be useless. If the problem persists, could you also send the PJSIP log file with log level 5 too?
BR,
nanang
On Fri, Sep 6, 2019 at 4:35 PM Branko Zebec <branko.zebec@xxxxxxxxx> wrote:
Hi Nanang,I mean with high settings H264 profile, etc. profile-level-id=428028. My application use low level id.As described below I have success 3pty video call only with all three equal terminals which has the same seettings. With all another there is no video for first one. Very interesting is that only from first user which is called firstly is no video. This is always the same also with another video terminals.Two test cases:TEST1:1. All three android phones (user1, user2, user3) with my application has equal settings for H264 encoder/decoder (levelid=42801e).2. user1 is initiator and first call user 2 and then user3. Before 3pty user2 is on hold and user3 is connected with video call with user 1.3. After 3pty user2 is connected again with user1 and also user3. Function connectSessionConfVideoPoints is called.void SIP::connectSessionConfVideoPoints(int session1, int session2)
{
LogInf("session1=%d, session2=%d", session1, session2);
int pjCallId1 = this->convABSTRcallId2PJcallId(session1);
int pjCallId2 = this->convABSTRcallId2PJcallId(session2);
if (PJSUA_INVALID_ID != pjCallId1 && PJSUA_INVALID_ID != pjCallId2)
{
// Get video ports of call 1, there are two ports as in a video call as
// encoding and decoding directions may not use the same frame rate or
// resolution.
pjsua_conf_port_id call_1_dec_port = pjsua_call_get_vid_conf_port(pjCallId1, PJMEDIA_DIR_DECODING);
pjsua_conf_port_id call_1_enc_port = pjsua_call_get_vid_conf_port(pjCallId1, PJMEDIA_DIR_ENCODING);
if ((call_1_dec_port == PJSUA_INVALID_ID) || (call_1_enc_port == PJSUA_INVALID_ID)) {
LogErr("ERROR pjsua_call_get_vid_conf_port pjCallId1=%d", pjCallId1);
return;
}
// Get video ports of call 2
pjsua_conf_port_id call_2_dec_port = pjsua_call_get_vid_conf_port(pjCallId2, PJMEDIA_DIR_DECODING);
pjsua_conf_port_id call_2_enc_port = pjsua_call_get_vid_conf_port(pjCallId2, PJMEDIA_DIR_ENCODING);
if ((call_2_dec_port == PJSUA_INVALID_ID) || (call_2_enc_port == PJSUA_INVALID_ID)) {
LogErr("ERROR pjsua_call_get_vid_conf_port pjCallId2=%d", pjCallId2);
return;
}
// Connect video ports of call 1 and call 2.
// Note that the source is the stream port in decoding direction,
// and the sink is the stream port in encoding direction.
pj_status_t status = PJ_FALSE;
status = pjsua_vid_conf_connect(call_1_dec_port, call_2_enc_port, NULL);
if (status != PJ_SUCCESS)
{
LogErr("ERROR pjsua_vid_conf_connect call_1_dec_port -> call_2_enc_port");
}
status = pjsua_vid_conf_connect(call_2_dec_port, call_1_enc_port, NULL);
if (status != PJ_SUCCESS)
{
LogErr("ERROR pjsua_vid_conf_connect call_2_dec_port -> call_1_enc_port");
}
pjsua_vid_win_id wid1, wid2;
pjsua_vid_win_info win2_info;
// Put incoming video stream from call 1 into call 2 window
wid2 = pjsua_call_get_vid_win(pjCallId2);
if (wid2 == PJSUA_INVALID_ID)
{
LogErr("ERROR pjsua_call_get_vid_win pjCallId2");
return;
}
pjsua_vid_win_get_info(wid2, &win2_info);
if (status != PJ_SUCCESS)
{
LogErr("ERROR pjsua_vid_win_get_info win2_info");
}
pjsua_vid_conf_connect(call_1_dec_port, win2_info.slot_id, NULL);
if (status != PJ_SUCCESS)
{
LogErr("ERROR pjsua_vid_conf_connect call_1_dec_port -> win2_info.slot_id");
}
// Now hide the video window of call 1
wid1 = pjsua_call_get_vid_win(pjCallId1);
if (wid1 == PJSUA_INVALID_ID)
{
LogErr("ERROR pjsua_call_get_vid_win pjCallId1");
return;
}
pjsua_vid_win_set_show(wid1, PJ_FALSE);
if (status != PJ_SUCCESS)
{
LogErr("ERROR pjsua_vid_win_set_show wid1");
}
}
else
{
LogErr("ERROR PJSUA_INVALID_ID pjCallId1=%d, pjCallId2=%d", pjCallId1, pjCallId2);
}
}4. After that user3 does not see video from user2. Picture is green. But only see video from user1 which is initiator.5. If I change fps=15 to fps=30 for encoder then user3 see video from user2.This works:// For video H264With
//
// profile Baseline 42 hex - 66 dec
// profile Main 4d hex - 77 dec// profile High 64 hex - 100
int h264profileId = h264_PROFILE[1].id;
int h264profileIop = 128;
// https://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Levels
int h264level = H264_LEVELS_INFO[8].id; // 3.0 from table H264_LEVELS_INFO
int width = 352;
int height = 288;
int fps = 30;
int h264bitrate = 256; // bandwith in (kbps)
setCodecH264Profile(h264profileId, h264profileIop, h264level, width, height, fps, h264bitrate, 0);int fps = 15;it does not works.TEST2:1. Two android phones (user1, user3) with my application has equal settings for H264 encoder/decoder (levelid=42801e)2. user2 is other manufacturer and has (levelid=428028). This can't be changed.3. user1 is initiator and first call user 2 and then user3. Before 3pty user2 is on hold and user3 is connected with video call with user 1.4. After 3pty user2 is connected again with user1 and also user3. Function connectSessionConfVideoPoints is called.No setting helps here fps, resolution, levelid on user1 and user3.After 3pty user3 always does not see video from this user2 which is other manufacturer and with other settings. Picture is green.I tried a lot of different settings for:int h264level = H264_LEVELS_INFO[8].id; // 3.0 from table H264_LEVELS_INFO
int width = 352;
int height = 288;
int fps = 30;
int h264bitrate = 256; // bandwith in (kbps)And it does not help. Picture is still green.For test encoder/decoder settings is used function:/* Declaration of H.264 level info */
typedef struct h264_level_info_t
{
unsigned id; /* Level id. */
unsigned max_mbps; /* Max macroblocks per second. */
unsigned max_mb; /* Max macroblocks. */
unsigned bitrate; /* Max bitrate (kbps). */
unsigned def_w; /* Default width. */
unsigned def_h; /* Default height. */
unsigned def_fps; /* Default fps. */
} h264_level_info_t;
/* Declaration of H.264 profile info */
typedef struct h264_profile_info_t
{
unsigned idx; /* profile index */
unsigned id; /* profile id. */
char* name; /* profile name */
} h264_profile_info_t;
static const h264_level_info_t H264_LEVELS_INFO[] =
{
{ 10, 1485, 99, 64, 176, 144, 15 },
{ 9, 1485, 99, 128, 176, 144, 15 }, /*< level 1b */
{ 11, 3000, 396, 192, 320, 240, 10 },
{ 12, 6000, 396, 384, 352, 288, 15 },
{ 13, 11880, 396, 768, 352, 288, 15 },
{ 20, 11880, 396, 2000, 352, 288, 30 },
{ 21, 19800, 792, 4000, 352, 288, 30 },
{ 22, 20250, 1620, 4000, 352, 288, 30 },
{ 30, 40500, 1620, 10000, 720, 480, 30 },
{ 31, 108000, 3600, 14000, 1280, 720, 30 },
{ 32, 216000, 5120, 20000, 1280, 720, 30 },
{ 40, 245760, 8192, 20000, 1920, 1080, 30 },
{ 41, 245760, 8192, 50000, 1920, 1080, 30 },
{ 42, 522240, 8704, 50000, 1920, 1080, 30 },
{ 50, 589824, 22080, 135000, 1920, 1080, 30 },
{ 51, 983040, 36864, 240000, 1920, 1080, 30 },
};
static const h264_profile_info_t h264_PROFILE[] =
{
{ 0, 0, "No" }, // when profile="" and level=0 - "RCDO" - RCDO bitstream MUST obey to all the constraints of the Baseline profile
{ 1, 66, "Baseline" }, // 42 in-hex
{ 2, 77, "Main" }, // 4d in-hex
{ 3, 88, "Extended" }, // 58 in-hex
{ 4, 100, "High" }, // 64 in-hex
{ 5, 110, "High 10" }, // 6e in-hex
{ 6, 122, "High 4:2:2" }, // 7a in-hex
{ 7, 244, "High 4:4:4" }, // f4 in-hex
{ 8, 44, "CAVLC 4:4:4" }, // 2c in-hex
//profiles for SVC - Scalable Video Coding extension to H.264
{ 9, 83, "Scalable Baseline" }, // 53 in-hex
{ 10, 66, "Scalable High" }, // 56 in-hex
//profiles for MVC - Multiview Video Coding extension to H.264
{ 11, 128, "Stereo High" }, // 80 in-hex
{ 12, 118, "Multiview High" }, // 76 in-hex
{ 13, 138, "Multiview Depth High" }, // 8a in-hex
};// For video H264
//
// profile Baseline 42 hex - 66 dec
// profile Main 4d hex - 77 dec// profile High 64 hex - 100
int h264profileId = h264_PROFILE[1].id;
int h264profileIop = 128;
// https://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Levels
int h264level = H264_LEVELS_INFO[8].id; // 3.0 from table H264_LEVELS_INFO
int width = 352;
int height = 288;
int fps = 30;
int h264bitrate = 256; // bandwith in (kbps)
setCodecH264Profile(h264profileId, h264profileIop, h264level, width, height, fps, h264bitrate, 0);void SIPcodecManager::setCodecH264Profile(unsigned profile_id, unsigned profile_iop, unsigned level_id, unsigned width, unsigned height, unsigned fps, unsigned avg_kbps, unsigned max_kbps)
{
pj_status_t status = PJ_ENOTSUP;
#if PJMEDIA_HAS_VIDEO
LogInf("Set H264 profile %d-%d %dx%d@%d %dkbps", profile_id, level_id, width, height, fps, avg_kbps);
pjmedia_vid_codec_param param;
unsigned i;
const pj_str_t codec_id = { "H264", 4 };
const pj_str_t PROFILE_LEVEL_ID = {"profile-level-id", 16};
h264_level_info_t level_info;
int macro_block_size_sec;
char profile_level_id_str[7];
status = PJ_EINVAL;
status = pjsua_vid_codec_get_param(&codec_id, ¶m);
if(status != PJ_SUCCESS) {
return;
}
if(level_id == 0 && width > 0 && height > 0 && fps > 0){
macro_block_size_sec = ((width+15) / 16) * ((height+15) / 16 ) * fps;
int idx;
for(idx = 0; idx < PJ_ARRAY_SIZE(H264_LEVELS_INFO); ++idx){
if(H264_LEVELS_INFO[idx].max_mbps <= macro_block_size_sec){
level_id = H264_LEVELS_INFO[idx].id;
}else{
break;
}
}
}
bool ret = getH264LevelInfo( (level_id > 0) ? level_id : 30, &level_info);
if(ret != true) {
return;
}
// Check level regarding width/height parameters
if(width > 0 && height > 0 && fps > 0){
macro_block_size_sec = ((width+15) / 16) * ((height+15) / 16 ) * fps;
if(macro_block_size_sec > level_info.max_mbps){
// Invalid reg selected level
width = height = fps = 0;
}
}else{
// If we have not the 3 params, it's invalid
width = height = fps = 0;
}
LogInf("Found default infos for this level %d %dx%d@%d", level_info.id, level_info.def_w, level_info.def_h, level_info.def_fps);
param.enc_fmt.det.vid.size.w = (width > 0) ? width : level_info.def_w;
param.enc_fmt.det.vid.size.h = (height > 0) ? height : level_info.def_h;
param.enc_fmt.det.vid.fps.num = (fps > 0 ) ? fps : level_info.def_fps;
param.enc_fmt.det.vid.fps.denum = 1;
if(avg_kbps == 0){
/* H264 primer from adobe : w x h x fps x motion rank x 0.07 */
avg_kbps = ((float)(param.enc_fmt.det.vid.size.w * param.enc_fmt.det.vid.size.h * param.enc_fmt.det.vid.fps.num) ) * 0.07;
}
if(max_kbps == 0) {
max_kbps = avg_kbps;
}
param.enc_fmt.det.vid.avg_bps = ( avg_kbps <= level_info.bitrate ) ? avg_kbps * 1000 : level_info.bitrate * 1000;
param.enc_fmt.det.vid.max_bps = ( max_kbps <= level_info.bitrate ) ? max_kbps * 1000 : level_info.bitrate * 1000;
// We expect here to already have fmtp_level_profile_id
for (i = 0; i < param.dec_fmtp.cnt; ++i) {
if (pj_stricmp(¶m.dec_fmtp.param[i].name, &PROFILE_LEVEL_ID) == 0) {
if(param.dec_fmtp.param[i].val.slen == 6) {
// First copy current value
pj_memcpy(profile_level_id_str, param.dec_fmtp.param[i].val.ptr, param.dec_fmtp.param[i].val.slen * sizeof(char));
// Set profile_id
if(profile_id > 0){
pj_val_to_hex_digit(profile_id, (profile_level_id_str));
}
// Set profile_iop
if(profile_iop > 0){
pj_val_to_hex_digit(profile_iop, (profile_level_id_str+2));
}
// Set level_id
if(level_id > 0) {
pj_val_to_hex_digit(level_id, (profile_level_id_str+4));
}
profile_level_id_str[6] = '\0';
param.dec_fmtp.param[i].val = pj_str(profile_level_id_str);
LogInf("Profile is now %.*s", param.dec_fmtp.param[i].val.slen, param.dec_fmtp.param[i].val.ptr);
} else {
LogInf("Impossible to set dec_fmtp %d", param.dec_fmtp.param[i].val.slen);
}
}
}
status = pjsua_vid_codec_set_param(&codec_id, ¶m);
if(status != PJ_SUCCESS) {
return;
}
#endif
}For Android window I call:void* window is pointer from java:static Surface surfaceInVideoWindow;
static Surface surfacePreviewVideoWindow;void SIP::setVideoWindow(int windowId, void* window) {
pj_status_t status = PJ_SUCCESS;
LogInf("Entry...");
pjmedia_vid_dev_hwnd vhwnd;
vhwnd.type = PJMEDIA_VID_DEV_HWND_TYPE_ANDROID;
vhwnd.info.window = window;
if(windowId == 0) {
status = pjsua_vid_win_set_win(windowId, &vhwnd);
if (status != PJ_SUCCESS) {
LogInf("Error pjsua_vid_win_set_win");
}
} else if (windowId == 1) {
pjsua_vid_win_id wid;
pjsua_vid_win_info wi;
pjsua_vid_preview_param pre_param;
if(window == NULL) {
status = pjsua_vid_preview_stop(PJMEDIA_VID_DEFAULT_CAPTURE_DEV);
if (status != PJ_SUCCESS) {
LogInf("Error pjsua_vid_preview_stop");
}
} else {
pjsua_vid_preview_param_default(&pre_param);
pre_param.rend_id = PJMEDIA_VID_DEFAULT_RENDER_DEV;
pre_param.show = PJ_TRUE;
pre_param.wnd = vhwnd;
status = pjsua_vid_preview_start(PJMEDIA_VID_DEFAULT_CAPTURE_DEV, &pre_param);
if (status != PJ_SUCCESS) {
LogInf("Error pjsua_vid_preview_start");
}
}
} else if(windowId == 2){
status = pjsua_vid_win_set_win(windowId, &vhwnd);
if(status != PJ_SUCCESS){
LogInf("Error pjsua_vid_win_set_win");
}
}
}Thanks for the help.Best regards,Branko_______________________________________________V V pet., 6. sep. 2019 ob 07:52 je oseba Nanang Izzuddin <nanang@xxxxxxxxx> napisala:Hi Branko,It is interesting to hear any feedback on the new video conference feature. But I am afraid I don't completely understand the question, could you elaborate more about the problem, e.g: what settings is in "high settings"?If it is about frame rate & resolution, the video conference should automatically readjust the frame rate (and also resize accordingly) video data from the source ports.BR,nanang_______________________________________________On Wed, Aug 14, 2019 at 3:47 PM Branko Zebec <branko.zebec@xxxxxxxxx> wrote:Hello,_______________________________________________The problem was because of wrong decoding direction of call 1 and encoding direction of call 2 (frame rate, resolution).But question is how to solve it if we have different terminals with different settings. My Android terminal can decode with high settings, but can't encode with high settings.Thanks.Regards,Branko---------- Forwarded message ---------
Od: Branko Zebec <branko.zebec@xxxxxxxxx>
Date: V tor., 13. avg. 2019 ob 13:11
Subject: Video conference issue
To: pjsip list <pjsip@xxxxxxxxxxxxxxx>Hello,I did 3pty conference (new version of PJSIP 2.9) and second subscriber does not see right picture from first subscriber.Picture is in green color.I have it in Android project with openH264 library and with GLESv2, EGL.Test scenario:1. Initiator call subscriber 12. Between initiator and subscriber 1 is video fine.3. Initiator put subscriber 1 to HOLD4. Initiator call subscriber 25. Between initiator and subscriber 2 is video fine.6. Initiator create 3pty7. Video from initiator to subsciber 1 and subscriber 2 is fine on the left side of the window of both subscriber 1 and 2.Video from subscriber 2 is fine on the right side of the window of subscriber 1.Video from subscriber 1 is not ok and is green color on the right side of the window of subscriber 2. And this is main issue.I have equal settings on all three (bandwith, codec profile).What else could i check?Thanks for helping me.Regards,Branko
Visit our blog: http://blog.pjsip.org
pjsip mailing list
pjsip@xxxxxxxxxxxxxxx
http://lists.pjsip.org/mailman/listinfo/pjsip_lists.pjsip.org
Visit our blog: http://blog.pjsip.org
pjsip mailing list
pjsip@xxxxxxxxxxxxxxx
http://lists.pjsip.org/mailman/listinfo/pjsip_lists.pjsip.org
Visit our blog: http://blog.pjsip.org
pjsip mailing list
pjsip@xxxxxxxxxxxxxxx
http://lists.pjsip.org/mailman/listinfo/pjsip_lists.pjsip.org
_______________________________________________ Visit our blog: http://blog.pjsip.org pjsip mailing list pjsip@xxxxxxxxxxxxxxx http://lists.pjsip.org/mailman/listinfo/pjsip_lists.pjsip.org
