Hi Sergey, Thanks for your valuable insight and feedback. Let us do more testing first with this. Regards, Ming On Tue, Apr 5, 2016 at 8:14 PM, Сергей Митрофанов <goretz.m@xxxxxxxxx> wrote: > Hi Ming > > I know that old ticket, and I have some info from my experience for you > here. > I'm developing a free video chat using your superb library. Really, it is > awesome in comparison to undocumented linphone or non-sip solutions (laggy > red5, proprietary adobe air or deprecated vitamio). > But pjsip for sure has an issue with the camera orientation at least in > android platforms. > First of all we have a lot of devices to test on, so I'm talking with the > real world results in my hands now. > > For example if I use your sample app as is with the most recent svn version > of pjsip on major variety of our devices the front camera preview and stream > appears upside down in any orientation, while the back camera shows normal. > > This is because for the most devices initial rotation value from android > camera API is 90 for both front and back cameras. While other devices > (tablets, TV - from 20% to 30%) can have other values: 0 or 90 or 270 for > any of cameras. > > As it is obvious from pjsip code, you don't use at all the actual camera > rotation from android hardware.camera API. > Looks like you suggested "all back cameras are always rotated for 90 and all > front cameras are always rotated for 270". And that why you map android > device orientation with 90 offset to the library orientation in the test > app. > But in fact this simple mapping is not always true and device can also have > external plugged in cameras (in case of TVs). > > So to make it work correctly I have to make strange crutches (And I done it > before) to figure out the real initial rotation of device and fix the > orientation before passing it to the library. And since there is no straight > mapping between android cameras IDs and pjsip capture devices IDs it becomes > very ugly and unreliable. > Speaking specifically that how I had solve this issue before to make it work > as intended on all devices: > 1. Get the list of android cameras from hardware.camera API > 2. Get rotation for front and back cameras (only 2 values, just for front > and back, since we have no straight mapping to pjsip devices here) > 3. Tweak rotation values: rotationBack = 90 - cameraInfo.orientation and > rotationFront = 270 - cameraInfo.orientation > 4. Get list of the pjsip devices > 5. Create an Array of POJOs to keep the rotation values for pjsip devices > 6. Assign the rotationBack to the pjsip devices that contain in their name > the word "back" > 7. Assign the rotationFront to the pjsip devices that contain in their name > the word "front" > 8. Remove non-camera devices (driver is not "Android" or direction is not > "PJMEDIA_DIR_ENCODING") > And we have a list of pjsip camera IDs with their rotations values packed in > POJOs > Then at the time of rotation those one used like this: > 1. figure out the actualRotation = screenOrientation - cameraRotation (from > camera POJO - either rotationBack or rotationFront) > 2. Normalize the result value, so it will be between 0 and 360 (+ or - 360 > to the value) > 3. Make the same mapping as in your sample application (0 = 270DEG, 90 = > NATUAL etc.) > As you see it is a long and unstable way to achieve this. > After tweaking the android_dev.c I now can use the camera rotation > functionality without any extra crutches and wheels. And it works on any of > android devices that is out there now. > > The main issue in the current pjsip work with cameras - it does not respect > the real initial rotation of camera devices. > If this is not a problem for other platforms it doesn't mean that there is > no problem at all. > And if you want to keep behavior the same with other platforms to the > prejudice of library usability on other platforms, then it appears sad for > me... > > Anyway I just pointed out the big annoying issue with current camera > rotation calculations scheme on Android devices and I don't force you to use > this solution for the main branch. > Maybe later you will find a nice solution that will work as simple as with > my patch (without any excessive code-magic) across all platforms supported > by your team. And I will be very glad when you will make it and will make a > standing ovation for you guys! > But until this happen I will use this patch. > Maybe it will be useful for someone else, that why I publish it here... > > Best Regards, > Sergey > > вт, 5 апр. 2016 г. в 13:06, Ming <ming@xxxxxxxxx>: >> >> Hi Сергей, >> >> Unfortunately this will make the behavior different with other >> platforms. And FYI, we just changed the spec of pjmedia_orient in >> ticket #1880 (https://trac.pjsip.org/repos/ticket/1880) --> >> >> https://trac.pjsip.org/repos/browser/pjproject/trunk/pjmedia/include/pjmedia/types.h#L197 >> >> Regards, >> Ming >> >> On Mon, Apr 4, 2016 at 7:42 PM, Сергей Митрофанов <goretz.m@xxxxxxxxx> >> wrote: >> > Yesterday I have published a patch, but today rechecked and it produces >> > incorrect rotations for back camera. >> > Here is the correct one patch. >> > It works without any additional rotation offset calculations in app and >> > with >> > it pjsip rotations are mapped to android screen orientations directly. >> > >> > Was in test app: >> > >> > wm = (WindowManager)this.getSystemService(Context.WINDOW_SERVICE); >> > display = wm.getDefaultDisplay(); >> > rotation = display.getRotation(); >> > Log.d("CallActivity", "Device orientation changed: " + rotation); >> > switch (rotation) { >> > case Surface.ROTATION_0: // Portrait >> > orient = pjmedia_orient.PJMEDIA_ORIENT_ROTATE_270DEG; >> > break; >> > case Surface.ROTATION_90: // Landscape, home button on the right >> > orient = pjmedia_orient.PJMEDIA_ORIENT_NATURAL; >> > break; >> > case Surface.ROTATION_180: >> > orient = pjmedia_orient.PJMEDIA_ORIENT_ROTATE_90DEG; >> > break; >> > case Surface.ROTATION_270: // Landscape, home button on the left >> > orient = pjmedia_orient.PJMEDIA_ORIENT_ROTATE_180DEG; >> > break; >> > default: >> > orient = pjmedia_orient.PJMEDIA_ORIENT_UNKNOWN; >> > } >> > >> > With this patch: >> > >> > wm = (WindowManager)this.getSystemService(Context.WINDOW_SERVICE); >> > display = wm.getDefaultDisplay(); >> > rotation = display.getRotation(); >> > Log.d("CallActivity", "Device orientation changed: " + rotation); >> > switch (rotation) { >> > case Surface.ROTATION_0: // Portrait >> > orient = pjmedia_orient.PJMEDIA_ORIENT_NATURAL; >> > break; >> > case Surface.ROTATION_90: // Landscape, home button on the right >> > orient = pjmedia_orient.PJMEDIA_ORIENT_ROTATE_90DEG; >> > break; >> > case Surface.ROTATION_180: >> > orient = pjmedia_orient.PJMEDIA_ORIENT_ROTATE_180DEG; >> > break; >> > case Surface.ROTATION_270: // Landscape, home button on the left >> > orient = pjmedia_orient.PJMEDIA_ORIENT_ROTATE_270DEG; >> > break; >> > default: >> > orient = pjmedia_orient.PJMEDIA_ORIENT_UNKNOWN; >> > } >> > >> > And this works correct for both facing and back cameras without any >> > additional work in application. >> > >> > >> > вс, 3 апр. 2016 г. в 17:25, Сергей Митрофанов <goretz.m@xxxxxxxxx>: >> >> >> >> Hi! >> >> >> >> I have found that pjsip does not respect initial camera rotation in >> >> android devices. >> >> Sure the android_dev.c itself extracts the rotation value from android >> >> camera devices, >> >> but for now this value is actually not used, resulting wrong capture >> >> video >> >> orientation. >> >> I made a simple patch (a little bit raw code style) that resolves this >> >> bug. >> >> It takes the camera rotation value in the game and make PJSUA2 easier >> >> to >> >> use - developer need no more think about camera initial rotation and >> >> tweak >> >> the rotation value before giving it to PJSUA2 itself. >> >> >> >> Here is the diff patch for the changes: >> >> >> >> From 1ea38ce9103f6fadd9812f635c83442994fd37ea Mon Sep 17 00:00:00 2001 >> >> >> >> From: Sergey Mitrofanov <GOretZ.M@xxxxxxxxx> >> >> >> >> Date: Sun, 3 Apr 2016 17:21:48 +0300 >> >> >> >> Subject: [PATCH] Fixing to respect initial camera rotation in android >> >> devices. >> >> >> >> >> >> --- >> >> >> >> pjmedia/src/pjmedia-videodev/android_dev.c | 36 >> >> ++++++++++++++++++++++++++++++ >> >> >> >> 1 file changed, 36 insertions(+) >> >> >> >> >> >> diff --git a/pjmedia/src/pjmedia-videodev/android_dev.c >> >> b/pjmedia/src/pjmedia-videodev/android_dev.c >> >> >> >> index 1da7261..fd6e61d 100644 >> >> >> >> --- a/pjmedia/src/pjmedia-videodev/android_dev.c >> >> >> >> +++ b/pjmedia/src/pjmedia-videodev/android_dev.c >> >> >> >> @@ -72,6 +72,7 @@ typedef struct and_dev_info >> >> >> >> pjmedia_vid_dev_info info; /**< Base info */ >> >> >> >> unsigned dev_idx; /**< Original dev ID */ >> >> >> >> pj_bool_t facing; /**< Front/back camera?*/ >> >> >> >> + unsigned rotation; /**< initial camera rotation*/ >> >> >> >> unsigned sup_size_cnt; /**< # of supp'd size */ >> >> >> >> pjmedia_rect_size *sup_size; /**< Supported size */ >> >> >> >> unsigned sup_fps_cnt; /**< # of supp'd FPS */ >> >> >> >> @@ -532,6 +533,8 @@ static pj_status_t >> >> and_factory_refresh(pjmedia_vid_dev_factory *ff) >> >> >> >> } else { >> >> >> >> pj_ansi_strncpy(vdi->name, "Front camera", sizeof(vdi->name)); >> >> >> >> } >> >> >> >> + adi->rotation = (*jni_env)->GetIntField(jni_env, jdev_info, >> >> >> >> + jobjs.cam_info.f_orient); >> >> >> >> >> >> >> >> /* Get supported sizes */ >> >> >> >> jtmp = (*jni_env)->GetObjectField(jni_env, jdev_info, >> >> >> >> @@ -1001,6 +1004,39 @@ static pj_status_t >> >> and_stream_set_cap(pjmedia_vid_dev_stream *s, >> >> >> >> else if (eff_ori == PJMEDIA_ORIENT_ROTATE_270DEG) >> >> >> >> eff_ori = PJMEDIA_ORIENT_ROTATE_90DEG; >> >> >> >> } >> >> >> >> + /* Normalize the orientation for rotated camera */ >> >> >> >> + switch(adi->rotation){ >> >> >> >> + case 90: >> >> >> >> + if (eff_ori == PJMEDIA_ORIENT_ROTATE_90DEG) >> >> >> >> + eff_ori = PJMEDIA_ORIENT_ROTATE_180DEG; >> >> >> >> + else if (eff_ori == PJMEDIA_ORIENT_ROTATE_180DEG) >> >> >> >> + eff_ori = PJMEDIA_ORIENT_ROTATE_270DEG; >> >> >> >> + else if (eff_ori == PJMEDIA_ORIENT_ROTATE_270DEG) >> >> >> >> + eff_ori = PJMEDIA_ORIENT_NATURAL; >> >> >> >> + else if (eff_ori == PJMEDIA_ORIENT_NATURAL) >> >> >> >> + eff_ori = PJMEDIA_ORIENT_ROTATE_90DEG; >> >> >> >> + break; >> >> >> >> + case 180: >> >> >> >> + if (eff_ori == PJMEDIA_ORIENT_ROTATE_90DEG) >> >> >> >> + eff_ori = PJMEDIA_ORIENT_ROTATE_270DEG; >> >> >> >> + else if (eff_ori == PJMEDIA_ORIENT_ROTATE_180DEG) >> >> >> >> + eff_ori = PJMEDIA_ORIENT_NATURAL; >> >> >> >> + else if (eff_ori == PJMEDIA_ORIENT_ROTATE_270DEG) >> >> >> >> + eff_ori = PJMEDIA_ORIENT_ROTATE_90DEG; >> >> >> >> + else if (eff_ori == PJMEDIA_ORIENT_NATURAL) >> >> >> >> + eff_ori = PJMEDIA_ORIENT_ROTATE_180DEG; >> >> >> >> + break; >> >> >> >> + case 270: >> >> >> >> + if (eff_ori == PJMEDIA_ORIENT_ROTATE_90DEG) >> >> >> >> + eff_ori = PJMEDIA_ORIENT_NATURAL; >> >> >> >> + else if (eff_ori == PJMEDIA_ORIENT_ROTATE_180DEG) >> >> >> >> + eff_ori = PJMEDIA_ORIENT_ROTATE_90DEG; >> >> >> >> + else if (eff_ori == PJMEDIA_ORIENT_ROTATE_270DEG) >> >> >> >> + eff_ori = PJMEDIA_ORIENT_ROTATE_180DEG; >> >> >> >> + else if (eff_ori == PJMEDIA_ORIENT_NATURAL) >> >> >> >> + eff_ori = PJMEDIA_ORIENT_ROTATE_270DEG; >> >> >> >> + break; >> >> >> >> + } >> >> >> >> pjmedia_vid_dev_conv_set_rotation(&strm->conv, eff_ori); >> >> >> >> >> >> >> >> PJ_LOG(4, (THIS_FILE, "Video capture orientation set to %d", >> >> >> >> -- >> >> >> >> 2.6.4 (Apple Git-63) >> >> >> >> >> > >> > _______________________________________________ >> > Visit our blog: http://blog.pjsip.org >> > >> > pjsip mailing list >> > pjsip@xxxxxxxxxxxxxxx >> > http://lists.pjsip.org/mailman/listinfo/pjsip_lists.pjsip.org >> > >> >> _______________________________________________ >> Visit our blog: http://blog.pjsip.org >> >> pjsip mailing list >> pjsip@xxxxxxxxxxxxxxx >> http://lists.pjsip.org/mailman/listinfo/pjsip_lists.pjsip.org > > > _______________________________________________ > Visit our blog: http://blog.pjsip.org > > pjsip mailing list > pjsip@xxxxxxxxxxxxxxx > http://lists.pjsip.org/mailman/listinfo/pjsip_lists.pjsip.org > _______________________________________________ Visit our blog: http://blog.pjsip.org pjsip mailing list pjsip@xxxxxxxxxxxxxxx http://lists.pjsip.org/mailman/listinfo/pjsip_lists.pjsip.org
