We have a custom board based on AM5728 SOC with omnivision 0v5640
camera installed on it running linux SDK 6.03 provided by TI. We want to
support maximum 4 party conference call using gstreamer. For POC the
following was done:-
POC was done for three party conference call. Three custom boards were taken namely A, B and C.
On A, a gstreamer pipeline was run to open self camera feed to send
it to both B and C. B and C were able to receive the feed by running
receiving pipelines on each.
Also, on B and C pipelines were run to send their respective camera feeds to A.
We want to display the feeds of both B and C at A. How can that be achieved?
The gstreamer pipelines used are as below:-
######################
Sender Pipeline at A
######################
B=192.168.8.139 #(IP address of party B)
C=192.168.8.140 #(IP address of party C)
A_BVidSendPort=5000
A_BAudSendPort=5001
A_CVidSendPort=5002
A_CAudSendPort=5003
gst-launch-1.0 -e v4l2src device=/dev/video1 io-mode=4 ! \
'video/x-raw,format=(string)YUY2,width=1280,height=720,framerate=(fraction)30/1' ! \
vpe num-input-buffers=8 ! \
queue ! ducatih264enc intra-interval=1 ! \
h264parse ! \
rtph264pay mtu=200 ! \
multiudpsink clients=$B:$A_BVidSendPort,$C:$A_CVidSendPort alsasrc device='hw:0,0' ! \
audioconvert ! \
'audio/x-raw,channels=1,depth=16,width=16,rate=16000,encoding-name=(string)L16' ! \
rtpL16pay ! \
multiudpsink clients=$B:$A_BAudSendPort,$C:$A_CAudSendPort sync=false
######################
Receiver Pipeline at A
######################
#TBD
#####################
Sender Pipeline at B
#####################
B=192.168.8.138 # (IP address of party A)
A=192.168.8.139 # (IP address of party B)
A_BVidSendPort=5004
A_BAudSendPort=5005
gst-launch-1.0 -e v4l2src device=/dev/video1 io-mode=4 ! \
'video/x-raw,format=(string)YUY2,width=1280,height=720,framerate=(fraction)30/1' ! \
vpe num-input-buffers=8 ! \
queue ! ducatih264enc intra-interval=1 ! \
h264parse ! \
rtph264pay mtu=200 ! \
udpsink host=$B port=$A_BVidSendPort alsasrc device='hw:0,0' ! \
audioconvert ! \
'audio/x-raw,channels=1,depth=16,width=16,rate=16000,encoding-name=(string)L16' ! \
rtpL16pay ! \
udpsink host=$B port=$A_BAudSendPort sync=false
########################
#Receiver Pipeline at B
########################
B_AVidSendPort=5000
B_AAudSendPort=5001
gst-launch-1.0 -v udpsrc port=$B_AVidSendPort ! \
'application/x-rtp,media=(string)video, clock-rate=(int)90000, payload=(int)96' ! \
rtph264depay ! \
h264parse ! \
ducatih264dec ! \
vpe ! \
'video/x-raw,format=(string)NV12, width=1280,height=1232' ! \
videoscale sharpen=1 ! \
kmssink sync=false \
udpsrc port=$B_AAudSendPort ! \
"application/x-rtp,media=(string)audio,clock-rate=(int)16000, width=16, height=16,\
encoding-name=(string)L16,\
encoding-params=(string)1, channels=(int)1, channel-positions=(int)1,\
payload=(int)96" !\
rtpL16depay ! audioconvert ! alsasink sync=false
#######################
#Sender pipeline at C
######################
B=192.168.8.138 # (IP address of party A)
A=192.168.8.140 # (IP address of party C)
A_BVidSendPort=5006
A_BAudSendPort=5007
gst-launch-1.0 -e v4l2src device=/dev/video1 io-mode=4 ! \
'video/x-raw,format=(string)YUY2,width=1280,height=720,framerate=(fraction)30/1' ! \
vpe num-input-buffers=8 ! \
queue ! ducatih264enc intra-interval=1 ! \
h264parse ! \
rtph264pay mtu=200 ! \
udpsink host=$B port=$A_BVidSendPort alsasrc device='hw:0,0' ! \
audioconvert ! \
'audio/x-raw,channels=1,depth=16,width=16,rate=16000,encoding-name=(string)L16' ! \
rtpL16pay ! \
udpsink host=$B port=$A_BAudSendPort sync=false
########################
#Receiver pipeline at C
#######################
B_AVidSendPort=5002
B_AAudSendPort=5003
gst-launch-1.0 -v udpsrc port=$B_AVidSendPort ! \
'application/x-rtp,media=(string)video, clock-rate=(int)90000, payload=(int)96' ! \
rtph264depay ! \
h264parse ! \
ducatih264dec ! \
vpe ! \
'video/x-raw,format=(string)NV12, width=1280,height=1232' ! \
videoscale sharpen=1 ! \
kmssink sync=false \
udpsrc port=$B_AAudSendPort ! \
"application/x-rtp,media=(string)audio,clock-rate=(int)16000, width=16, height=16,\
encoding-name=(string)L16,\
encoding-params=(string)1, channels=(int)1, channel-positions=(int)1,\
payload=(int)96" !\
rtpL16depay ! audioconvert ! alsasink sync=false
This is the POC for 3party call only. Since there are multiple
piplelines already in 3 party call, 4 party call will have furthermore
pipelines added to these.
Wouldn't it affect the system performance as the number of pipelines are increased?
Also can the sender and receiver pipelines be improved using muxing?
