-----Original Message----- From: Simon Horne [mailto:s.horne@packetizer.com]
....
Getting back to the initial question. I want to move forward with H.239 support in h323plus so can I remove the fixed frame size constraints from the video plugins so the project can move forward or if that's not recommended then, as I don't want to have different versions of the video plugins that break interoperability, can I put in a compiler directive to get us out of a pickle? Once these opal architectural glitches are resolved then the directive can be removed.
I am confused, the underlying system in OPAL has never really had any "fixed size" constraints. There are two OpalMediaOptions for width and height and they can be any value. Now come CODECS can only do fixed sizes, eg H.261, and that was part of the complexity Matthias and I were struggling with.
I really am confused on the codec issues, and the discrete video sizes with H.261/H.263 and the generic capabilities etc. The way this is done in H323plus is to detect the capabilities of the video device at application startup via the changes I made in the ptlib videodevice factory which allows the device capability list to be exposed without instantaneousing the device. You use the device capabilities list to determine the maximum frame size available for the device so in this way you can detect and support HD webcams etc. There is a H323Endpoint function that then goes through and removes all the capabilities unsupported for that particular webcam. Easy!
If I am reading this right, you are using the capabilities of the camera, which is used for transmit channels, to determine the H.323 capabilities which controls the RECEIVE channels. Surely the capability should use the VideoOutputDevice? Or is this for transmitVideo capabilities only?
Also, but what happens if you have a camera that only reports being able to do 320x240? Not sure this happens much anymore, but older cameras certainly used to. Again, if I am reading what you said right, you would not get H.261 or H.263 at all as they can't do that resolution.
On the OpenVideoChannel function callback the user can then set the frame size and fps on the wire. This sets the header height/width fields of the YUV420 frame which then goes back into the plugin codec to resize the codec. This is how it used to work in OpenH323 and it works just fine. The problem you refer to is, I guess, an open Opal issue perhaps?.
Given recent events I would be VERY careful about gibes like this.
Here is the problem as I see it, library neutral ...
Leaving out for the moment the added complexity of requiring symmetric codecs, the receive and transmit video streams are completely independent. Let's start with the receiver; there are three entities at work:
Video output device capabilities User/Application preferences Codec fundamentals
As a rule most output devices can do any resolution/frame rate, however it is possible with the YUVFile driver to indicate that it MUST be a size, e.g. using the filename "fred_qcif.yuv".
The User/Application may also have restrictions, most common is for PDA's where the screen size is such that you want to prevent 4CIF etc.
The Codec fundamentals are the most complicated as they may just be a maximum (via profile/level for H.264/MPEG4 and CustomPictureFormat for H.263+) or a set of discrete values as required by H.261 and H.263.
The presented capabilities sent to the remote must be derived from the above. I am not presenting a solution yet, just trying to state the problem in as complete a form as possible.
For the transmit side we have four entities:
Video input device capabilities User/Application preferences Codec fundamentals Remote capabilities
So, when selecting our specific video parameters we want to try and get as close to the user preference as possible given the constraints indicated by the other three entities.
One more random point: the Codec fundamentals are only known by the plug-in.
So, the result of a long discussion with Matthias (and others, but Matthias was the main person) we came up with a first cut solution. To simplify matters an assumption that the video devices can be told to go to any resolution is made. As we have the PColourConverter functionality which (mostly) can do scale/crop as it converts, I think that is fair.
Then we introduce the concept of "normalised" OpalMediaOptions and "custom" OpalMediaOptions. The normalised options are things like min/max width/height and the custom options are things like profile/level, or "QCIF MPI".
Then two functions are to be added to the plug in to set the options in an instance from normalised to custom, and back from custom to normalised options. In the process the plug-in can also apply any other rules it might have, such as discrete sizes.
And here is the sequence that I am stealing from the other thread describing how all the OpalMediaOptions get tweaked as they pass through the system:
OK, nothing better than a concrete example:
Media format on start up from a H.263 plug-in: Max Frame Width = 1408 Max Frame Height = 1152 Min Frame Width = 144 Min Frame Height = 96 Frame Width = 352 Frame Height = 288 Max Bit Rate = 384000 Target Bit Rate = 384000 Frame Time = 3000 SQCIF MPI = 1 QCIF MPI = 1 CIF MPI = 2 4CIF MPI = 3 16CIF MPI = 4 Annex D = 1
Then the user alters the following: Max Frame Width = 640 Max Frame Height = 480 Frame Width = 320 Frame Height = 240 Max Bit Rate = 128000 Frame Time = 6000
Then just before making a call the above, after user adjustment, is sent to the plug-in "from_normalised_options" function, the plug-in returns: Max Frame Width = 352 Max Frame Height = 288 Min Frame Width = 144 Min Frame Height = 96 Frame Width = 176 Frame Height = 144 Max Bit Rate = 128000 Target Bit Rate = 128000 Frame Time = 6000 SQCIF MPI = 2 QCIF MPI = 2 CIF MPI = 2 4CIF MPI = 5 16CIF MPI = 5 Annex D = 1
Note that as this particular H.263 implementation cannot do custom frame sizes, so all the frame sizes are adjusted appropriately. Some MPI's are set to an illegally large value (required for merging to work) and the new frame rate has made the MPI's that are left change upward.
This is encoded to: m=video 5002 RTP/AVP 34 a=rtpmap:34 h263/90000 a=fmtp:34 CIF=2;QCIF=2;SQCIF=2;D=1
The remote replies: m=video 5002 RTP/AVP 34 a=rtpmap:34 h263/90000 a=fmtp:34 CIF=3;4CIF=3
I deliberately tried to be a "rude" UA and return a frame size that was never offered to make sure it all works.
OPAL then constructs the following from the SDP: Max Frame Width = 640 Max Frame Height = 480 Min Frame Width = 144 Min Frame Height = 96 Frame Width = 320 Frame Height = 240 Max Bit Rate = 128000 Target Bit Rate = 128000 Frame Time = 6000 SQCIF MPI = 5 QCIF MPI = 5 CIF MPI = 3 4CIF MPI = 3 16CIF MPI = 5 Annex D = 0
Many of the above values are irrelevant at this stage and just inherited from the master format.
OPAL then merges the sent options with the received options to get: Max Frame Width = 352 Max Frame Height = 288 Min Frame Width = 144 Min Frame Height = 96 Frame Width = 176 Frame Height = 144 Max Bit Rate = 128000 Target Bit Rate = 128000 Frame Time = 6000 SQCIF MPI = 5 QCIF MPI = 5 CIF MPI = 3 4CIF MPI = 5 16CIF MPI = 5 Annex D = 0
Here the MaxMerge operator adjusts the MPI's, and the AndMerge operator turns off Annex D.
This is then passed to the plug-in "to_normalised_options" getting: Max Frame Width = 352 Max Frame Height = 288 Min Frame Width = 352 Min Frame Height = 288 Frame Width = 352 Frame Height = 288 Max Bit Rate = 128000 Target Bit Rate = 128000 Frame Time = 9009 SQCIF MPI = 5 QCIF MPI = 5 CIF MPI = 3 4CIF MPI = 5 16CIF MPI = 5 Annex D = 0
Where the Frame Size settles on its only possible value, and Frame Time gets adjusted to 10fps due to CPI MPI being 3. These options are then sent to the codec using set_codec_options and also merged with the YUV320P options so it can be used by the grabber.
Robert Jongbloed OPAL/OpenH323 Architect and Co-founder.