[h323plus] [Opalvoip-devel] Custom Video Frame Size

Simon Horne s.horne at packetizer.com
Mon Nov 5 21:55:29 EST 2007


I have CC'd this to the h323plus list.

Robert

Getting back to the initial question. I want to move forward with H.239
support in h323plus so can I remove the fixed frame size constraints from
the video plugins so the project can move forward or if that's not
recommended then, as I don't want to have different versions of the video
plugins that break interoperability, can I put in a compiler directive to
get us out of a pickle? Once these opal architectural glitches are resolved
then the directive can be removed.

I really am confused on the codec issues, and the discrete video sizes with
H.261/H.263 and the generic capabilities etc. The way this is done in
H323plus is to detect the capabilities of the video device at application
startup via the changes I made in the ptlib videodevice factory which allows
the device capability list to be exposed without instantaneousing the
device. You use the device capabilities list to determine the maximum frame
size available for the device so in this way you can detect and support HD
webcams etc. There is a H323Endpoint function that then goes through and
removes all the capabilities unsupported for that particular webcam. Easy!
On the OpenVideoChannel function callback the user can then set the frame
size and fps on the wire. This sets the header height/width fields of the
YUV420 frame which then goes back into the plugin codec to resize the codec.
This is how it used to work in OpenH323 and it works just fine. The problem
you refer to is, I guess, an open Opal issue perhaps?.

The "Extended Video Channel" is different to Hannes work in Opal.
ExtendedVideoCapability is a type of Video capability which contains a
subset of capabilites designed to be used for the likes of H.239. There is a
flag I have added to the codec definitions in the video plugins which marks
the codecs to be loaded into this subset group. Hannes's work is on having
multiple primary video windows which is not related in H.239. The secondary
or "Extended" video capability is opened via a function with sends a H.245
OLC and returns a channel number which you can then use to close the
channel. Since each channel has a unique channel number, multiple video
windows can be opened/closed on the fly.  There is a working example of this
in simple in applications directory in the H323plus CVS. This type of
concept opens the way to develop more advanced concepts like telepresence
where you can allocate 3 or more different video input for each secondary
channel. Since all this is done on a secondary video capability, existing
interoperability on the primary video is ensured and no existing
architectural changes in h323plus are required.

Simon

-----Original Message-----
From: opalvoip-devel-bounces at lists.sourceforge.net
[mailto:opalvoip-devel-bounces at lists.sourceforge.net]On Behalf Of Robert
Jongbloed
Sent: Tuesday, 6 November 2007 6:00 AM
To: Opalvoip-devel at lists.sourceforge.net
Subject: Re: [Opalvoip-devel] Custom Video Frame Size


> -----Original Message-----
> From: opalvoip-devel-bounces at lists.sourceforge.net [mailto:opalvoip-
> devel-bounces at lists.sourceforge.net] On Behalf Of Simon Horne
...
> There was some private discussion a few weeks back on setting custom
> video
> frame sizes in Opal.

I should probably summarise the result of that discussion and post the
general list.

> Currently looking at the video plugins (H.263
> for
> instance) only predefined standard frame sizes are allowed (I added a
> few
> more BTW :) ). Now with H.239 application sharing that is just not
> going to
> happen. I can just comment out the frame size checking to allow
> custom sizes
> into the encoder, and get things to work in H323plus but I thought
> I'd check
> here before commiting stuff that breaks Opal. :)

There is a branch where I started to implement the results of that huge long
thread. Unfortunately I found a show stopper with OpalMediaFormat where I
needed it to be polymorphic, but all through OPAL and H323plus is it used in
such a way that breaks polymorphism, e.g. code like:

... function(const OpalMediaFormat & videoFormat)
{
   OpalMediaFormat newFormat = videoFormat;

Which, if videoFormat an instance of OpalVideoFormat, that type is lost in
the copy constructor, including the correct virtual functions needed for
video.

This I need to fix, and soon ...

> On a side note honestly I cannot see how to set a custom frame size
> in Opal.
> Well that's not true, I can see how you can do it, if you know what
> the
> frame size is in advance but with H.239 the frame size is determined
> by the
> dimensions of the image capture (of course that changes when you
> resize the
> window) and also how do you get access to the videoInput device at
> creation
> to set the window handle for the Input device to grab so it can set
> the
> capture size in the first place. Maybe someone with more knowledge on
> this
> topic might be able to assist. :(

OK, this sounds like you are talking about changing the size of the
transmitted video in mid flight, correct?

That is one of the reasons why every internal RTP frame for raw video (of
type "YUV420P") has a small structure at the top with x,y,width,height in
it. It should be theoretically possible for the video source to change the
resolution of the generated video on a frame by frame basis. The code does
this now setting the width/height from PVideoInputDevice::GetFrameSize()
before handing to codec.

Now, your comments do raise an interesting point that is not yet allowed
for, and that is how to deal with codecs that inherently only do discrete
sizes such as H.261 and H.263. My understanding is all the others H.263+,
H.264, MPEG4 part 2, Theora (?) can all do anything so I don't think they
are an issue. Not sure how to address this one, first thought is a
OpalMediaOption (readonly) which indicates if dynamic resize is possible, if
not then something (OpalVideoConverter?) needs to scale/crop the image.

More thought needed on that one.

> For background in H323plus I created a new function
> OpenExtendedVideoChannel() to allow the user access to create and
> set,
> within the application input device (vidinput_app.h now in ptlib),
> the
> window handle to capture, then retrieve the application frame size
> from the
> input device so you can (if necessary) scale the actual frame size on
> the
> wire.

Is an "Extended Video Channel" a similar in concept to what Hannes was doing
with his OpalMediaType? The ability to open more than one video channel in a
call?

> I really would like to get some input on how you can envision this
> stuff to
> work in opal so when it comes time (or someone has the time) to port
> this
> stuff it will be a straight forward affair and not either a H.239
> code or
> architectural mashup.

My experience has been that video is never straightforward, there is always
some gotcha there. Especially, resolving the "old way" that H.261 and H.263
use (a million options in H.245) and the "new way" (profile/level) codecs
such as MPEG4 and H.264 seem to be doing it now, is proving to be hideous.
:-(


Robert Jongbloed
OPAL/OpenH323 Architect and Co-founder.





-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________
Opalvoip-devel mailing list
Opalvoip-devel at lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/opalvoip-devel




More information about the h323plus mailing list