<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
<META NAME="Generator" CONTENT="MS Exchange Server version 5.5.2653.12">
<TITLE>AW: Use of RTCP statistics for estimating QoS of VoIP</TITLE>
</HEAD>
<BODY>
<P><FONT SIZE=2>Alan,</FONT>
</P>
<P><FONT SIZE=2>I fully agree with Bob's reply below. In my view H.460.9 is now sufficiently open to be used for transporting also advanced statistical data, even if the initial choice reflects the content of today's RTCP reports. </FONT></P>
<P><FONT SIZE=2>The information content of QoS reports is only one part of the standard. Another - maybe the major - part is the mechanism for conveying these reports from endpoints to gatekeepers and between gatekeepers. And this part does not change if improved QoS statistics become available.</FONT></P>
<P><FONT SIZE=2>It was the unanimous opinion of the experts at the SG16 meeting that H.460.9 will be used for extended QoS reports as soon as the relevant work in IETF has finished. How this is best done will be decided when the RFC is available. Any input from your side is certainly welcome.</FONT></P>
<P><FONT SIZE=2>Regards,</FONT>
<BR><FONT SIZE=2>Ernst Horvath</FONT>
<BR><FONT SIZE=2>Siemens AG</FONT>
</P>
<P><FONT SIZE=2>> -----Ursprüngliche Nachricht-----</FONT>
<BR><FONT SIZE=2>> Von: Robert R. Gilman [<A HREF="mailto:rrg@AVAYA.COM">mailto:rrg@AVAYA.COM</A>]</FONT>
<BR><FONT SIZE=2>> Gesendet am: Donnerstag, 07. November 2002 02:13</FONT>
<BR><FONT SIZE=2>> An: ITU-SG16@echo.jf.INTEL.COM</FONT>
<BR><FONT SIZE=2>> Betreff: Re: Use of RTCP statistics for estimating QoS of VoIP</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> Alan-</FONT>
<BR><FONT SIZE=2>> Mike Buckley brought up the same questions you raise at the </FONT>
<BR><FONT SIZE=2>> last SG16 meeting.</FONT>
<BR><FONT SIZE=2>> I don't believe anyone objects to your arguments, and, in </FONT>
<BR><FONT SIZE=2>> fact, the resulting</FONT>
<BR><FONT SIZE=2>> work, H.460.9, was modified to permit extension to other </FONT>
<BR><FONT SIZE=2>> measures and the</FONT>
<BR><FONT SIZE=2>> exclusion of the current, IETF-standardized, measures. Once </FONT>
<BR><FONT SIZE=2>> the IETF draft</FONT>
<BR><FONT SIZE=2>> you reference becomes a standard, it will be easy to refer to </FONT>
<BR><FONT SIZE=2>> the resulting</FONT>
<BR><FONT SIZE=2>> RFC and to incorporate the resulting measures as appropriate. </FONT>
<BR><FONT SIZE=2>> In the meantime,</FONT>
<BR><FONT SIZE=2>> you have the option to use the existing measures and, through </FONT>
<BR><FONT SIZE=2>> the definition</FONT>
<BR><FONT SIZE=2>> of non-standard elements, any other measures you wish to </FONT>
<BR><FONT SIZE=2>> implement on a</FONT>
<BR><FONT SIZE=2>> permanent</FONT>
<BR><FONT SIZE=2>> or trial basis. It seemed better to set the framework now, so that</FONT>
<BR><FONT SIZE=2>> implementation</FONT>
<BR><FONT SIZE=2>> and use can begin, and to do so in such a way that future </FONT>
<BR><FONT SIZE=2>> improvements may be</FONT>
<BR><FONT SIZE=2>> incorporated quickly and easily. Does this make sense?</FONT>
<BR><FONT SIZE=2>> -Bob</FONT>
<BR><FONT SIZE=2>> ----------------------------------------------------</FONT>
<BR><FONT SIZE=2>> Bob Gilman rrg@avaya.com +1 303 538 3868</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> Alan Clark wrote:</FONT>
<BR><FONT SIZE=2>> ></FONT>
<BR><FONT SIZE=2>> > Paul</FONT>
<BR><FONT SIZE=2>> ></FONT>
<BR><FONT SIZE=2>> > I am responding to the recent SG16 email that encouraged </FONT>
<BR><FONT SIZE=2>> direct input from</FONT>
<BR><FONT SIZE=2>> > those not participating directly in the work of the Study Group.</FONT>
<BR><FONT SIZE=2>> ></FONT>
<BR><FONT SIZE=2>> > One particular recent issue relates to the recent addition </FONT>
<BR><FONT SIZE=2>> of QoS reporting</FONT>
<BR><FONT SIZE=2>> > based on RTCP statistics. We have spent the last three </FONT>
<BR><FONT SIZE=2>> years analyzing and</FONT>
<BR><FONT SIZE=2>> > modeling the effects of IP network impairments on VoIP and </FONT>
<BR><FONT SIZE=2>> would like to</FONT>
<BR><FONT SIZE=2>> > share our experience on this issue.</FONT>
<BR><FONT SIZE=2>> ></FONT>
<BR><FONT SIZE=2>> > RTCP statistics were intended to provide a "rough" real </FONT>
<BR><FONT SIZE=2>> time feedback</FONT>
<BR><FONT SIZE=2>> > between endpoints to give some sense of network quality and </FONT>
<BR><FONT SIZE=2>> facilitate</FONT>
<BR><FONT SIZE=2>> > potential adaptation. IP network impairments are transient </FONT>
<BR><FONT SIZE=2>> in nature and</FONT>
<BR><FONT SIZE=2>> > the "averaged" metrics that RFC1889 produces are as a result very</FONT>
<BR><FONT SIZE=2>> > misleading. The addition of QoS reporting based on RFC1889 </FONT>
<BR><FONT SIZE=2>> metrics is, in</FONT>
<BR><FONT SIZE=2>> > our opinion, a backward step as it infers that these </FONT>
<BR><FONT SIZE=2>> statistics are useful</FONT>
<BR><FONT SIZE=2>> > for VoIP quality estimation whereas they are in fact almost </FONT>
<BR><FONT SIZE=2>> meaningless.</FONT>
<BR><FONT SIZE=2>> ></FONT>
<BR><FONT SIZE=2>> > (i) Packet Loss - counts</FONT>
<BR><FONT SIZE=2>> > RFC1889 counts of loss are not compensated for duplicate </FONT>
<BR><FONT SIZE=2>> packets, hence it</FONT>
<BR><FONT SIZE=2>> > is possible to have incorrect and even negative (!!) loss counts.</FONT>
<BR><FONT SIZE=2>> ></FONT>
<BR><FONT SIZE=2>> > (ii) Packet Loss- distribution</FONT>
<BR><FONT SIZE=2>> > Voice quality is very sensitive to the distribution of </FONT>
<BR><FONT SIZE=2>> packet loss - if 2%</FONT>
<BR><FONT SIZE=2>> > loss occurs but the losses are spread out evenly then with </FONT>
<BR><FONT SIZE=2>> PLC quality would</FONT>
<BR><FONT SIZE=2>> > be affected only marginally, if 1% loss occurs but losses </FONT>
<BR><FONT SIZE=2>> occur during short</FONT>
<BR><FONT SIZE=2>> > intervals (typically the case) then quality can be affected quite</FONT>
<BR><FONT SIZE=2>> > significantly.</FONT>
<BR><FONT SIZE=2>> > RFC1889 does not consider the distribution of lost packets </FONT>
<BR><FONT SIZE=2>> and hence packet</FONT>
<BR><FONT SIZE=2>> > loss numbers can only grossly be equated to call quality </FONT>
<BR><FONT SIZE=2>> (i.e. loss rate of</FONT>
<BR><FONT SIZE=2>> > 20% is bad but you couldn't say for sure if 2% was bad or not)</FONT>
<BR><FONT SIZE=2>> ></FONT>
<BR><FONT SIZE=2>> > (iii) Jitter - metric</FONT>
<BR><FONT SIZE=2>> > The average packet-to-packet delay variation used in </FONT>
<BR><FONT SIZE=2>> RFC1889 does not</FONT>
<BR><FONT SIZE=2>> > properly reflect the effects of congestion related delay </FONT>
<BR><FONT SIZE=2>> changes. For</FONT>
<BR><FONT SIZE=2>> > example</FONT>
<BR><FONT SIZE=2>> > (a) if congesion occurs and a queue starts to fill then </FONT>
<BR><FONT SIZE=2>> delay increases</FONT>
<BR><FONT SIZE=2>> > however the packet-to-packet delay change may be small </FONT>
<BR><FONT SIZE=2>> however the increase</FONT>
<BR><FONT SIZE=2>> > in delay may cause the jitter buffer to resynchronize at some stage.</FONT>
<BR><FONT SIZE=2>> > (b) LAN congestion can cause short term spikes in delay, </FONT>
<BR><FONT SIZE=2>> and hence may lead</FONT>
<BR><FONT SIZE=2>> > to packet discards. The average packet-to-packet delay </FONT>
<BR><FONT SIZE=2>> variation will not</FONT>
<BR><FONT SIZE=2>> > reflect this due to the smoothing effect of the averaging algorithm.</FONT>
<BR><FONT SIZE=2>> ></FONT>
<BR><FONT SIZE=2>> > (iv) Jitter - reporting interval</FONT>
<BR><FONT SIZE=2>> > As jitter is reported as a running average with a scaling </FONT>
<BR><FONT SIZE=2>> factor of 16, an</FONT>
<BR><FONT SIZE=2>> > RTCP report only describes the jitter level during the </FONT>
<BR><FONT SIZE=2>> 200mS or so prior to</FONT>
<BR><FONT SIZE=2>> > the report being generated and says nothing about the </FONT>
<BR><FONT SIZE=2>> jitter level during</FONT>
<BR><FONT SIZE=2>> > the much larger period between reports.</FONT>
<BR><FONT SIZE=2>> ></FONT>
<BR><FONT SIZE=2>> > We have spoken to, and worked with, people in the industry </FONT>
<BR><FONT SIZE=2>> that have been</FONT>
<BR><FONT SIZE=2>> > looking at this issue in detail and have found general </FONT>
<BR><FONT SIZE=2>> agreement with the</FONT>
<BR><FONT SIZE=2>> > comments above. Unfortunately, there are also a fair </FONT>
<BR><FONT SIZE=2>> number of people that</FONT>
<BR><FONT SIZE=2>> > blindly accept that something must be "ok" because it is </FONT>
<BR><FONT SIZE=2>> "in a standard" ..</FONT>
<BR><FONT SIZE=2>> > this obviously raised the concern that incorporating these </FONT>
<BR><FONT SIZE=2>> metrics into the</FONT>
<BR><FONT SIZE=2>> > H series of standards further proliferates this viewpoint.</FONT>
<BR><FONT SIZE=2>> ></FONT>
<BR><FONT SIZE=2>> > Within the IETF AVT group this issue was discussed almost a </FONT>
<BR><FONT SIZE=2>> year ago, and</FONT>
<BR><FONT SIZE=2>> > the group agreed to develop a set of metrics that were more </FONT>
<BR><FONT SIZE=2>> accurate and</FONT>
<BR><FONT SIZE=2>> > appropriate for VoIP QoS reporting purposes. These include </FONT>
<BR><FONT SIZE=2>> measures of</FONT>
<BR><FONT SIZE=2>> > packet loss, packets discarded due to jitter, length and </FONT>
<BR><FONT SIZE=2>> density of bursts</FONT>
<BR><FONT SIZE=2>> > and gaps (of combined loss and discard), delay, call </FONT>
<BR><FONT SIZE=2>> quality metrics...</FONT>
<BR><FONT SIZE=2>> > This is contained in the following:-</FONT>
<BR><FONT SIZE=2>> ></FONT>
<BR><FONT SIZE=2>> > draft-ietf-avt-rtcp-report-extns-01.txt</FONT>
<BR><FONT SIZE=2>> ></FONT>
<BR><FONT SIZE=2>> > I would very much appreciate understanding the technical </FONT>
<BR><FONT SIZE=2>> rationale behind</FONT>
<BR><FONT SIZE=2>> > incorporating RFC1889 based metrics into VoIP QoS reports, given the</FONT>
<BR><FONT SIZE=2>> > comments above.</FONT>
<BR><FONT SIZE=2>> ></FONT>
<BR><FONT SIZE=2>> > Regards</FONT>
<BR><FONT SIZE=2>> ></FONT>
<BR><FONT SIZE=2>> > Alan Clark</FONT>
<BR><FONT SIZE=2>> > Telchemy</FONT>
<BR><FONT SIZE=2>> ></FONT>
<BR><FONT SIZE=2>> > </FONT>
<BR><FONT SIZE=2>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~</FONT>
<BR><FONT SIZE=2>> > For help on this mail list, send "HELP ITU-SG16" in a message to</FONT>
<BR><FONT SIZE=2>> > listserv@lists.intel.com</FONT>
<BR><FONT SIZE=2>> </FONT>
<BR><FONT SIZE=2>> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~</FONT>
<BR><FONT SIZE=2>> For help on this mail list, send "HELP ITU-SG16" in a message to</FONT>
<BR><FONT SIZE=2>> listserv@lists.intel.com</FONT>
<BR><FONT SIZE=2>> </FONT>
</P>
</BODY>
</HTML>