Use of RTCP statistics for estimating QoS of VoIP

[Alan Clark]

Paul

I am responding to the recent SG16 email that encouraged direct input from
those not participating directly in the work of the Study Group.

One particular recent issue relates to the recent addition of QoS reporting
based on RTCP statistics.  We have spent the last three years analyzing and
modeling the effects of IP network impairments on VoIP and would like to
share our experience on this issue.

RTCP statistics were intended to provide a "rough" real time feedback
between endpoints to give some sense of network quality and facilitate
potential adaptation.  IP network impairments are transient in nature and
the "averaged" metrics that RFC1889 produces are as a result very
misleading.  The addition of QoS reporting based on RFC1889 metrics is, in
our opinion, a backward step as it infers that these statistics are useful
for VoIP quality estimation whereas they are in fact almost meaningless.

(i) Packet Loss - counts
RFC1889 counts of loss are not compensated for duplicate packets, hence it
is possible to have incorrect and even negative (!!) loss counts.

(ii) Packet Loss- distribution
Voice quality is very sensitive to the distribution of packet loss - if 2%
loss occurs but the losses are spread out evenly then with PLC quality would
be affected only marginally, if 1% loss occurs but losses occur during short
intervals (typically the case) then quality can be affected quite
significantly.
RFC1889 does not consider the distribution of lost packets and hence packet
loss numbers can only grossly be equated to call quality (i.e. loss rate of
20% is bad but you couldn't say for sure if 2% was bad or not)

(iii) Jitter - metric
The average packet-to-packet delay variation used in RFC1889 does not
properly reflect the effects of congestion related delay changes.  For
example
(a) if congesion occurs and a queue starts to fill then delay increases
however the packet-to-packet delay change may be small however the increase
in delay may cause the jitter buffer to resynchronize at some stage.
(b) LAN congestion can cause short term spikes in delay, and hence may lead
to packet discards.  The average packet-to-packet delay variation will not
reflect this due to the smoothing effect of the averaging algorithm.

(iv) Jitter - reporting interval
As jitter is reported as a running average with a scaling factor of 16, an
RTCP report only describes the jitter level during the 200mS or so prior to
the report being generated and says nothing about the jitter level during
the much larger period between reports.

We have spoken to, and worked with, people in the industry that have been
looking at this issue in detail and have found general agreement with the
comments above.  Unfortunately, there are also a fair number of people that
blindly accept that something must be "ok" because it is "in a standard" ..
this obviously raised the concern that incorporating these metrics into the
H series of standards further proliferates this viewpoint.

Within the IETF AVT group this issue was discussed almost a year ago, and
the group agreed to develop a set of metrics that were more accurate and
appropriate for VoIP QoS reporting purposes.  These include measures of
packet loss, packets discarded due to jitter, length and density of bursts
and gaps (of combined loss and discard), delay, call quality metrics...
This is contained in the following:-

draft-ietf-avt-rtcp-report-extns-01.txt

I would very much appreciate understanding the technical rationale behind
incorporating RFC1889 based metrics into VoIP QoS reports, given the
comments above.

Regards

Alan Clark
Telchemy