VoIP

Book Excerpt:
Authorized Self-Study Guide: Cisco Voice over IP (CVoice), Second Edition

In this book excerpt, the author describes the metrics used to measure voice quality on the internet.

by Kevin Wallace in Authorized Self-Study Guide, Cisco Voice over IP (CVoice), Second Edition, published by Cisco Press [October 24, 2006]

This is excerpt is from Chapter 7: Improving and Maintaining Voice Quality, pp. 373-381 of Authorized Self-Study Guide, Cisco Voice over IP (CVoice), Second Edition, by Cisco Press

Quality Metrics

Quality must be measurable in order to be manageable. Three quality metrics include the Mean Opinion Score (MOS), the Perceptual Speech Quality Measurement (PSQM), and the Perceptual Evaluation of Speech Quality (PESQ).

MOS
MOS is a scoring system for voice quality. An MOS score is generated when listeners evaluate prerecorded sentences that are subject to varying conditions, such as compression algorithms. Listeners then assign the sentences values, based on a scale from 1 to 5, where 1 is the worst and 5 is the best. The sentence used for English language MOS testing is, "Nowadays, a chicken leg is a rare dish." This sentence is used because it contains a wide range of sounds found in human speech, such as long vowels, short vowels, hard sounds, and soft sounds.

The test scores are then averaged to a composite score. The test results are subjective because they are based on the opinions of the listeners. The tests are also relative, because a score of 3.8 from one test cannot be directly compared to a score of 3.8 from another test. Therefore, a baseline needs to be established for all tests, such as G.711, so that the scores can be normalized and compared directly.

PSQM
PSQM is an automated method of measuring speech quality "in service," or as the speech happens. PSQM software usually resides with IP call-management systems, which are sometimes integrated into Simple Network Management Protocol (SNMP) systems.

Equipment and software that can measure PSQM is available through third-party vendors but is not implemented in Cisco devices. The PSQM measurement is made by comparing the original transmitted speech to the resulting speech at the far end of the transmission channel. PSQM systems are deployed as in-service components. The PSQM measurements are made during real conversation on the network. This automated testing algorithm has over 90 percent accuracy compared to subjective listening tests, such as MOS. Scoring is based on a scale from 0 to 6.5, where 0 is the best and 6.5 is the worst. Because it was originally designed for circuit-switched voice, PSQM does not take into account the jitter or delay problems that are experienced in packet-switched voice systems.

PESQ
MOS and PSQM are not recommended for present-day VoIP networks. Both were originally designed before the emergence of VoIP technologies and do not measure typical VoIP problems such as jitter and delay. For example, it is possible to obtain an MOS score of 3.8 on a VoIP network when the one-way delay exceeds 500 ms, because the MOS evaluator has no concept of a two-way conversation and listens only to audio quality. The one-way delay is not evaluated.

PESQ, whose operation is illustrated in Figure 7-4, was originally developed by British Telecom, Psytechnics, and KPN Research of the Netherlands. It has evolved into ITU Standard P.862, which is considered the current standard for voice quality measurement. PESQ can take into account CODEC errors, filtering errors, jitter problems, and delay problems that are typical in a VoIP network. PESQ combines the best of the PSQM method along with a method called Perceptual Analysis Measurement System (PAMS). PESQ scores range from 1 (worst) to 4.5 (best), with 3.8 considered "toll quality" (that is, acceptable quality in a traditional telephony network). PESQ is meant to measure only one aspect of voice quality. The effects of two-way communication, such as loudness loss, delay, echo, and sidetone, are not reflected in PESQ scores.

Figure 7-4 PESQ

Many equipment vendors offer PESQ measurement systems. Such systems are either stand-alone or they plug into existing network management systems. PESQ was designed to mirror the MOS measurement system. So, if a score of 3.2 is measured by PESQ, a score of 3.2 should be achieved using MOS methods.

Quality Measurement Comparison
Early quality measurement methods, such as MOS and PSQM, were designed before widespread acceptance of VoIP technology. PESQ was designed to address the shortcomings of MOS and PSQM.

MOS uses subjective testing where the average opinion of a group of test users is calculated to create the MOS score. This method is both time-consuming and expensive, and might not provide consistent results between groups of testers.

PSQM and PESQ use objective testing where an original reference file sent into the system is compared with the impaired signal that came out. This testing method provides an automated test mechanism that does not rely on human interpretation for result calculations. However, PSQM was originally designed for circuit-switched networks and does not take into account the effects of jitter and packet loss.

PESQ measures the effect of end-to-end network conditions, including CODEC processing, jitter, and packet loss. Therefore, PESQ is the preferred method of testing voice quality in an IP network. Table 7-3 offers a comparison of the various quality metrics.

Table 7-3 Voice Quality Measurement Comparison

Feature	MOS	PSQM	PESQ
Test method	Subjective	Objective	Objective
End-to-end packet test	Inconsistent	No	Yes
End-to-end jitter test	Inconsistent	No	Yes

Reproduced from the book Authorized Self-Study Guide, Cisco Voice over IP (CVoice), Second Edition. Copyright 2006, Cisco Systems, Inc.. Reproduced by permission of Pearson Education, Inc., 800 East 96th Street, Indianapolis, IN 46240.

Visit www.ciscopress.com for a detailed description and to learn how to purchase this title.

< Back to part one