Determining the effects of new types of impairments on perceived quality of a voice service

• US 20060126529A1
• Filed: 04/11/2005
• Published: 06/15/2006
• Est. Priority Date: 12/24/1998
• Status: Active Grant

• Current Assignee: Far North Patents, LLC
• Original Assignee: MCI Incorporated (Verizon Communications Inc.)
• Inventors: Hardy, William Christopher
• Granted Patent: US 8,068,437 B2
• Time in Patent Office: Days
• US Class Current: 370/252
• CPC Class Codes: H04L 41/5009 Determining service level performance parameters or violations of service level contracts, e.g. violations of agreed response time or mean time between failures [MTBF]
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04M 3/2227 Quality of service monitoring

Abstract

Empirically derived models are used to relate user perception to objectively measurable characteristics, under a criterion of nearly perfect co-variance between predicted MOS values and the variations in the values of the objectively measurable characteristics. In the example context of evaluating packet switched telephony service, the present invention eliminates the need to perform subjective tests to characterize each combination of codec type and transmission protocol.

Claims

1. Where a first communications service exhibits a first performance characteristic affecting communications quality and a second communications service exhibits both the first performance characteristic and a second performance characteristic not exhibited by the first communications service, a method for determining the effect of the second performance characteristic upon the quality of the second communications service comprising:

generating an assumed model for the second communications service;

establishing communications via the second communications service, wherein the second communications service relies upon a known codec;

obtaining a set of subjective ratings of the first performance characteristic based upon knowledge of the first communication service;

altering the subjective ratings to reflect the effects of the second performance characteristic using the assumed model for the effects of the second performance characteristic upon the subjective ratings of the first performance characteristic; and

computing quality index values from the altered subjective ratings.

11. The method of claim 1, wherein the first performance characteristic is speech distortion.

12. The method of claim 1, wherein the second performance characteristic is packet loss rate.

13. The method of claim 1, wherein the second communications service relies upon a known transmission protocol.

14. The method of claim 1, wherein generating the assumed model for the second communications service comprises:

establishing communications via the second communications service having the known codec;

obtaining objective measurements of the first performance characteristic and the second performance characteristic;

transforming the objective measurements into impairment ratings reflective of the first performance characteristic;

altering the impairment ratings to reflect the effects of the second performance characteristic using an initial model for the effects of the second performance characteristic upon the impairment ratings, thereby generating the assumed model;

computing quality index values from the altered impairment ratings; and

and

adjusting the assumed model to improve a correlation between the quality index values and the second performance characteristic.

15. The method of claim 1, further comprising:

analyzing the quality index values over time as a function of the second performance characteristic to identify acceptable and unacceptable quality index values;

generating a maximum proportion of unacceptable quality index values to acceptable quality index values based on at least the analyzed quality index values; and

determining that the quality of the second communications service is comparable to a quality of the first communications service if the proportion of unacceptable quality index values to acceptable quality index values is below the maximum proportion of unacceptable quality index values to acceptable quality index values.

16. The method of claim 15, wherein the quality index values include at least one of a mean opinion score (MOS) or a percentage (P) of calls or connections that elicit unusable (U), difficult (D), or irritating (I) responses (P(UDI)).

17. The method of claim 1 wherein said the computing quality index values is performed by convolving the altered subjective ratings with an effects matrix.

2-10. -10. (canceled)

18. A method for determining acceptability of quality of a second communications service, in comparison to a first communications service which is deemed to exhibit acceptable quality, where the second communications service has a known codec, comprising:

generating an assumed model, wherein the assumed model is used to transform data regarding a first performance characteristic to reflect effects from a second performance characteristic;

establishing a communications session via the second communications service, wherein the second communications service uses a known codec;

obtaining subjective ratings of the first performance characteristic;

generating altered subjective ratings using the assumed model to reflect effects of the second performance characteristic on the subjective ratings;

generating quality index values from the altered subjective ratings; and

determining whether the quality of the second communications service is comparable to a quality of the first communications service based on the quality index values.

19. The method of claim 18, wherein the first performance characteristic is distortion and the second performance characteristic is one of packet loss or packet delay.

20. The method of claim 18, wherein generating the assumed model comprises:

establishing communications via the second communications service using the known codec;

obtaining objective measurements of the first performance characteristic relative to the second performance characteristic;

grouping the objective measurements into sets associated with the second performance characteristic;

transforming each set of objective measurements into a corresponding set of subjective ratings reflective of the first performance characteristic;

generating altered sets of subjective ratings corresponding to the effects of the second performance characteristic using an initial model for the effects of the second performance characteristic upon the subjective ratings of the first performance characteristic, thereby generating the assumed model associated with the known codec of the second communications service;

computing quality index values from the altered sets of subjective ratings; and

and

adjusting the assumed model to improve the correlation between the quality index values and the second performance characteristic.

21. A method for determining acceptability of quality of a second communications service, in comparison to a first communications service which is deemed to exhibit acceptable quality, where the second communications service relies upon a known codec, comprising:

establishing a communications session via the second communications service;

obtaining objective data regarding a first performance characteristic of the first communications service;

generating a quality index for the first communications service based on the objective data;

determining an acceptable quality index for the second communications service based upon the quality index for the first communications service;

modifying the objective data to reflect effects of a second performance characteristic using a model associated with the second communications service; and

computing quality index values from the modified objective data.

22. The method of claim 21, wherein the first performance characteristic is speech distortion.

23. The method of claim 21, wherein the second performance characteristic is at least one of packet loss or packet delay.

24. The method of claim 21, wherein the first communications service is a circuit switched network and wherein the second communications service is a packet switched network.

25. The method of claim 21, wherein the model associated with the known codec of the second communications service comprises:

establishing communications via the second communications service using the known codec;

obtaining objective measurements of the first performance characteristic and the second performance characteristic;

transforming the objective measurements into impairment ratings reflective of the first performance characteristic;

altering the impairment ratings to reflect the effects of the second performance characteristic using an initial model for the effects of the second performance characteristic upon the impairment ratings, thereby generating the model;

computing quality index values from the altered impairment ratings; and

and

adjusting the model to improve a correlation between the quality index values and the second performance characteristic.

26. The method of claim 21, wherein the quality index values include at least one of a mean opinion score (MOS) or a percentage (P) of calls or connections that elicit unusable (U), difficult (D), or irritating (I) responses (P(UDI)).