Method for measuring the usability of a system

• US 5,808,908 A
• Filed: 05/19/1997
• Issued: 09/15/1998
• Est. Priority Date: 05/31/1994
• Status: Expired due to Term

• Current Assignee: Patentes Talgo SL (Pegaso Rail International SA)
• Original Assignee: Lucent Technologies, Inc. (Nokia Corporation)
• Inventors: Ghahramani, Bahador
• Primary Examiner(s): Voeltz, Emanuel T.
• Assistant Examiner(s): Stamber, Eric W.
• Application Number:
  US08/858,134
• Time in Patent Office: 484 Days
• Field of Search: 364/551.01, 364/554, 705/1, 705/7, 705/10, 434/236, 434/322, 434/323, 434/362, 434/363
• US Class Current: 702/182
• CPC Class Codes: G06F 11/3419 by assessing time
• CPC Class Codes: G06F 11/3452 Performance evaluation by statistical analysis
• CPC Class Codes: G06F 11/3616 using software metrics
• CPC Class Codes: G06Q 10/06375 Prediction of business process outcome or impact based on a proposed change
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising

Abstract

A method for quantitatively and objectively measuring the usability of a system. The method provides quantitative measures for usability satisfaction, usability performance, and usability performance indicators. Usability satisfaction is measured by acquiring data from a system user population with respect to a set of critical factors that are identified for the system. Usability performance is measured by acquiring data for quantifying the statistical significance of the difference in the mean time for an Expert population to perform a task on a particular number of trials and the estimated mean time for a Novice population to perform the task on the same number of trials. The estimated mean time is calculated according to the Power Law of Practice. Usability Performance Indicators include Goal Achievement Indicators, Work Rate Usability Indicators, and Operability Indicators which are calculated according to one or more measurable parameters which include performance times, numbers of problems encountered, number of actions taken, time apportioned to problems, learning time, number of calls for assistance, and the number of unsolved problems.

Claims

1. A method for operating a computer to provide a quantitative usability measure for evaluating usability of a system, said system including a process, product, or manufacturing procedure and having means for performing a task, said task including at least one step, said method comprising the computer executed steps of:

measuring first time value signals for each member of an Expert population to perform said task on each of a plurality of trials;

measuring second time value signals for each member of a Novice population to perform said task on each of at least one trial;

estimating, according to a predetermined relationship, a mean time for said Novice population to perform said task on a future trial;

comparing a mean time for said Expert population to perform said task on a trial equivalent in number of trials to said future trial with said mean time for said Novice population to generate a statistical index indicating a significance of the difference therebetween; and

providing said quantitative usability measure as a function of said statistical index.

2. The method according to claim 1, wherein said predetermined relationship is based on a Power Law of Practice relationship.

3. The method according to claim 1, wherein said comparing step includes generating a Student t-Distribution test statistic for said Expert population and said Novice population, wherein said statistical index is based on a significance level corresponding to said Student t-Distribution test statistic.

4. The method according to claim 1, further comprising the steps of:

identifying critical factors for said system;

identifying at least one bipolar adjective pair for each critical factor, said bipolar adjective pair consisting of a positive adjective and a negative adjective having opposite semantic meaning;

acquiring adjective pair data from each member of a system user population for each said bipolar adjective pair, said adjective pair data for each bipolar adjective pair having a numerical value within a range bounded by a value assigned to said positive adjective and a value assigned to said negative adjective and linearly related to the positive adjective relative to the negative adjective in characterizing said critical factor according to the member;

acquiring significance data from each member of said system user population for each said critical factor, said significance data numerically representing the relative significance of said critical factor according to the member; and

generating a usability satisfaction value according to said adjective pair data and said significance data.

5. The method according to claim 4, wherein said usability satisfaction value represents the satisfaction of a group of users with respect to a given critical factor.

6. The method according to claim 4, wherein said usability satisfaction value represents the satisfaction of a given user with respect to a set of critical factors.

7. The method according to claim 4, wherein said usability satisfaction value includes, for at least one of said critical factors, a product of a sum of said adjective pair data and said significance data.

8. The method according to claim 7, wherein said usability satisfaction value includes a sum of said product for each critical factor for a given user.

9. The method according to claim 7, wherein said usability satisfaction value includes a sum of said product for each user for a given critical factor.

10. The method according to claim 4, further comprising the step of generating at least one Usability Performance Indicator according to measured parameters.

11. The method according to claim 10, wherein said measured parameters include performance time and number of steps performed for completing said task.

12. The method according to claim 10, wherein said Usability Performance Indicator is selected from the group of Goal Achievement Indicator, Work Rate Indicator, and Operability Indicator.

13. The method according to claim 1, further comprising the steps of generating at least one Usability Performance Indicator according to measured parameters.

14. The method according to claim 13, wherein said at least one Usability Performance Indicator is selected from a group of indicators including a Novice user effectiveness, and a Relative Efficiency of a Novice user to an Expert user.

15. The method according to claim 14, wherein said Usability Performance Indicator is selected from the group of Usability Goal Achievement Indicator, Usability Work Rate Indicator, and Usability Operability Indicator.

16. The method according to claim 1, wherein said system is a computer system.

17. The method according to claim 1, wherein said quantitative usability measure equals said statistical index.

18. A method for operating a computer to provide a quantitative usability measure for quantitatively evaluating usability of a system, comprising the computer executed steps of:

acquiring first data for providing a usability satisfaction measure according to a set of critical factors of said system;

measuring second data for providing a usability performance measure according to the difference between a measured performance time for an Expert population to perform a task on a given trial and an estimated performance time for a Novice population to perform the task on the given trial;

measuring third data for providing a Usability Performance Indicator; and

processing said first, second, and third data to provide quantitative values for said usability satisfaction, said usability performance, and said Usability Performance Indicator.