Full service research bureau and test center method and apparatus

• US 6,741,967 B1
• Filed: 03/26/1999
• Issued: 05/25/2004
• Est. Priority Date: 11/02/1998
• Status: Expired due to Term

• Current Assignee: Dynatrace LLC (Dynatrace, Inc.)
• Original Assignee: Vividence Corp. (Dynatrace, Inc.)
• Inventors: Wu, Arthur F., Ketchpel, Steven P.
• Primary Examiner(s): Hafiz, Tariq R.
• Assistant Examiner(s): ALLEN, AKIBA KANELLE
• Application Number:
  US09/277,574
• Time in Patent Office: 1,887 Days
• Field of Search: 705/10, 705/11, 705/7, 705/1, 705/500, 709/1, 709/100, 709/310, 709/224, 709/223, 709/225, 709/226, 702/186
• US Class Current: 705/7.32
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0203 Market surveys; Market polls
• CPC Class Codes: G06Q 30/0204 Market segmentation
• CPC Class Codes: G06Q 40/00 Finance; Insurance; Tax strategies; Processing of corporate or income taxes
• CPC Class Codes: G06Q 99/00 Subject matter not provided for in other groups of this subclass

Abstract

A centralized full-service research bureau and test center method and apparatus is provided, which assists a customer, typically a Web product manager, in designing a usability test for typical tasks faced by a visitor to the customer'"'"'s site. The centralized full-service research bureau and test center method and apparatus provides proprietary software to administer the usability test to a pre-qualified pool of testers meeting desired demographic constraints. The usability test measures not only the visitor'"'"'s success in achieving objectives, but also prompts a tester for context-specific feedback ranging from aesthetics to behavior. Statistics are aggregated across the testing population and presented not just as data, but with recommended actions backed up by analysis.

Claims

1. A method of testing the usability and feature quality of a World Wide Web site, comprising the steps of:

providing a first proprietary software, said first proprietary software comprising a usability and feature quality test comprising a set of specific, predefined objectives to be achieved by a tester in connection with World Wide Web sites to which no additional HTML is added, and which may be independently selected by the tester while completing said objective, where said tester performs actions necessary to achieve said objectives in a manner chosen by the said tester in connection with said tester selected World Wide Web sites;

providing a second proprietary software to administer said usability and feature quality test to a plurality of pre-qualified testers within a specified sample population that comprises individuals who are selected specifically for purposes of taking said test;

providing a storing and retrieval means to store and retrieve data resulting from said administered usability and feature quality test; and

providing a World Wide Web based architecture, wherein said first proprietary software comprises a modified browser and/or software working in conjunction with a browser;

whereby said stored data are aggregatable across a plurality of pre-qualified testers and usable in a future analysis associated with said set of specific, predefined objectives.

2. The full-service research bureau and test center method of claim 1, wherein said software working in conjunction with a browser comprises any of, but not limited to a plug-in, a browser helper object, an applet, and a proxy.

3. The full-service research bureau and test center method of claim 2, wherein said modified browser is downloaded and installed on a personal machine of said pre-qualified tester.

4. The full-service research bureau and test center method of claim 2, wherein said modified browser is adapted to collect a plurality of browser events and to transmit said browser events to said second proprietary software.

5. The full-service research bureau and test center method of claim 4, wherein said plurality of browser events comprises any of:

a submission of a page request;

a fulfillment of a page request;

use of a back, forward, home, or stop button;

access of a page from a history or bookmark list;

use of browser functions, including any of adding a bookmark searching for a text string, or viewing HTML source;

a page-scrolling event; and

a mouse-over event.

6. The full-service research bureau and test center method of claim 4, wherein said transmitted browser events are in a log form.

7. The full-service research bureau and test center method of claim 2, wherein said modified browser is adapted to pose a question to said pre-qualified tester, said question associated with a browser event from said plurality of browser events, and to transmit a response to said question to said second proprietary software.

8. The full-service research bureau and test center method of claim 2, wherein said modified browser is adapted to pose a question to said pre-qualified tester, said question associated with a message from said second proprietary software and to transmit a response to said message to said second proprietary software.

9. The full-service research bureau and test center method of claim 2, wherein modified browser comprises:

a plurality of additional buttons, any of said additional buttons used by said pre-qualified tester to provide comments or interact with either of a human or automated test moderator; and

wherein said modified browser is adapted to transmit said comments to said second proprietary software.

10. The full-service research bureau and test center method of claim 2, further comprising the step of:

loading a World Wide Web page directly from a host server, and wherein said modified browser is adapted to display said World Wide Web page without adding additional Hyper-Text Markup Language (HTML), conveying an appearance and a latency as displayed by a non-modified browser.

11. The full-service research bureau and test center method of claim 1, wherein said second proprietary software is a server application, said server application adapted to support a plurality of usability test sessions, any of said sessions used simultaneously by any of said plurality of pre-qualified testers.

12. The full-service research bureau and test center method of claim 1, wherein said usability test comprises a series of questions, said series of questions representing all or a part of said set of customer objectives.

13. The full-service research bureau and test center method of claim 12, further comprising the step of:

providing an additional question associated with a specific answer to one of said series of questions or associated with particular actions taken during the usability test session.

14. The full-service research bureau and test center method of claim 1, wherein said storing and retrieving means further comprises the step of:

providing a data warehouse comprising a plurality of entries, wherein each entry of said plurality of entries is tagged with a test sequence indicator and a pre-qualified tester identifier, such that said storing and retrieval means is responsive to a complex query.

15. The full-service research bureau and test center method of claim 14, further comprising the step of:

providing a set of queried data from said complex query, wherein said queried data is associated with a predetermined set of demographic properties.

16. The full-service research bureau and test center method of claim 14, further comprising the step of:

providing a set of queried data from said complex query, wherein said queried data is associated with a predetermined set of test-specific properties.

17. The full-service research bureau and test center method of claim 1, further comprising any of the steps of:

providing a test building means, which said customer uses to create and initiate said usability test;

providing a reporter means, which said customer uses to submit a set of queries to said storing and retrieval means and, thereby to obtain an ad hoc report;

providing a report builder, whereby said report builder is adapted to process all or a part of said stored data to provide analysis of said all or a part of said stored data;

providing a World Wide Web site, whereby said World Wide Web site is adapted to facilitate communication between said customers and said testers; and

providing a test auditor, whereby a predetermined standard of quality of said data is assured.

18. The full-service research bureau and test center method of claim 17, wherein said test building means is automatic;

and further comprising the step of;

providing a specifying means, whereby said customer can specify any of;

a predetermined number of said plurality of testers;

a desired demographic cross-section;

said set of customer objectives; and

other questions.

19. The full-service research bureau and test center method of claim 17, wherein said reporter means comprises a World Wide Web application.

20. The full-service research bureau and test center method of claim 17, wherein said reporter means provides interactive interpretive activities.

21. The full-service research bureau and test center method of claim 17, wherein said report builder is adapted to assist in highlighting substantially relevant quotes and statistics.

22. The full-service research bureau and test center method of claim 17, wherein said report builder is adapted to provide an automated personalized report.

23. The full-service research bureau and test center method of claim 17, further comprising any of the steps of:

providing a customer educating means on said World Wide Web site for said customer to learn about benefits of said usability test;

providing a tester educating means on said World Wide Web site for said tester to learn about said benefits of said usability test;

providing an enrolling means for said tester on said World Wide Web site;

providing an access means for said customer on World Wide Web site to access any of;

a pre-generated report;

said ad hoc report; and

said test builder means;

creating a plurality of communities on said World Wide Web site for all or a portion of said plurality of testers and a plurality of customers; and

providing a means for managing a plurality of electronic commercial (e-commerce) transactions.

24. The full-service research bureau and test center method of claim 17, further comprising the step of:

coupling said test auditor to said second proprietary software, and wherein said test auditor is adapted to measure an amount of effort spent by said tester on a task of said usability test and an amount of time spent by said tester on said task of said usability test.

25. The full-service research bureau and test center method of claim 17, wherein said test auditor provides a bonus to said tester, said tester having provided a substantially high quality response.

26. An apparatus for testing the usability and feature quality of a World Wide Web site, comprising:

a first proprietary software, said first proprietary software comprising a usability and feature quality test;

comprising a set of specific, predefined objectives to be achieved by a tester in connection with World Wide Web sites to which no additional HTML is added, and which may be independently selected by the tester while completing said objective, where said tester performs actions necessary to achieve said objectives in a manner chosen by said tester in connection with said tester selected World Wide Web sites;

a second proprietary software to administer said usability and feature quality test to a plurality of pre-qualified testers within a specified sample population that comprises individuals who are selected specifically for purposes of taking the test;

a storing and retrieval means to store and retrieve data resulting from said administered usability and feature quality test; and

a World Wide Web based architecture, wherein said first proprietary software comprises a modified browser and/or software working in conjunction with a browser;

whereby said stored data are aggregatable across a plurality of pre-qualified testers and usable in a future analysis associated with said set of specific, predefined objectives.

27. The full-service research bureau and test center apparatus of claim 26, wherein said software working in conjunction with a browser comprises any of, but not limited to a plug-in, a browser helper object, an applet, and a proxy.

28. The full-service research bureau and test center apparatus of claim 27, wherein said modified browser is downloaded and installed on a personal machine of said pre-qualified tester.

29. The full-service research bureau and test center apparatus of claim 27, wherein said modified browser is adapted to collect a plurality of browser events and to transmit said browser events to said second proprietary software.

30. The full-service research bureau and test center apparatus of claim 29, wherein said plurality of browser events comprises any of:

a submission of a page request;

a fulfillment of a page request;

use of a back, forward, home, or stop button;

access of a page from a history or bookmark list;

use of browser functions, including any of adding a bookmark, searching for a text string, or viewing HTML source;

a page-scrolling event; and

a mouse-over event.

31. The full-service research bureau and test center apparatus of claim 29, wherein said transmitted browser events are in a log form.

32. The full-service research bureau and test center apparatus of claim 27, wherein said modified browser is adapted to pose a question to said pre-qualified tester, said question associated with a browser event from said plurality of browser events, and to transmit a response to said question to said second proprietary software.

33. The full-service research bureau and test center apparatus of claim 27, wherein said modified browser is adapted to pose a question to said pre-qualified tester, said question associated with a message from said second proprietary software and to transmit a response to said message to said second proprietary software.

34. The full-service research bureau and test center apparatus of claim 27, wherein modified browser comprises:

a plurality of additional buttons, any of said additional buttons used by said pre-qualified tester to provide comments or interact with either a human or automated test moderator; and

wherein said modified browser is adapted to transmit said comments to said second proprietary software.

35. The full-service research bureau and test center apparatus of claim 27, further comprising:

a World Wide Web page, said World Wide Web page loaded directly from a host server, and wherein said modified browser is adapted to display said World Wide Web page without adding additional Hyper-Text Markup Language (HTML), conveying an appearance and a latency as displayed by a non-modified browser.

36. The full-service research bureau and test center apparatus of claim 26, wherein said second proprietary software is a server application, said server application adapted to support a plurality of usability test sessions, any of said sessions used simultaneously by any of said plurality of pre-qualified testers.

37. The full-service research bureau and test center apparatus of claim 26, wherein said usability test comprises a series of questions, said series of questions representing all or a part of said set of customer objectives.

38. The full-service research bureau and test center apparatus of claim 37, further comprising:

an additional question associated with a specific answer to one of said series of questions or associated with particular actions taken during the usability test session.

39. The full-service research bureau and test center apparatus of claim 26, wherein said storing and retrieving means further comprises:

a data warehouse comprising a plurality of entries, wherein each entry of said plurality of entries is tagged with a test sequence indicator and a pre-qualified tester identifier, such that said storing and retrieval means is responsive to a complex query.

40. The full-service research bureau and test center apparatus of claim 39, further comprising:

a set of queried data from said complex query, wherein said queried data is associated with a predetermined set of demographic properties.

41. The full-service research bureau and test center apparatus of claim 39, further comprising:

a set of queried data from said complex query, wherein said queried data is associated with a predetermined set of test-specific properties.

42. The full-service research bureau and test center apparatus of claim 26, further comprising any of:

a test building means, which said customer uses to create and initiate said usability test;

a reporter means, which said customer uses to submit a set of queries to said storing and retrieval means and, thereby to obtain an ad hoc report;

a report builder, whereby said report builder is adapted to process all or a part of said stored data to provide analysis of said all or a part of said stored data;

a World Wide Web site, whereby said World Wide Web site is adapted to facilitate communication between said customers and said testers; and

a test auditor, whereby a predetermined standard of quality of said data is assured.

43. The full-service research bureau and test center apparatus of claim 42, wherein said test building means is automatic;

and further comprising;

a specifying means, whereby said customer can specify any of;

a predetermined number of said plurality of testers;

a desired demographic cross-section;

said set of customer objectives; and

other questions.

44. The full-service research bureau and test center apparatus of claim 42, wherein said reporter means comprises a World Wide Web application.

45. The full-service research bureau and test center apparatus of claim 42, wherein said reporter means provides interactive interpretive activities.

46. The full-service research bureau and test center apparatus of claim 42, wherein said report builder is adapted to assist in highlighting substantially relevant quotes and statistics.

47. The full-service research bureau and test center apparatus of claim 42, wherein said report builder is adapted to provide an automated personalized report.

48. The full-service research bureau and test center apparatus of claim 42, further comprising any of:

a customer educating means on said World Wide Web site for said customer to learn about benefits of said usability test;

a tester educating means on said World Wide Web site for said tester to learn about said benefits of said usability test;

an enrolling means for said tester on said World Wide Web site;

an access means for said customer on World Wide Web site to access any of;

a pre-generated report;

said ad hoc report; and

said test builder means;

a plurality of communities on said World Wide Web site for all or a portion of said plurality of testers and a plurality of customers; and

a means for managing a plurality of electronic commercial (e-commerce) transactions.

49. The full-service research bureau and test center apparatus of claim 42, wherein said test auditor is coupled to said second proprietary software, and wherein said test auditor is adapted to measure an amount of effort spent by said tester on a task of said usability test and an amount of time spent by said tester on said task of said usability test.

50. The full-service research bureau and test center apparatus of claim 42, wherein said test auditor provides a bonus to said tester, said tester having provided a substantially high quality response.

51. A method of testing the usability and feature quality of a World Wide Web site, comprising the steps of:

providing any of, but not limited to a modified browser, software working in conjunction with a browser, a plug-in, a browser helper object, an applet, and a proxy;

logging onto a full-service research bureau and test center;

designing a test script for a test site comprising a set of specific, predefined objectives to be achieved by a tester in connection with World Wide Web sites to which no additional HTML is added, and which may be independently selected by the tester while completing said objective, where said tester performs actions necessary to achieve said objectives in a manner chosen by said tester in connection with said tester selected World Wide Web sites;

defining a composition of a desired test panel;

defining requirements for a test completion date;

authorizing a test run;

retrieving raw test result data;

analyzing raw test result data;

optionally comparing analyzed test result data with at least a second analyzed test result data of a second test run by a second tester;

wherein said second tester is different from said tester;

optionally retrieving an industry report on a plurality of comparative test runs across an industry segment; and

optionally building at least one report for delivery to a stakeholder in said test site.

52. The method of claim 51, further, comprising the steps of:

logging onto said full-service research bureau and test center;

registering to become a paid tester;

downloading a tester client modified browser and/or software working in conjunction with a browser;

searching for a new usability test session; and

optionally checking a personal account for a payment status and to manage a personal profile.

53. The full-service research bureau and test center method of claim 52, wherein said software working in conjunction with a browser comprises any of, but not limited to a plug-in, a browser helper object, an applet, and a proxy.