Methods and systems for evaluating the compliance of software to a quality benchmark

• US 20070006161A1
• Filed: 06/02/2006
• Published: 01/04/2007
• Est. Priority Date: 06/02/2005
• Status: Active Grant

• Current Assignee: United States Postal Service (Government of the United States of America)
• Original Assignee: United States Postal Service (Government of the United States of America)
• Inventors: Zelinski, Lillian, Key, Wanda, Kuester, Anthony
• Granted Patent: US 7,788,632 B2
• Time in Patent Office: Days
• US Class Current: 717/126
• CPC Class Codes: G06F 11/3672 Test management
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling

Abstract

A method for evaluating the compliance of software to a standard quality process includes establishing a standard quality process that includes a plurality of stages, each of the plurality of stages corresponding to a software development process. A plurality of objective qualifiers associated with the standard quality process are defined, and a weighting factor is assigned to each of the plurality of objective qualifiers. The method also includes applying each of the plurality of objective qualifiers to a software application, and determining a field worthiness score corresponding to the correlation of the software application with each of the objective qualifiers.

Claims

1. A method for evaluating the compliance of a software comprising:

establishing a standard quality process that includes a plurality of stages, each of the plurality of stages corresponding to a software development process;

defining a plurality of objective qualifiers associated with the standard quality process;

assigning a weighting factor to each of the plurality of objective qualifiers;

applying each of the plurality of objective qualifiers to a software application; and

determining a field worthiness score corresponding to the correlation of the software application with each of the objective qualifiers.

2. The method of claim 1, wherein the standard quality process includes a software development life cycle associated with the development of one or more software programs.

3. The method of claim 2, wherein the software development life cycle includes at least one of a requirement definition phase, a project planning and design phase, a build phase, a test phase, or a deliver phase.

4. The method of claim 1, wherein defining a plurality of objective qualifiers includes developing, for each of the plurality of objective qualifiers, a plurality of objective questions, wherein each possible response to each of the plurality of objective questions is assigned a value.

5. The method of claim 4, wherein applying each of the plurality of objective qualifiers to the software application further includes:

analyzing the software application; and

determining a response to one or more of the plurality of questions based on the analysis.

6. The method of claim 5, wherein determining the field worthiness score includes:

determining, based on the determined responses, a raw score associated with each of the plurality of objective qualifiers, wherein the raw score is calculated as the sum of the values corresponding to responses to each of the plurality of questions; and

calculating the field worthiness score as the sum of the raw scores associated with each of the plurality of objective qualifiers, wherein each raw score is adjusted by the weighting factor associated with a corresponding objective qualifier.

7. The method of claim 6, further including:

comparing the field worthiness score to a predetermined threshold;

approving, if the field worthiness score exceeds the predetermined threshold, an implementation of the software application; and

determining, if the field worthiness score does not exceed a predetermined threshold, that the software application requires further development prior to implementation.

8. The method of claim 7, further including documenting one or more of the field worthiness score and the raw score associated with each of the plurality of objective qualifiers for future analysis.

9. The method of claim 8, further including providing a report indicative of a potential risk associated with the software application based on the raw score associated with each of the plurality of objective qualifiers.

10. The method of claim 5, wherein analyzing the software application includes evaluating at least a portion of the software application using a predetermined benchmark associated with the software development process.

13. The method of claim 1, further including providing a report indicative of a potential risk associated with the software application based on the field worthiness score.

14. The method of claim 1, wherein the weighting factor is predetermined based on a desired level of importance that each of the plurality of object qualifiers has in determining the field worthiness score.

15. The method of claim 14, wherein the desired level of importance is based on empirical data gathered from a previous software development process.

11. The method of claim 11, wherein the predetermined benchmark includes a statement of work under which the software application was developed.

12. The method of claim 11, wherein the predetermined benchmark includes a predefined software evaluation checklist.

16. A computer-readable medium for use on a computer system, the computer-readable medium having computer-executable instruction for performing a method comprising:

comparing a software application to a plurality of objective qualifiers associated with a predetermined standard quality process, the standard quality process having a plurality of stages, each of the plurality of stages corresponding to a software development process, wherein each of the plurality of objective qualifiers is weighted according to a weighting criteria; and

determining, based on the comparison. a field worthiness score corresponding to the correlation of the software application with each of the objective qualifiers.

17. The method of claim 16, wherein the standard quality process includes a software development life cycle associated with the development of one or more software programs.

18. The method of claim 17, wherein the software development life cycle includes at least one of a requirement definition phase, a project planning and design phase, a build phase, a test phase, or a deliver phase.

19. The method of claim 16, wherein the plurality of objective qualifiers includes a plurality of objective questions, wherein each possible response to each of the plurality of objective questions is assigned a value.

20. The method of claim 19, wherein comparing the software application to a plurality of objective qualifiers further includes:

analyzing the software application; and

determining a response to one or more of the plurality of questions based on the analysis.

21. The method of claim 20, wherein determining the field worthiness score includes:

determining, based on the determined responses, a raw score associated with each of the plurality of objective qualifiers, wherein the raw score is calculated as the sum of the values corresponding to responses to each of the plurality of questions; and

calculating the field worthiness score as the sum of the raw scores associated with each of the plurality of objective qualifiers, wherein each raw score is adjusted by a weighting factor associated with a corresponding objective qualifier.

22. The method of claim 21, further including:

comparing the field worthiness score to a predetermined threshold;

approving, if the field worthiness score exceeds the predetermined threshold, an implementation of the software application; and

determining, if the field worthiness score does not exceed a predetermined threshold, that the software application requires further development prior to implementation.

23. The method of claim 22, further including documenting one or more of the field worthiness score and the raw score associated with each of the plurality of objective qualifiers for future analysis.

24. The method of claim 23, further including providing a report indicative of a potential risk associated with the software application based on the raw score associated with each of the plurality of objective qualifiers.

25. The method of claim 20, wherein analyzing the software application includes evaluating at least a portion of the software application using a predetermined benchmark associated with the software development process.

26. The method of claim 25, wherein the predetermined benchmark includes a statement of work under which the software application was developed.

27. The method of claim 25, wherein the predetermined benchmark includes a predefined software evaluation checklist associated with the software development process.

28. The method of claim 16, wherein the weighting criteria is predetermined based on a desired level of importance that each of the plurality of object qualifiers has in determining the field worthiness score.

29. The method of claim 28, wherein the desired level of importance is based on empirical data gathered from a previous software development process.

30. The method of claim 16, further including providing a report indicative of a potential risk associated with the software application based on the field worthiness score.

31. A computer system, comprising:

a display;

at least one input device; and

a processor in communication with the display and the at least one input device and configured to;

compare a software application to a plurality of objective qualifiers associated with a predetermined standard quality process, the standard quality process having a plurality of stages, each of the plurality of stages corresponding to a software development process, wherein each of the plurality of objective qualifiers is weighted according to a weighting criteria; and

determine, based on the comparison. a field worthiness score corresponding to the correlation of the software application with each of the objective qualifiers.

32. The computer system of claim 31, wherein the standard quality process includes a software development life cycle associated with the development of one or more software programs.

33. The computer system of claim 32, wherein the software development life cycle includes at least one of a requirement definition phase, a project planning and design phase, a build phase, a test phase, or a deliver phase.

34. The computer system of claim 31, wherein the plurality of objective qualifiers includes a plurality of objective questions, wherein each possible response to each of the plurality of objective questions is assigned a value.

35. The computer system of claim 34, wherein comparing the software application to a plurality of objective qualifiers further includes:

analyzing the software application; and

determining a response to one or more of the plurality of questions based on the analysis.

36. The computer system of claim 35, wherein determining the field worthiness score includes:

determining, based on the determined responses, a raw score associated with each of the plurality of objective qualifiers, wherein the raw score is calculated as the sum of the values corresponding to responses to each of the plurality of questions; and

calculating the field worthiness score as the sum of the raw scores associated with each of the plurality of objective qualifiers, wherein each raw score is adjusted by a weighting factor associated with a corresponding objective qualifier.

37. The computer system of claim 36, wherein the processor is further configured to:

compare the field worthiness score to a predetermined threshold;

approve, if the field worthiness score exceeds the predetermined threshold, an implementation of the software application; and

determine, if the field worthiness score does not exceed a predetermined threshold, that the software application requires further development prior to implementation.

38. The computer system of claim 37, wherein the processor is further configured to document one or more of the field worthiness score and the raw score associated with each of the plurality of objective qualifiers for future analysis.

39. The computer system of claim 38, wherein the processor is further configured to provide a report indicative of a potential risk associated with the software application based on the raw score associated with each of the plurality of objective qualifiers.

40. The computer system of claim 35, wherein analyzing the software application includes evaluating at least a portion of the software application using a predetermined benchmark associated with the software development process.

41. The computer system of claim 40, wherein the predetermined benchmark includes a statement of work under which the software application was developed.

42. The computer system of claim 40, wherein the predetermined benchmark includes a predefined software evaluation checklist associated with the software development process.

43. The computer system of claim 31, wherein the weighting criteria is predetermined based on a desired level of importance that each of the plurality of object qualifiers has in determining the field worthiness score.

44. The computer system of claim 43, wherein the desired level of importance is based on empirical data gathered from a previous software development process.

45. The computer system of claim 31, further including providing a report indicative of a potential risk associated with the software application based on the field worthiness score.