Methods and systems for evaluating the compliance of software to a quality benchmark

• US 7,788,632 B2
• Filed: 06/02/2006
• Issued: 08/31/2010
• Est. Priority Date: 06/02/2005
• Status: Active Grant

• Current Assignee: United States Postal Service (Government of the United States of America)
• Original Assignee: United States Postal Service (Government of the United States of America)
• Inventors: Kuester, Anthony E., Zelinski, Lillian, Key, Wanda L.
• Primary Examiner(s): Khatri; Anil
• Application Number:
  US11/445,201
• Publication Number:
  US 20070006161A1
• Time in Patent Office: 1,551 Days
• Field of Search: 717100-106, 717120-124, 717203-204
• US Class Current: 717/100
• CPC Class Codes: G06F 11/3672 Test management
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling

Abstract

A method for evaluating the compliance of software to a standard quality process includes establishing a standard quality process that includes a plurality of stages, each of the plurality of stages corresponding to a software development process. A plurality of objective qualifiers associated with the standard quality process are defined, and a weighting factor is assigned to each of the plurality of objective qualifiers. The method also includes applying each of the plurality of objective qualifiers to a software application, and determining a field worthiness score corresponding to the correlation of the software application with each of the objective qualifiers.

Claims

1. A method for evaluating the compliance of a software comprising:

accepting on the computer system an establishing of a standard quality process that includes a plurality of stages, each of the plurality of stages corresponding to a software development process;

accepting on the computer system a defining of a plurality of objective qualifiers associated with the standard quality process, each of the plurality of objective qualifiers having an enumerated plurality of potential values;

accepting on the computer system an assigning of a weighting factor to each of the potential values for each of the plurality of objective qualifiers;

accepting on the computer system results of applying each of the plurality of objective qualifiers to a software application; and

determining a field worthiness score corresponding to the correlation of the software application with each of the objective qualifiers utilizing the computer system;

wherein determining the field worthiness score includes;

determining, based on the determined responses, a raw score associated with each of the plurality of objective qualifiers, wherein the raw score is calculated as the sum of the values corresponding to responses to each of the plurality of questions; and

calculating the field worthiness score as the sum of the raw scores associated with each of the plurality of objective qualifiers, wherein each raw score is adjusted by the weighting factor associated with a corresponding objective qualifier;

comparing the field worthiness score to a predetermined threshold;

approving, if the field worthiness score exceeds the predetermined threshold, an implementation of the software application; and

determining, if the field worthiness score does not exceed a predetermined threshold, that the software application requires further development prior to implementation.

2. The method of claim 1, wherein the standard quality process includes a software development life cycle associated with the development of one or more software programs.

3. The method of claim 2, wherein the software development life cycle includes at least one of a requirement definition phase, a project planning and design phase, a build phase, a test phase, or a deliver phase.

4. The method of claim 1, wherein defining a plurality of objective qualifiers includes developing, for each of the plurality of objective qualifiers, a plurality of objective questions, wherein each possible response to each of the plurality of objective questions is assigned a value.

5. The method of claim 4, wherein applying each of the plurality of objective qualifiers to the software application further includes:

analyzing the software application; and

determining a response to one or more of the plurality of questions based on the analysis.

6. The method of claim 5, further including documenting one or more of the field worthiness score and the raw score associated with each of the plurality of objective qualifiers for future analysis.

7. The method of claim 6, further including providing a report indicative of a potential risk associated with the software application based on the raw score associated with each of the plurality of objective qualifiers.

8. The method of claim 5, wherein analyzing the software application includes evaluating at least a portion of the software application using a predetermined benchmark associated with the software development process.

9. The method of claim 8, wherein the predetermined benchmark includes a statement of work under which the software application was developed.

10. The method of claim 9, wherein the predetermined benchmark includes a predefined software evaluation checklist.

11. The method of claim 1, further including providing a report indicative of a potential risk associated with the software application based on the field worthiness score.

12. The method of claim 1, wherein the weighting factor is predetermined based on a desired level of importance that each of the plurality of object qualifiers has in determining the field worthiness score.

13. The method of claim 12, wherein the desired level of importance is based on empirical data gathered from a previous software development process.

14. A computer-readable storage medium for use on a computer system, the computer-readable medium containing computer-executable instructions for performing a method on a computer system comprising:

comparing a software application to a plurality of objective qualifiers associated with a predetermined standard quality process, the standard quality process having a plurality of stages, each of the plurality of stages corresponding to a software development process, wherein each of the plurality of objective qualifiers has an enumerated plurality of potential values, each of which is weighted according to a weighting criteria; and

determining, based on the comparison a field worthiness score corresponding to the correlation of the software application with each of the objective qualifiers;

determining the field worthiness score includes;

determining, based on the determined responses, a raw score associated with each of the plurality of objective qualifiers, wherein the raw score is calculated as the sum of the values corresponding to responses to each of the plurality of questions; and

calculating the field worthiness score as the sum of the raw scores associated with each of the plurality of objective qualifiers, wherein each raw score is adjusted by a weighting factor associated with a corresponding objective qualifier;

comparing the field worthiness score to a predetermined threshold;

approving, if the field worthiness score exceeds the predetermined threshold, an implementation of the software application; and

determining, if the field worthiness score does not exceed a predetermined threshold, that the software application requires further development prior to implementation.

15. The computer-readable medium storage of claim 14, wherein the standard quality process includes a software development life cycle associated with the development of one or more software programs.

16. The computer-readable medium storage of claim 15, wherein the software development life cycle includes at least one of a requirement definition phase, a project planning and design phase, a build phase, a test phase, or a deliver phase.

17. The computer-readable medium storage of claim 14, wherein the plurality of objective qualifiers includes a plurality of objective questions, wherein each possible response to each of the plurality of objective questions is assigned a value.

18. The computer-readable medium storage of claim 17, wherein comparing the software application to a plurality of objective qualifiers further includes:

analyzing the software application; and

determining a response to one or more of the plurality of questions based on the analysis.

19. The computer-readable storage medium of claim 18, wherein the method further includes documenting one or more of the field worthiness score and the raw score associated with each of the plurality of objective qualifiers for future analysis.

20. The computer-readable storage medium of claim 19, wherein the method further includes providing a report indicative of a potential risk associated with the software application based on the raw score associated with each of the plurality of objective qualifiers.

21. The computer-readable storage medium of claim 18, wherein analyzing the software application includes evaluating at least a portion of the software application using a predetermined benchmark associated with the software development process.

22. The computer-readable storage medium of claim 21, wherein the predetermined benchmark includes a statement of work under which the software application was developed.

23. The computer-readable storage medium of claim 21, wherein the predetermined benchmark includes a predefined software evaluation checklist associated with the software development process.

24. The computer-readable storage medium of claim 14, wherein the weighting criteria is predetermined based on a desired level of importance that each of the plurality of object qualifiers has in determining the field worthiness score.

25. The computer-readable storage medium of claim 24, wherein the desired level of importance is based on empirical data gathered from a previous software development process.

26. The computer-readable storage medium of claim 14, wherein said method further includes providing a report indicative of a potential risk associated with the software application based on the field worthiness score.

27. A computer system, comprising:

a display;

at least one input device; and

a processor in communication with the display and the at least one input device and programmed to;

compare a software application to a plurality of objective qualifiers associated with a predetermined standard quality process, the standard quality process having a plurality of stages, each of the plurality of stages corresponding to a software development process, wherein each of the plurality of objective qualifiers has an enumerated plurality of potential values, each of which is weighted according to a weighting criteria; and

determine, based on the comparison a field worthiness score corresponding to the correlation of the software application with each of the objective qualifiers;

wherein determining the field worthiness score includes;

determining, based on the determined responses, a raw score associated with each of the plurality of objective qualifiers, wherein the raw score is calculated as the sum of the values corresponding to responses to each of the plurality of questions; and

calculating the field worthiness score as the sum of the raw scores associated with each of the plurality of objective qualifiers, wherein each raw score is adjusted by a weighting factor associated with a corresponding objective qualifier;

wherein the processor is further configured to;

compare the field worthiness score to a predetermined threshold;

approve, if the field worthiness score exceeds the predetermined threshold, an implementation of the software application; and

determine, if the field worthiness score does not exceed a predetermined threshold, that the software application requires further development prior to implementation.

28. The computer system of claim 27, wherein the standard quality process includes a software development life cycle associated with the development of one or more software programs.

29. The computer system of claim 28, wherein the software development life cycle includes at least one of a requirement definition phase, a project planning and design phase, a build phase, a test phase, or a deliver phase.

30. The computer system of claim 27, wherein the plurality of objective qualifiers includes a plurality of objective questions, wherein each possible response to each of the plurality of objective questions is assigned a value.

31. The computer system of claim 30, wherein comparing the software application to a plurality of objective qualifiers further includes:

analyzing the software application; and

determining a response to one or more of the plurality of questions based on the analysis.

32. The computer system of claim 31, wherein the processor is further configured to document one or more of the field worthiness score and the raw score associated with each of the plurality of objective qualifiers for future analysis.

33. The computer system of claim 32, wherein the processor is further configured to provide a report indicative of a potential risk associated with the software application based on the raw score associated with each of the plurality of objective qualifiers.

34. The computer system of claim 31, wherein analyzing the software application includes evaluating at least a portion of the software application using a predetermined benchmark associated with the software development process.

35. The computer system of claim 34, wherein the predetermined benchmark includes a statement of work under which the software application was developed.

36. The computer system of claim 34, wherein the predetermined benchmark includes a predefined software evaluation checklist associated with the software development process.

37. The computer systems of claim 27, wherein the weighting criteria is predetermined based on a desired level of importance that each of the plurality of object qualifiers has in determining the field worthiness score.

38. The computer system of claim 37, wherein the desired level of importance is based on empirical data gathered from a previous software development process.

39. The computer system of claim 27, further including providing a report indicative of a potential risk associated with the software application based on the field worthiness score.