Method, apparatus, and computer-readable medium for identifying character coordinates

• US 20040190773A1
• Filed: 03/31/2003
• Published: 09/30/2004
• Est. Priority Date: 03/31/2003
• Status: Active Grant

• Current Assignee: American Megatrends International, LLC
• Original Assignee: American Megatrends Incorporated
• Inventors: Seshadri, Ramkumar, Ayanam, Varadachari Sudan, Dharmapuri, Srinivasan S., Messer, Jason Andrew
• Granted Patent: US 7,418,141 B2
• Time in Patent Office: Days
• US Class Current: 382/182
• CPC Class Codes: G06V 20/62 Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G06V 30/10 Character recognition

Abstract

Methods, apparatus, and computer-readable mediums for recognizing or identifying the location of characters within a computer image or display are provided. These methods, apparatuses, and computer-readable mediums make use of dynamic link libraries (“DLLs”) and application program interfaces (“APIs”) to locate, indicate the state of, and return the coordinates of specified characters or text within an image or display. The returned state and coordinates of the characters may also be used in conjunction with automated testing programs to assist the programs in delivering precise instructions to computer programs being tested. Further, the methods and computer-readable mediums make use of one or more operating system independent video and input/output redirection devices that allow remote control of a computer system from any network-connected location. By using such redirection devices in conjunction with an automated testing program utilizing character coordinates, software components may be precisely and efficiently tested from virtually anywhere in the world.

Claims

1. A method for identifying the location of one or more characters within an image, the method comprising:

receiving a search string identifying the characters and a request to locate the characters within the image;

generating a bitmap of the characters;

identifying the location of the bitmap within the image; and

in response to the request, returning the location of the bitmap within the image.

2. The method of claim 1, wherein the location is expressed using one or more pixel coordinates.

3. The method of claim 2, wherein the request further identifies a color value for the characters of the search string, and wherein the bitmap is generated using the character color value.

4. The method of claim 3, wherein the request further identifies a color value for the background of the search string, and wherein the bitmap is generated using the background color value.

5. The method of claim 4, wherein the request further identifies a font for the characters of the search string, and wherein the bitmap is generated using the font.

6. The method of claim 5, wherein the request further identifies a font size or range of two or more font sizes for the characters of the search string, and wherein the bitmap is generated using the font sizes.

7. The method of claim 6, wherein the request further comprises a designation as to whether the characters of the search string are bold, and wherein the method further comprises:

in response to a bold designation, generating the bitmap using the bold designation.

8. The method of claim 7, wherein the request further comprises a designation as to whether the characters of the search string are underlined;

and wherein the method further comprises;

in response to an underline designation, generating the bitmap using the underline designation.

9. The method of claim 8, wherein the request further comprises a designation as to whether the characters of the search string are italic, and wherein the method further comprises:

in response to an italic designation, generating the bitmap using the italic designation.

10. The method of claim 9, wherein the request further comprises a designation as to whether the characters of the search string are strikeout and wherein the method further comprises:

in response to a strikeout designation, generating the bitmap using the strikeout designation.

11. The method of claim 4, wherein the characters comprise scalable fonts.

12. The method of claim 3, wherein a segment of the character color value is stored and used to identify the location of the bitmap within the image.

13. A computer-controlled apparatus capable for performing the method of claim 1.

14. A computer-readable medium comprising computer executable instructions which, when executed by a computer, cause the computer to perform the method of claim 1.

15. A method for recognizing the location of one or more characters within an image, comprising:

receiving a search string identifying the characters and a request to locate the characters within the image;

generating a binary representation of the image and the search string; and

in response to the request, locating the characters of the search string within the binary representation of the image and returning the location of the characters within the binary representation of the image.

16. The method of claim 15, wherein generating a binary representation of the image comprises:

designating a range for a color value of one or more pixels within the image;

determining whether the color value for each pixel within the image is within the range;

in response to the color value for a pixel being within the range, setting a value in the binary representation corresponding to the pixel, to a first value; and

in response to the color value for a pixel being outside the range, setting a value in the binary representation corresponding to the pixel, to a second value.

17. The method of claim 16, wherein generating a binary representation of the search string comprises:

determining a text code for each character within the search string;

based on the text code for each character, determining a vector value for each character; and

based on the vector value for each character, generating a binary representation of each character and thereby generating a binary representation of the search string.

18. The method of claim 17, further comprising:

generating an inverse of the binary representation of the search string; and

finding the inverse of the binary representation of the search string within the image;

in response to finding the inverse, returning a corresponding value indicating the search string is highlighted within the image; and

in response to finding the binary representation of the search string, returning a corresponding value indicating the search string is unselected within the image.

19. The method of claim 18, further comprising in response finding the inverse, reading and returning the location of the characters next to the highlighted search string.

20. The method of claim 15, wherein the location is expressed using one or more pixel coordinates.

21. A computer-controlled apparatus capable of performing the method of claim 15.

22. A computer-readable medium comprising computer executable instructions which, when executed by a computer, cause the computer to perform the method of claim 15.

23. A method for identifying a location of one or more characters within a display generated on a remote computer wherein the character locations are used for testing a computer program executing on the remote computer wherein the computer program is tested from a local computer utilizing an automated testing program, the method comprising:

receiving the display from the remote computer at the local computer;

generating a bitmap of the display;

receiving a search string identifying the characters and a request to locate the characters within the display;

generating a bitmap of the characters;

identifying the location of the bitmap of the characters within the display; and

in response to the request, returning the location of the bitmap of the characters within the display to the automated testing program executing on the local computer for use in testing the computer program executing on the remote computer.

24. The method of claim 23, wherein the characters comprise scalable fonts and the location is expressed using one or more pixel coordinates.

25. The method of claim 24, wherein the request further identifies a color value for the characters of the search string, and wherein the bitmap is generated using the character color value.

26. The method of claim 25, wherein the request further identifies a color value for the background of the search string, and wherein the bitmap is generated using the background color value.

27. The method of claim 25, wherein the request further identifies a font and a font size for the characters of the search string, and wherein the bitmap is generated using the font and the size.

28. The method of claim 27, wherein a segment of the character color value is stored and used to identify the location of the bitmap within the image.

29. The method of claim 28, wherein the request further comprises a designation as to whether the characters of the search string are bold, underlined, italic, or strikeout, and wherein the method further comprises:

in response to a bold designation, generating the bitmap using the bold designation;

in response to an underline designation, generating the bitmap using the underline designation;

in response to an italic designation, generating the bitmap using the italic designation; and

in response to a strikeout designation, generating the bitmap using the strikeout designation.

30. The method of claim 29, wherein the display of the remote computer is transmitted to the local computer by an input/output redirection device located on the remote computer.

31. The method of claim 30, wherein the remote computer and the local computer are physically located at two different locations and wherein the remote computer and the local computer are connected via a communications network.

32. The method of claim 31, wherein the computer program executing on the remote computer comprises an operating system program.

33. The method of claim 31, wherein the computer program executing on the remote computer comprises a basic input/output system program.

34. The method of claim 31, wherein the computer program executing on the remote computer comprises an application program.

35. A computer-controlled apparatus capable of performing the method of claim 23.

36. A computer-readable medium comprising computer executable instructions which, when executed by a computer, cause the computer to perform the method of claim 23.

37. A method for recognizing the coordinates one or more characters within a display generated on a first computer wherein the character coordinates are used for testing a computer program executing on the first computer wherein the computer program is tested from a second computer, the method comprising:

receiving the display of the first computer at the second computer;

receiving a search string identifying the characters and a request to locate the characters within the display;

generating a binary representation of the display and the search string;

in response to the request, locating the characters of the search string within the display and returning the location of the characters within the display and an indicator as to whether the characters are highlighted; and

utilizing the indicator and the coordinates of the characters in an automated test program executing on the second computer to test the computer program executing on the first computer.

38. The method of claim 37, wherein generating a binary representation of the display comprises:

designating a range for a color value of one or more pixels within the display;

determining whether the color value for each pixel within the display is within the range;

in response to the color value for a pixel being within the range, setting a value in the binary representation corresponding to the pixel, to a first value; and

in response to the color value for a pixel being outside the range, setting a value in the binary representation corresponding to the pixel, to a second value.

39. The method of claim 38, wherein generating a binary representation of the search string comprises:

determining the text code for each character within the search string;

based on the text code for each character, determining a vector value for each character; and

based on the vector value for each character, generating a binary representation of each character and thereby generating a binary representation of the search string.

40. The method of claim 39, further comprising:

generating an inverse of the binary representation of the search string;

finding the inverse of the binary representation of the search string within the display; and

in response to finding the inverse, returning a corresponding value indicating the search string is highlighted within the display.

41. The method of claim 40, further comprising in response finding the inverse, utilizing the coordinates to read and return the characters next to the highlighted search string.

42. The method of claim 41, wherein the location is expressed using one or more pixel coordinates.

43. The method of claim 37, wherein the display of the first computer is transmitted to the second computer by an input/output redirection device located on the first computer.

44. The method of claim 43, wherein the first computer and the second computer are physically located at two different locations and wherein the first computer and the second computer are connected via a communications network.

45. The method of claim 44, wherein the computer program executing on the first computer comprises an operating system program.

46. The method of claim 44, wherein the computer program executing on the remote computer comprises a basic input/output system program.

47. The method of claim 44, wherein the computer program executing on the remote computer comprises an application program.

48. A computer-controlled apparatus capable of performing the method of claim 37.

49. A computer-readable medium comprising computer executable instructions which, when executed by a computer, cause the computer to perform the method of claim 37.

50. A method for locating one or more characters within a display generated on a first computer wherein the character locations are used for testing a computer program executing on the first computer wherein the computer program is tested from a second computer using an automated test program, the method comprising:

receiving the display of the first computer at the second computer;

generating a binary representation of the display;

receiving a search string identifying the characters and a request to locate the characters within the display;

generating one or more binary representations of the characters to create a binary representation of the search string;

comparing the binary representations of the characters with the binary representation of the display;

identifying the location of the binary representation of the characters within the display;

in response to the request, returning the location of the binary representation of the characters within the display; and

from a local computer utilizing the location of the binary representation of the characters to navigate in the computer program executing on the first computer.