PROVIDING VEHICLE OWNER'S MANUAL INFORMATION USING OBJECT RECOGNITION IN A MOBILE DEVICE

• US 20150347848A1
• Filed: 06/02/2014
• Published: 12/03/2015
• Est. Priority Date: 06/02/2014
• Status: Active Grant

• Current Assignee: General Motors LLC (General Motors Company)
• Original Assignee: General Motors LLC (General Motors Company)
• Inventors: Fry, Keith A., McCutchen, Shane M., Rudman, Daniel E., Gurovich, Danilo E., Goree, Joseph M. II
• Granted Patent: US 9,552,519 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

There is provided a communication system that includes a vehicle and a mobile device, and a method uses that system to provide vehicle owner'"'"'s manual information. The method includes the steps of: receiving a digital image of a portion of a vehicle; identifying an object of the vehicle in the digital image; associating the object with at least one reference object image stored in an object library comprising a plurality of reference object images; displaying owner'"'"'s manual information based on the association; and storing at least a portion of the digital image in the object library as a new reference object image for future identifications.

Claims

1. A computer program product, comprising:

a non-transitory computer readable medium having stored thereon program instructions for use by a camera-enabled mobile device, wherein, upon execution by the mobile device, the program instructions stored on the computer readable medium cause the mobile device to carry out the steps of;

receiving a digital image of a portion of a vehicle;

identifying a primary object of the vehicle in the digital image;

associating the primary object with at least one reference object image stored in an object library comprising a plurality of reference object images;

displaying owner'"'"'s manual information based on the association; and

storing the digital image in the object library as a new reference object image for future identifications.

2. The computer program product of claim 1, wherein the identifying step includes determining the identity of the primary object based the primary object'"'"'s location within a centralized region of the digital image.

3. The computer program product of claim 1, wherein the object library includes more than one reference object image for at least some vehicle primary objects.

4. The computer program product of claim 3, wherein for at least some vehicle primary objects, the more than one reference object image includes one or more of the reference object image captured at a first photographic angle, the reference object image captured at a second photographic angle different than the first photographic angle, and a non-photographic version of the reference object image.

5. The computer program product of claim 1, wherein the storing step further comprises one of:

storing the entire digital image in the object library, or storing only a portion of the digital image in the object library.

6. The computer program product of claim 1, wherein the new reference object image has at least one different lighting or color characteristic than the previously stored reference object images.

7. The computer program product of claim 1, wherein the displaying step further comprises displaying one or more menu items directly-associated with the primary object, or displaying one or more predictive menu items, wherein the predictive menu items are indirectly-associated with the primary object, or both.

8. The computer program product of claim 1, further comprising automatically highlighting at least one object in the digital scan to suggest owner'"'"'s manual information that is available to the user.

9. The computer program product of claim 1, wherein the program instructions further comprise identifying a secondary object of the vehicle in the digital image while concurrently identifying the primary object.

10. The computer program product of claim 9, wherein the program instructions further comprise:

associating the secondary object with at least one reference object image stored in an object library,

wherein the displaying step includes;

displaying information directly-associated with both the first and second objects, or

displaying information pertaining to a classification of objects to which both the first and second objects are associated.

11. The computer program product of claim 10, wherein the step of identifying the secondary object includes determining the identity of the secondary object based the secondary object'"'"'s location outside the centralized region of the digital image.

12. The computer program product of claim 1, wherein the program instructions further comprise:

receiving speech audio containing one or more identifying terms associated with at least one object identified in the digital image;

determining the one or more identifying terms using an automatic speech recognition (ASR) engine; and

determining the owner'"'"'s manual information to be displayed based at least in part on the one or more identifying terms.

13. A computer program product, comprising:

a non-transitory computer readable medium having stored thereon program instructions for use by a camera-enabled mobile device, wherein, upon execution by the mobile device, the program instructions stored on the computer readable medium cause the mobile device to carry out the steps of;

receiving a digital scan containing at least one vehicle object;

receiving speech audio containing one or more words associated with vehicle object(s);

identifying the at least one vehicle object based on the digital scan and the one or more words of the speech audio; and

displaying owner'"'"'s manual information associated with the at least one vehicle object.

14. The computer program product of claim 13, the identifying step further comprising associating the at least one vehicle object with at least one reference object image stored in an object library comprising a plurality of reference object images.

15. The computer program product of claim 14, further comprising storing the digital scan in the object library as a new reference object image for future identifications, or storing the speech audio in an audio library for future identifications, or both.

16. The computer program product of claim 14, the identifying step further comprising automatically highlighting at least one object in the digital scan to suggest owner'"'"'s manual information that is available to the user.

17. The computer program product of claim 14, wherein the at least one reference object image includes one or more of a reference object image captured at a first photographic angle, the same reference object image captured at a second photographic angle different than the first photographic angle, and a graphic-design version of the same reference object image.

18. The computer program product of claim 13, wherein the at least one vehicle object includes a first vehicle object and a second vehicle object, wherein the displaying step further comprises concurrently displaying owner'"'"'s manual information pertaining to both the first and second vehicle objects.

19. The computer program product of claim 18, wherein the first vehicle object is located within a centralized region of the digital scan and the second vehicle object is located at least partially outside of the centralized region, wherein the owner'"'"'s manual information pertaining to the first vehicle object is displayed more prominently than the owner'"'"'s manual information pertaining to the second vehicle object.

20. A method of providing vehicle owner'"'"'s manual information, comprising the steps of:

receiving a digital image of a portion of a vehicle;

identifying an object of the vehicle in the digital image;

associating the object with at least one reference object image stored in an object library comprising a plurality of reference object images;

displaying owner'"'"'s manual information based on the association; and

storing at least a portion of the digital image in the object library as a new reference object image for future identifications.