User augmented reality for camera-enabled mobile devices

• US 8,606,317 B2
• Filed: 10/31/2012
• Issued: 12/10/2013
• Est. Priority Date: 10/18/2007
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Yahoo! Inc. (Apollo Global Management, Inc.)
• Inventors: Ahmad Athsani, Athellina R., Kalaboukis, Chris T.
• Primary Examiner(s): WANG-HURST, KATHY W
• Application Number:
  US13/664,876
• Publication Number:
  US 20130057585A1
• Time in Patent Office: 405 Days
• Field of Search: 455/566, 455/556.1, 455/556.2, 455/457, 455/550.1, 725/62, 725/81
• US Class Current: 455/557
• CPC Class Codes: G06F 16/434 using image data, e.g. images, photos, pictures taken by a user
• CPC Class Codes: G06F 16/487 using geographical or spatial information, e.g. location
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: H04M 1/72403 with means for local support of applications that increase the functionality
• CPC Class Codes: H04M 1/72427 for supporting games or graphical animations
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions
• CPC Class Codes: H04M 2250/52 including functional features of a camera

Abstract

Apparatus and methods are described for providing a user augmented reality (UAR) service for a camera-enabled mobile device, so that a user of such mobile device can use the mobile device to obtain meta data regarding one or more images/video that are captured with such device. The meta data is interactive and allows the user to obtain additional information or specific types of information, such as information that will aid the user in making a decision regarding the identified objects or selectable action options that can be used to initiate actions with respect to the identified objects.

Claims

1. A computer-implemented method for providing information regarding a moving object captured in an image or video by a camera of a mobile device, comprising:

receiving the image or video captured by the camera of the mobile device;

processing the image or video using an image processing technique to identify the moving object;

generating meta data relating to the moving object, the meta data comprising information about the moving object that relates to a social network of a user of the mobile device;

causing a visual representation of the meta data to be overlaid on a visual representation of the image or video in the display of the mobile device; and

causing action information to be displayed in the display of the mobile device, the action information being configured to enable the user to perform a specific action relating to the moving object and the social network.

2. The method of claim 1 wherein the visual representation of the meta data includes one or more of text, an image, video, audio, or a link.

3. The method of claim 1 further comprising causing decision support information to be displayed in the display of the mobile device, the decision support information being configured to help the user of the mobile device make decisions about specific actions relating to the moving object.

4. The method of claim 1 further comprising determining a location associated with the mobile device, wherein identification of the moving object is accomplished with reference to the location.

5. The method of claim 1 wherein generating the meta data comprises conducting a search relating to the moving object, thereby generating one or more search results.

6. The method of claim 1 wherein the moving object is a person, and wherein the meta data include publicly available information about the person.

7. The method of claim 1 further comprising storing the meta data in a database, and responding to a search using at least a portion of the meta data, the search relating to a second image or video determined to be similar to the image or video captured by the camera of the mobile device.

8. The method of claim 1 further comprising storing the meta data in a database, and providing at least a portion of the meta data to the user in response to a request from the user.

9. A mobile device for providing information regarding a moving object captured in an image or video, comprising:

a camera for capturing the image or video;

a display;

at least one a processor and at least one memory, the at least one processor and at least one memory being configured to;

transmit the image or video to a remote device for processing using an image processing technique to identify the moving object;

receive meta data from the remote device relating to the moving object, the meta data comprising information about the moving object that relates to a social network of a user of the mobile device;

overlay a visual representation of the meta data on a visual representation of the image or video in the display of the mobile device; and

display action information in the display of the mobile device, the action information being configured to enable the user to perform a specific action relating to the moving object and the social network.

10. The mobile device of claim 9 wherein the visual representation of the meta data includes one or more of text, an image, video, audio, or a link.

11. The mobile device of claim 9 wherein the at least one processor and at least one memory are further configured to display decision support information in the display of the mobile device, the decision support information being configured to help the user of the mobile device make decisions about specific actions relating to the moving object.

12. The mobile device of claim 9 wherein the at least one processor and at least one memory are further configured to determine a location associated with the mobile device, and transmit the location to the remote device, wherein identification of the moving object by the remote device is accomplished with reference to the location.

13. The mobile device of claim 9 wherein the meta data comprises search results of a search relating to the moving object.

14. The mobile device of claim 9 wherein the moving object is a person, and wherein the meta data include publicly available information about the person.

15. A wearable display device configured to provide information regarding an object in a field of view of a user wearing the device, comprising:

goggles or glasses configured to be worn by the user;

a camera mounted on the goggles or glasses for capturing an image of the field of view;

circuitry configured to transmit the image to a processor for image processing, and to receive meta data from the processor regarding the object in the field of view that relates to a social network of a user of the mobile device; and

a display configured to display the meta data regarding the object and action information in the field of view of the user.

16. The wearable display device of claim 15 wherein the circuitry is a wireless transceiver and the processor is included in a remote device.

17. The wearable display device of claim 15 wherein the wearable display device is also a mobile communication device.

18. The wearable display device of claim 15 wherein the display includes a screen in the field of view of the user.