Visual overlay for augmenting reality

• US 8,743,145 B1
• Filed: 08/26/2010
• Issued: 06/03/2014
• Est. Priority Date: 08/26/2010
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Price, Roy F.
• Primary Examiner(s): Wu, Xiao
• Assistant Examiner(s): BUTTRAM, ALAN T
• Application Number:
  US12/869,294
• Time in Patent Office: 1,377 Days
• Field of Search: 345629-634, 715/757
• US Class Current: 345/633
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/954 Navigation, e.g. using categorised browsing
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

Augmented reality may be provided to one or more users in a real-world environment. For instance, information related to a recognized object may be displayed as a visual overlay appearing to be in the vicinity of the object in the real-world environment that the user is currently viewing. The information displayed may be determined based on at least one of captured images and transmissions from other devices. In one example, a portable apparatus receives a transmitted user ID and may submit the user ID to a remote computing device that compares a profile of a user corresponding to the user ID with a profile associated with the portable apparatus for determining, at least in part, information to be displayed as the visual overlay. As another example, the portable apparatus may include a camera to capture images that are analyzed for recognizing objects and identifying other users.

Claims

1. An apparatus comprising:

a transceiver configured to transmit and receive data;

a processor;

a memory communicatively coupled to the processor;

a module maintained in the memory and executed on the processor to;

receive a transmission related to an object in a vicinity of the apparatus, the transmission comprising an identifier (ID) corresponding to the object, wherein the object represents a different user, determine specific information related to the object based, at least in part, on the transmission,

transmit at least part of the specific information and the ID to a remote computing device to request the remote computing device to;

determine an identity of the object that represents the different user,

identify an outbound permission rule of the different user based on the ID, and

determine a permission rule determined based, at least in part, on finding a commonality between the outbound permission rule of the different user and an inbound permission rule of the user, and

receive additional information related to the object from the remote computing device based, at least in part, on the permission rule; and

a transparent or semi-transparent interface interposed between a user of the apparatus and the object, the interface comprising;

a visual overlay component to display, as a visual overlay on the interface, the specific information and the additional information related to the object while the object is at least partially within a real-world field of view of the user.

2. The apparatus as recited in claim 1, further comprising at least one camera for capturing images of the real-world field of view of the user, wherein:

the module recognizes the object based, at least in part, on the images; and

the module employs the images, at least in part, when determining placement of the specific information and additional information related to the object in the visual overlay.

3. The apparatus as recited in claim 1, further comprising one or more speakers for providing audio information relevant to the object to the user.

4. The apparatus as recited in claim 1, further comprising a tactile feedback device for providing tactile information relevant to the object to the user.

5. The portable apparatus as recited in claim 1, further comprising displaying at least one virtual figure in the visual overlay appearing to be in the field of view of the user.

6. The portable apparatus as recited in claim 5, wherein the virtual figure is displayed so as to appear to interact with the object in the field of view.

7. The portable apparatus as recited in claim 5, wherein:

the object comprises a predetermined path upon which the user is traveling; and

the virtual figure comprises a virtual person displayed as the visual overlay to appear to be traveling on the predetermined path ahead of the user.

8. The apparatus as recited in claim 1, wherein a signal related to the object is received from a portable device that is physically detached from the apparatus and wirelessly communicating with the apparatus.

9. A method comprising:

receiving, at a portable device of a first user, information related to an object associated with a second user;

recognizing, by using one or more processors, the object from among multiple objects in a field of view for associating the information with the object;

determining a display permission rule based, at least in part, on finding a commonality between an outbound permission rule of the second user and an inbound permission rule of the first user; and

displaying, in a vicinity of the object on an interface of the portable device, a portion of the information associated with the second user, in response to the displaying of the portion being permitted by the display permission rule, as a visual overlay in the field of view of the first user.

10. The method as recited in claim 9, further comprising:

detecting an identifier (ID) transmitted for identifying the object; and

providing the ID to a remote computing device to receive the information related to the object.

11. The method as recited in claim 10, wherein:

the displaying the information as a visual overlay is in the field of view of the first user;

the identifier (ID) of the object identifies the second user using another portable device for transmitting the identifier;

the identifier enables the remote computing device to locate a user profile of the second user; and

the user profile of the second user is compared with a user profile of the first user for determining, at least in part, the information related to the second user to be displayed to the first user in the visual overlay.

12. The method as recited in claim 11, wherein the user profile of the second user specifies that different information related to the second user be displayed depending on whether the user profile of the first user matches one or more categories of users.

13. The method as recited in claim 9, wherein the object is a person and displaying the information in the vicinity of the object as a visual overlay in the field of view further comprises changing an appearance of the person by at least one of:

changing the appearance of skin of the person;

changing the appearance of clothing of the person;

or

change the appearance of a face of the person.

14. The method as recited in claim 9, wherein the recognizing the object from among multiple objects in the field of view further comprises using at least one of:

shape recognition;

facial recognition;

measured radio frequency signal strength or direction;

infrared light reception characteristics;

or Global Positioning System information.

15. The method as recited in claim 9, wherein:

the receiving information related to the object in the field of view further comprises receiving the information from an application that examines image data captured from the field of view and carries out a recognition function to recognize the object from among the multiple objects; and

the application provides the information based, at least in part, on recognizing the object.

16. The method as recited in claim 15, wherein:

the application obtains an image of the object from a remote computing device based, at least in part, on an identifier corresponding to the object received by a wireless transmission; and

the recognition function includes a comparison of the image of the object obtained from the remote computing device with the image data captured from the field of view.

17. The method as recited in claim 9, wherein the receiving information related to the object in the field of view further comprises receiving an advertisement for one or more products to be displayed as a visual overlay in the field of view of the first user, the advertisement being specified by a user profile set by the second user in exchange for compensation when the advertisement is displayed in the vicinity to the object associated with the second user.

18. The method as recited in claim 17, further comprising filtering the advertisement based, at least in part, on the display permission rule;

and displaying the filtered advertisement in the vicinity of the object as a visual overlay.

19. A method comprising:

receiving a transmission of specific information related to an object in a vicinity of the user but not currently in the field of view of the user at a portable apparatus carried by a user, wherein the specific information is provided by another user in a vicinity of the user;

determining a display permission rule based, at least in part, on finding a commonality between an outbound permission rule of the other user and an inbound permission rule of the user;

displaying a visual overlay related to the specific information on an interface interposed between the user and a real-world field of view of the user in response to the displaying the visual overlay of the specific information being permitted by the display permission rule; and

providing alert information associated with the object in response to a detected condition, wherein the alert information comprises at least one of;

a sound, a visual signal, a vibration, or a message for a Braille device, wherein at least one of the visual overlay or the alert information indicates a direction for the user to look to bring the object within the field of view of the user.

20. The method as recited in claim 19, further comprising:

determining an advertisement for one or more products to be displayed to the user based at least in part on the specific information, the other user being compensated for providing the advertisement displayed adjacent to the other user on an interface of the user.

21. The method as recited in claim 20, wherein the determining the advertisement is based, at least in part, on finding the commonality between an outbound permission rule of the other user and an inbound permission rule of the user.

22. The method as recited in claim 19, wherein:

the object is a person located behind the user; and

the visual overlay or the alert information provides a notification that the person is behind the user.

23. The method as recited in claim 19, wherein the information comprises at least one of:

an email;

a text message;

a video message;

notification of a message or voicemail;

notification of a telephone call;

or

a video call.

24. A system comprising:

at least one computing device configured to implement one or more services, wherein the one or more services are configured to;

receive, at a portable device of a first user, information related to an object associated with a second user;

recognize the object from among multiple objects in a field of view for associating the information with the object;

determine a display permission rule based, at least in part, on finding a commonality between an outbound permission rule of the second user and an inbound permission rule of the first user; and

display, in a vicinity of the object on an interface of the portable device, a portion of the information associated with the second user, in response to the displaying of the portion being permitted by the display permission rule, as a visual overlay in the field of view of the first user.

25. The system as recited in claim 24, wherein the one or more services are further configured to:

detect an identifier (ID) transmitted for identifying the object; and

provide the ID to a remote computing device to receive the information related to the object.

26. The system as recited in claim 25, wherein:

the display of the information as a visual overlay is in the field of view of the first user;

the identifier (ID) of the object identifies the second user-using another portable device for transmitting the identifier;

the identifier enables the remote computing device to locate a user profile of the second user; and

the user profile of the second user is compared with a user profile of the first user for determining, at least in part, the information related to the second user to be displayed to the first user in the visual overlay.

27. The system as recited in claim 26, wherein the user profile of the second user specifies that different information related to the second user be displayed depending on whether the user profile of the first user matches one or more categories of users.

28. The system as recited in claim 24, wherein the object is a person and displaying the information in the vicinity of the object as a visual overlay in the field of view further comprises changing an appearance of the person by at least one of:

changing the appearance of skin of the person;

changing the appearance of clothing of the person;

or

change the appearance of a face of the person.

29. The system as recited in claim 24, wherein the recognizing the object from among multiple objects in the field of view further comprises using at least one of:

shape recognition;

facial recognition;

measured radio frequency signal strength or direction;

infrared light reception characteristics;

or Global Positioning System information.

30. The system as recited in claim 24, wherein:

the receiving information related to the object in the field of view further comprises receiving the information from an application that examines image data captured from the field of view and carries out a recognition function to recognize the object from among the multiple objects; and

the application provides the information based, at least in part, on recognizing the object.

31. The system as recited in claim 24, wherein the receiving information related to the object in the field of view further comprises receiving an advertisement for one or more products to be displayed as a visual overlay in the field of view of the first user, the advertisement being specified by a user profile set by the second user in exchange for compensation when the advertisement is displayed in the vicinity to the object associated with the second user.