Quasi-virtual objects in an augmented reality environment

• US 9,418,479 B1
• Filed: 12/23/2010
• Issued: 08/16/2016
• Est. Priority Date: 12/23/2010
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Worley, William Spencer III, Betts-LaCroix, Jonathan
• Primary Examiner(s): Tseng, Charles
• Application Number:
  US12/977,876
• Time in Patent Office: 2,063 Days
• Field of Search: 345629-641
• US Class Current: 1/1
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/235 using a wireless connection, e.g. infrared or piconet
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: A63F 13/27 characterised by a large display in a public venue, e.g. in a movie theatre, stadium or game arena
• CPC Class Codes: A63F 2300/8082 Virtual reality
• CPC Class Codes: G01B 11/2513 with several lines being projected in more than one direction, e.g. grids, patterns
• CPC Class Codes: G06F 2203/012 Walk-in-place systems for allowing a user to walk in a virtual environment while constraining him to a given position in the physical environment
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A virtual overlay alters the appearance of a quasi-virtual object within an augmented reality environment. This quasi-virtual object may display information to the user, accept input from the user via motions or physical interactions between the user and the object, or both. Quasi-virtual objects allow a virtual object to have a physical manifestation within the augmented reality environment.

Claims

1. An augmented reality system comprising:

a processor;

a projector and a camera coupled to the processor;

a moveable quasi-virtual object configured to present an image from the projector;

an augmented reality module coupled to the processor, the projector, and the camera, the augmented reality module comprising;

a tracking and control module configured to;

determine that an object resides within an environment;

identify the object as the quasi-virtual object; and

track the quasi-virtual object within the environment; and

a virtual overlay module configured to;

generate a virtual overlay image to alter an appearance of the quasi-virtual object;

cause a modification to the virtual overlay image based at least partly on a determination that the quasi-virtual object is within a predetermined distance from a different object; and

cause a second modification to the different object based at least partly on the determination that the quasi-virtual object is within the predetermined distance from the different object, the second modification to the different object including projecting a cautionary virtual overlay onto at least a portion of the different object.

2. The system of claim 1, the quasi-virtual object further comprising a plurality of watermarks configured to be imaged by the camera.

3. The system of claim 2, wherein:

the watermarks are configured to be imaged with a non-visible wavelength of light;

or

the watermarks comprise a pseudorandom noise pattern.

4. The system of claim 1, wherein the quasi-virtual object comprises a passive plastic shell.

5. The system of claim 1, further comprising a receiver coupled to the processor and wherein the quasi-virtual object comprises an active identification device configured emit a signal for receipt by the receiver which, when interpreted by the tracking and control module, uniquely identifies the quasi-virtual object.

6. The system of claim 5, wherein the active identification device comprises a radio-frequency identification tag.

7. The system of claim 1, wherein the quasi-virtual object comprises a machine readable tag configured to be scanned by the camera and which encodes a unique identifier which, when interpreted by the tracking and control module, identifies the quasi-virtual object.

8. The system of claim 1, wherein the augmented reality module further comprises a structured light module configured to scan the quasi-virtual object with structured light.

9. The system of claim 1, wherein the tracking and control module is further configured to determine an orientation of the quasi-virtual object.

10. The system of claim 1, wherein the augmented reality module is further configured to modify one or more features of the environment as one or more users approach one or more physical hazards.

11. The system of claim 1, wherein the tracking and control module is further configured to determine that the object initially resides within a first zone of multiple zones of the environment, the first zone being a designated zone in which the tracking and control module detects objects within the environment, wherein the camera is configured to capture a first image and a second image that each depicts the first zone, and wherein the tracking and control module is further configured to:

determine that the first image does not include the object within the first zone;

determine that the second image includes the object within the first zone; and

identify the object as the quasi-virtual object based at least partly on at least one of a shape of the object, dimensions of the object, surface characteristics of the object, or a position of the object within the first zone.

12. The system of claim 1, wherein the objects are not detectable by the tracking and control module within a second zone of the multiple zones.

13. The system of claim 1, wherein the modification to the virtual overlay image includes adjusting a brightness level of the virtual overlay image, the brightness level of the virtual overlay image incrementally decreasing as a distance between the quasi-virtual object and the different object decreases.

14. The system of claim 1, further comprising a speaker that outputs sound that is associated with the virtual overlay image, wherein the modification to the virtual overlay image includes adjusting a volume of the sound.

15. The system of claim 14, wherein the volume of the sound incrementally decreases as a distance between the quasi-virtual object and the different object decreases.

16. The system of claim 1, wherein the virtual overlay module is further configured to prevent presentation of the virtual overlay image in one or more areas that are occluded by a user.

17. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed, cause one or more processors to perform acts comprising:

identifying an object as a quasi-virtual object for application of a virtual overlay;

determining a location and an orientation of the quasi-virtual object in an augmented reality environment;

generating the virtual overlay for the quasi-virtual object;

presenting the virtual overlay on at least a portion of the quasi-virtual object; and

causing a modification to the virtual overlay based at least partly on a determination that the quasi-virtual object is within a predetermined distance from a different object, the modification to the virtual overlay including adjusting a brightness level of the virtual overlay, the brightness level of the virtual overlay decreasing as a distance between the quasi-virtual object and the different object decreases.

18. The one or more non-transitory computer-readable storage media of claim 17, wherein the identifying the quasi-virtual object comprises receiving data from an object identification tag or a watermark associated with the quasi-virtual object.

19. The one or more non-transitory computer-readable storage media of claim 17, wherein the determining the location and the orientation for the quasi-virtual object comprises scanning a scene including the quasi-virtual object with a structured light pattern.

20. The one or more non-transitory computer-readable storage media of claim 17, wherein the presenting comprises directing a projector to project an image of the virtual overlay onto at least a portion of the quasi-virtual object.

21. The one or more non-transitory computer-readable storage media of claim 17, wherein the presenting comprises directing a conformal display on a surface of the quasi-virtual object to present an image of the virtual overlay.

22. The one or more non-transitory computer-readable storage media of claim 17, wherein the generating the virtual overlay comprises:

scaling an image overlay to fit the quasi-virtual object;

adjusting the image overlay to compensate for color of the quasi-virtual object; and

modifying the image overlay to compensate for a topology of the quasi-virtual object.

23. The one or more non-transitory computer-readable storage media of claim 17, wherein the generating the virtual overlay comprises modifying the virtual overlay to compensate for and conceal visible light watermarks on the quasi-virtual object.

24. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed, cause one or more processors to perform acts comprising:

identifying a first object by determining that the first object is currently located within an environment;

generating a virtual overlay for the first object based at least in part on the first object residing within the environment;

presenting the virtual overlay on at least a portion of the first object;

tracking the first object to present the virtual overlay on at least a portion of the first object during movement of the first object within the environment; and

causing a modification to the virtual overlay based at least partly on a determination that the first object is within a predetermined distance from a second object, the modification to the virtual overlay including adjusting a brightness level of the virtual overlay, the brightness level of the virtual overlay decreasing as a distance between the first object and the second object decreases.

25. The one or more non-transitory computer-readable storage media of claim 24, further comprising determining a location and orientation of the first object at least in part by scanning a scene including the first object with a structured light pattern.

26. The one or more non-transitory computer-readable storage media of claim 24, further comprising adjusting the virtual overlay to compensate for a color of the first object.

27. The one or more non-transitory computer-readable storage media of claim 24, further comprising determining an orientation of the first object via one or more tracking targets disposed on the first object.

28. The one or more non-transitory computer-readable storage media of claim 24, wherein the presenting the virtual overlay comprises projecting at least a portion of the virtual overlay onto the first object.

29. The one or more non-transitory computer-readable storage media of claim 24, wherein the presenting the virtual overlay on at least a portion of the first object comprises sending at least a portion of the virtual overlay to the first object for presentation on a display.