Designation of zones of interest within an augmented reality environment

• US 9,111,326 B1
• Filed: 12/21/2010
• Issued: 08/18/2015
• Est. Priority Date: 12/21/2010
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Rawles, LLC (Amazon.com, Inc.)
• Inventors: Worley, William Spencer III, Weatherford, William Thomas, Coley, Christopher, Crump, Edward Dietz, Betts-LaCroix, Jonathan
• Primary Examiner(s): Sams, Michelle L
• Application Number:
  US12/975,175
• Time in Patent Office: 1,701 Days
• Field of Search: 348153-155, 348/159, 340/541, 715/863
• US Class Current: 1/1
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/00 Image analysis
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/113 Recognition of static hand signs
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

An augmented reality system is configured to identify and track user gestures, sounds, and interaction with physical objects to designate active zones. These active zones may be allocated additional processing and functional resources. Gestures may include particular hand or body motions, orientation of a user'"'"'s head, and so forth. Sounds may include clapping, clicks, whistles, taps, footfalls, humming, singing, speech, and so forth. Active areas as well as inactive areas of lesser or no interest may be designated as well.

Claims

1. An augmented reality functional node comprising:

a processor;

a projector and a camera coupled to the processor;

a spatial analysis module configured to scan a scene;

an augmented reality module configured to generate an augmented reality environment associated with the scene, the augmented reality module comprising;

a tracking and control module configured to determine one or more zones with input activity above a pre-determined threshold and over a pre-determined period of time, designate the one or more zones as active zones and allocate scanning and interpretation resources to actions occurring within the one or more active zones to determine one or more user inputs.

2. The system of claim 1, wherein the active zone is designated upon receipt of a pre-determined attention command.

3. The system of claim 2, wherein the pre-determined attention command comprises an audible signal.

4. The system of claim 3, wherein a user at least in part generates the audible signal.

5. The system of claim 3, wherein the audible signal comprises human speech.

6. The system of claim 2, wherein the pre-determined attention command comprises a signal generated by a signaling device.

7. The system of claim 1, wherein the tracking and control module is further configured to designate one or more inactive zones and de-allocate scanning and interpretation resources within the one or more inactive zones while monitoring for attention command inputs from within the one or more inactive zones.

8. The system of claim 1, wherein the spatial analysis module is configured to scan the scene to sense inaudible input and one or more auditory sensors configured to sense audible input.

9. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed, cause one or more processors to perform acts comprising:

determining a zone within a scene with input activity below a pre-determined threshold and over a pre-determined period of time;

designating the zone as an inactive zone;

storing an indication that the zone is designated as an inactive zone;

monitoring the inactive zone for an attention command input;

accepting the attention command input from within the inactive zone and disregarding or not detecting other input from within the inactive zone.

10. The one or more non-transitory computer-readable storage media of claim 9, wherein the pre-determined threshold of input activity comprises a count of inputs received.

11. The one or more non-transitory computer-readable storage media of claim 9, wherein the monitoring the inactive zone comprises using a reduced scan rate, reduced set of sensors, or both compared to monitoring an active zone.

12. The one or more non-transitory computer-readable storage media of claim 9, further comprising:

receiving the attention command input;

designating a volume as an active zone within the inactive zone; and

monitoring the active zone for input.

13. The one or more non-transitory computer-readable storage media of claim 12, wherein the volume of the active zone is proximate to or encompasses at least a portion of a source the attention command input.

14. The one or more non-transitory computer-readable storage media of claim 12, wherein the monitoring the active zone comprises a greater scan rate, additional set of sensors, or both compared to monitoring the inactive zone.

15. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed, cause one or more processors to perform acts comprising:

determining a zone with input activity above a pre-determined threshold and over a pre-determined period of time;

designating the zone as an active zone;

monitoring the active zone for input;

accepting the input from within the active zone;

monitoring for attention command input outside the active zone; and

accepting the attention command input from outside the active zone and disregarding or not detecting other input from outside the active zone.

16. The one or more non-transitory computer-readable storage media of claim 15, wherein the pre-determined threshold of input activity comprises a count of inputs received.

17. The one or more non-transitory computer-readable storage media of claim 15, the monitoring for attention command input inside and outside the active zone comprising a reduced scan rate, reduced set of sensors, or both compared to monitoring the active zone for input.

18. The one or more non-transitory computer-readable storage media of claim 15, further comprising:

receiving the attention command input;

designating a volume as an active zone within an inactive zone; and

monitoring the active zone for input.

19. The one or more non-transitory computer-readable storage media of claim 18, wherein the volume of the active zone is proximate to or encompasses the attention command input.

20. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed, cause one or more processors to perform acts comprising:

identifying an object within an augmented reality scene;

when the object is identified as a designated input object;

designating a zone of the augmented reality scene proximate to or encompassing the object as an active zone of the augmented reality scene;

monitoring the active zone of the augmented reality scene for input;

accepting the input from within the active zone of the augmented reality scene;

when the object is identified as a non-designated input object;

designating a zone of the augmented reality scene proximate to or encompassing the object as an inactive zone of the augmented reality scene;

storing an indication that the zone of the augmented reality scene proximate to or encompassing the object is designated as an inactive zone of the augmented reality scene;

monitoring the inactive zone of the augmented reality scene for an attention command input; and

accepting the attention command input from within the inactive zone of the augmented reality scene and disregarding or not detecting other input from within the inactive zone of the augmented reality scene.

21. The one or more non-transitory computer-readable storage media of claim 20, wherein the designated input object comprises at least a portion of a user.

22. The one or more non-transitory computer-readable storage media of claim 20, wherein the designated input object comprises a portable projection screen.

23. The one or more non-transitory computer-readable storage media of claim 20, wherein a size of the zone associated with the object is proportionate to the function of the object.

24. The one or more non-transitory computer-readable storage media of claim 20, wherein the attention command input comprises a signal initiated by a user.

25. A method comprising:

receiving an attention command input within an augmented reality environment, the attention command input comprising a signal generated at least in part by a user;

designating, by one or more hardware processors, a first zone of the environment as an active zone within the environment;

receiving other input commands from within the active zone of the environment;

designating a second zone of the environment as an inactive zone of the environment;

storing an indication that the second zone of the environment is designated as an inactive zone of the environment;

monitoring the inactive zone of the environment for the attention command input; and

accepting the attention command input from within the inactive zone of the environment and disregarding or not detecting other input from within the inactive zone of the environment.

26. The method of claim 25, wherein the signal comprises speech of the user.

27. The method of claim 25, wherein the active zone is configured to track an object associated with the signal.

28. The method of claim 25, wherein the signal comprises an audible signal and wherein the receiving further comprises locating within the environment the source of the attention command with acoustic location via a plurality of microphones within the environment.

29. The method of claim 25, wherein the volume of the active zone is proximate to or encompasses the attention command input.