CROSS-REALITY SELECT, DRAG, AND DROP FOR AUGMENTED REALITY SYSTEMS

• US 20140282162A1
• Filed: 03/15/2013
• Published: 09/18/2014
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Elwha LLC (Intellectual Ventures LLC)
• Original Assignee: Elwha LLC (Intellectual Ventures LLC)
• Inventors: Fein, Gene, Levien, Royce A., Lord, Richard T., Lord, Robert W., Malamud, Mark A., Rinaldo, John D. Jr., Tegreene, Clarence T.
• Granted Patent: US 10,025,486 B2
• Time in Patent Office: Days
• US Class Current: 715/769
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers

Abstract

Methods, apparatuses, computer program products, devices and systems are described that carry out detecting a first action of a user at a location in a real world field of view of an augmented reality device; displaying an augmented reality representation in response to at least one of a user input or detecting a first action of a user at a location in a real world field of view of an augmented reality device; moving the displayed augmented reality representation on a display of the augmented reality device according to at least one detected second action of the user; and registering the displayed augmented reality representation at a location in the display of the augmented reality device in response to at least one of a user input or moving a displayed augmented reality representation on a display of the augmented reality device according to at least one detected second action of the user.

Claims

1. A system comprising:

circuitry for detecting a first action of a user at a location in a real world field of view of an augmented reality device;

circuitry for displaying an augmented reality representation in response to at least one of a user input or an output of the circuitry for detecting a first action of a user at a location in a real world field of view of an augmented reality device;

circuitry for moving the displayed augmented reality representation on a display of the augmented reality device according to at least one detected second action of the user; and

circuitry for registering the displayed augmented reality representation at a location in the display of the augmented reality device in response to at least one of a user input or an output of the circuitry for moving a displayed augmented reality representation on a display of the augmented reality device according to at least one detected second action of the user.

2. The system of claim 1 wherein the circuitry for detecting a first action of a user at a location in a real world field of view of an augmented reality device comprises:

circuitry for detecting at least one of a pointing gesture indicating selection of an object, a voice command indicating selection of an object, or a touch command indicating selection of an object of a user at a location in a real world field of view of an augmented reality device.

5. The system of claim 1 wherein the circuitry for detecting a first action of a user at a location in a real world field of view of an augmented reality device comprises:

circuitry for detecting a first action of a user at a location in a real world field of view of an augmented reality device, wherein the circuitry is further configured to highlight on a display of the augmented reality device one or more objects for possible selection near the location in the real world field of view.

6. The system of claim 1 wherein the circuitry for detecting a first action of a user at a location in a real world field of view of an augmented reality device comprises:

circuitry for detecting a first action of a user at a location in a real world field of view of an augmented reality device, wherein the circuitry is further configured to detect the first action of the user based on one or more features of the real world field of view.

7. The system of claim 6 wherein the circuitry for detecting a first action of a user at a location in a real world field of view of an augmented reality device, wherein the circuitry is further configured to detect the first action of the user based on one or more features of the real world field of view comprises:

circuitry for detecting a first action of a user at a location in a real world field of view of an augmented reality device, wherein the circuitry is further configured to detect the first action of the user based on at least one of a business feature, an object type feature, a facial feature, a text feature, an image feature, or a sound feature of the real world field of view.

8. The system of claim 1 wherein the circuitry for detecting a first action of a user at a location in a real world field of view of an augmented reality device comprises:

circuitry for detecting a first action of a user at a location in a real world field of view of an augmented reality device, wherein the circuitry is further configured to detect the first action of the user based on one or more constraints present within the augmented reality device.

9. The system of claim 8 wherein the circuitry for detecting a first action of a user at a location in a real world field of view of an augmented reality device, wherein the circuitry is further configured to detect the first action of the user based on one or more constraints present within the augmented reality device comprises:

circuitry for detecting a first action of a user at a location in a real world field of view of an augmented reality device, wherein the circuitry is further configured to detect the first action of the user based on at least one of a program running on the augmented reality device, a time of day, a geographic location detected by the augmented reality device, or a mode of the augmented reality device.

12. The system of claim 1 wherein the circuitry for detecting a first action of a user at a location in a real world field of view of an augmented reality device comprises:

circuitry for detecting at least one of image data, audio data, or touch data indicative of a first action of a user at a location in a real world field of view of an augmented reality device.

13. The system of claim 12 wherein the circuitry for detecting at least one of image data, audio data, or touch data indicative of a first action of a user at a location in a real world field of view of an augmented reality device comprises:

circuitry for detecting at least one of a hand, finger, or pointing implement gesture at a location in a real world field of view of an augmented reality device.

14. The system of claim 12 wherein the circuitry for detecting at least one of image data, audio data, or touch data indicative of a first action of a user at a location in a real world field of view of an augmented reality device comprises:

circuitry for detecting a voice command indicating selection of an object at a location in a real world field of view of an augmented reality device.

17. The system of claim 1 wherein the circuitry for detecting a first action of a user at a location in a real world field of view of an augmented reality device comprises:

circuitry for detecting a touch by the user of a real world object at a location in a real world field of view of an augmented reality device.

19. The system of claim 1 wherein the circuitry for displaying an augmented reality representation in response to at least one of a user input or an output of the circuitry for detecting a first action of a user at a location in a real world field of view of an augmented reality device comprises:

circuitry for displaying at least one of a photo, a computer generated image, or a cartoon in response to at least one of a user input or an output of the circuitry for detecting a first action of a user at a location in a real world field of view of an augmented reality device.

21. The system of claim 1 wherein the circuitry for displaying an augmented reality representation in response to at least one of a user input or an output of the circuitry for detecting a first action of a user at a location in a real world field of view of an augmented reality device comprises:

circuitry for displaying an augmented reality representation in response to an output of the circuitry for detecting a first action of a user at a location in a real world field of view of an augmented reality device, wherein the output of the circuitry for detecting a first action of a user at a location in a real world field of view of an augmented reality device matches at least one of a stored gesture, a stored image, or a stored voice command.

22. The system of claim 1 wherein the circuitry for moving the displayed augmented reality representation on a display of the augmented reality device according to at least one detected second action of the user comprises:

circuitry for dragging the displayed augmented reality representation across the display of the augmented reality device according to at least one detected second action of the user.

26. The system of claim 1 wherein the circuitry for registering the displayed augmented reality representation at a location in the display of the augmented reality device in response to at least one of a user input or an output of the circuitry for moving a displayed augmented reality representation on a display of the augmented reality device according to at least one detected second action of the user comprises:

circuitry for registering the displayed augmented reality representation at a drop destination in the display of the augmented reality device in response to at least one of a user input or an output of the circuitry for moving a displayed augmented reality representation on a display of the augmented reality device according to at least one detected second action of the user.

29. The system of claim 1 wherein the circuitry for registering the displayed augmented reality representation at a location in the display of the augmented reality device in response to at least one of a user input or an output of the circuitry for moving a displayed augmented reality representation on a display of the augmented reality device according to at least one detected second action of the user comprises:

circuitry for registering the displayed augmented reality representation at a location in the display of the augmented reality device in response to at least one of a user gesture, a user voice command, or a user touch command indicating that a drop destination has been reached.

3-4. -4. (canceled)

10-11. -11. (canceled)

15-16. -16. (canceled)

18. (canceled)

20. (canceled)

23-25. -25. (canceled)

27-28. -28. (canceled)

30. A computer-implemented method comprising:

detecting a first action of a user at a location in a real world field of view of an augmented reality device;

displaying an augmented reality representation in response to at least one of a user input or detecting a first action of a user at a location in a real world field of view of an augmented reality device;

moving the displayed augmented reality representation on a display of the augmented reality device according to at least one detected second action of the user; and

registering the displayed augmented reality representation at a location in the display of the augmented reality device in response to at least one of a user input or moving a displayed augmented reality representation on a display of the augmented reality device according to at least one detected second action of the user.

31-58. -58. (canceled)

59. A computer program product comprising:

an article of manufacture including a signal-bearing medium bearing;

(1) one or more instructions for detecting a first action of a user at a location in a real world field of view of an augmented reality device;

(2) one or more instructions for displaying an augmented reality representation in response to at least one of a user input or detecting a first action of a user at a location in a real world field of view of an augmented reality device;

(3) one or more instructions for moving the displayed augmented reality representation on a display of the augmented reality device according to at least one detected second action of the user; and

(4) one or more instructions for registering the displayed augmented reality representation at a location in the display of the augmented reality device in response to at least one of a user input or moving a displayed augmented reality representation on a display of the augmented reality device according to at least one detected second action of the user.

60-62. -62. (canceled)

63. A system comprising:

a computing device; and

instructions that when executed on the computing device cause the computing device to

(1) detect a first action of a user at a location in a real world field of view of an augmented reality device;

(2) display an augmented reality representation in response to at least one of a user input or detecting a first action of a user at a location in a real world field of view of an augmented reality device

(3) move the displayed augmented reality representation on a display of the augmented reality device according to at least one detected second action of the user; and

(4) register the displayed augmented reality representation at a location in the display of the augmented reality device in response to at least one of a user input or moving a displayed augmented reality representation on a display of the augmented reality device according to at least one detected second action of the user.

64. (canceled)

65. A system comprising:

means for detecting a first action of a user at a location in a real world field of view of an augmented reality device;

means for displaying an augmented reality representation in response to at least one of a user input or detecting a first action of a user at a location in a real world field of view of an augmented reality device;

means for moving the displayed augmented reality representation on a display of the augmented reality device according to at least one detected second action of the user; and

means for registering the displayed augmented reality representation at a location in the display of the augmented reality device in response to at least one of a user input or moving a displayed augmented reality representation on a display of the augmented reality device according to at least one detected second action of the user.

66. A system comprising:

circuitry for detecting a first action of a user with respect to a displayed augmented reality representation at a location in a display of an augmented reality device;

circuitry for moving the displayed augmented reality representation on the display of the augmented reality device according to at least one detected second action of the user; and

circuitry for registering the displayed augmented reality representation at a location in a real world field of view of the augmented reality device in response to at least one of a user input or an output of the circuitry for moving the displayed augmented reality representation on the display of the augmented reality device according to at least one detected second action of the user.

67-89. -89. (canceled)