AUGMENTED REALITY CREATION USING A REAL SCENE

• US 20130307875A1
• Filed: 02/08/2012
• Published: 11/21/2013
• Est. Priority Date: 02/08/2012
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Anderson, Glen J.
• Granted Patent: US 9,330,478 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: A63F 13/92 Video game devices specially adapted to be hand-held while playing
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay

Abstract

The creation of augmented reality is described using a real scene. In one example, a process includes observing a real scene through a camera of a device, observing a user gesture through the camera of the device, presenting the scene and the gesture on the display of the device, generating a virtual object and placing it in the scene based on the observed user gesture, and presenting the virtual object in the real scene on the display.

Claims

1. A method comprising:

observing a real scene through a camera of a device;

observing a user gesture through the camera of the device;

presenting the scene and the gesture on the display of the device;

generating a virtual object and placing it in the scene based on the observed user gesture; and

presenting the virtual object in the real scene on the display.

2. The method of claim 1, further comprising observing further user gestures through the camera of the display and wherein generating a virtual object comprises selecting from among a plurality of virtual objects.

3. The method of claim 1, further comprising

observing further user gestures through the camera of the display;

and

modifying characteristics of the virtual object based on the further user gestures.

4. The method of claim 3, wherein the characteristics comprise size and rotational position.

5. The method of claim 3, wherein the characteristics comprise behavior of the virtual object in time.

6. The method of claim 1, further comprising:

observing further user gestures through the camera of the display; and

determining a path of movement of the virtual object across the real scene based on the further user gestures.

7. The method of claim 1, further comprising:

tracking a location of the virtual object in the real scene as the virtual object moves through the real scene; and

updating the presentation of the virtual object on the display using the tracked location.

8. The method of claim 7, further comprising:

presenting real objects in the real scene on the display;

presenting an interaction between the real objects and the virtual objects on the display; and

presenting the effect of the interaction on the real object on the display.

9. The method of claim 8, wherein presenting an interaction comprises presenting an interaction when the tracked location of the virtual object is proximate a real object

10. The method of claim 1, further comprising determining a location of the virtual object in the real scene and storing a record of the location.

11. The method of claim 10, further comprising transferring data regarding the virtual object location to another device for display on the other device.

12. The method of claim 1, further comprising presenting audio representations of the virtual object in the real scene, and modifying the audio representations using the tracked location.

13. The method of claim 1, further comprising receiving an additional command as at least one of a touch screen gesture, a keyboard input, and a voice command and wherein generating the virtual object comprises generating the virtual object using the received additional command.

14. A computer-readable medium having instructions stored thereon that, when executed by the computer, cause the computer to perform operations comprising:

observing a real scene through a camera of a device;

observing a user gesture through the camera of the device;

presenting the scene and the gesture on the display of the device;

generating a virtual object and placing it in the scene based on the observed user gesture; and

presenting the virtual object in the real scene on the display.

15. The medium of claim 14, the operations further comprising:

presenting real objects in the real scene on the display;

tracking a location of the virtual object in the real scene as the virtual object moves through the real scene;

updating the presentation of the virtual object on the display using the tracked location,

presenting an interaction between the real objects and the virtual objects on the display; and

presenting the effect of the interaction on the real object on the display.

16. The medium of claim 15, wherein presenting an interaction comprises presenting an interaction when the tracked location of the virtual object is proximate a real object

17. A device comprising:

a gesture recognition module to observe a real scene and to observe a user gesture through a camera of a device;

a screen rendering module to present the scene and the gesture on a display of the device;

an object generation module to generate a virtual object; and

a virtual object tracker module to place the generated virtual object and place it in the scene based on the observed user gesture,

wherein the screen rendering module is further to present the virtual object in the real scene on the display.

18. The device of claim 17, wherein the gesture recognition module observes further user gestures through the camera of the display, the device further comprising a virtual object behavior module to modify characteristics of the virtual object based on the further user gestures.

19. The device of claim 18, wherein the characteristics comprise size and rotational position.

20. A system comprising:

a rear camera to observe a real scene and to observe a user gesture;

a processor to generate a virtual object and place it in the scene based on the observed user gesture;

a display to present the virtual object in the scene and the gesture; and

a speaker to present a sound effect associated with the displayed virtual object.

21. The system of claim 20, further comprising a touch screen to sense user gestures and wherein the processor generate a virtual object based on a selection from among a plurality of virtual objects selected by touching the touch screen.

22. The system of claim 20, further comprising a location subsystem to determine a location of the virtual object in the real scene and a storage device to store a record of the location.