METHOD AND SYSTEM OF AUGMENTED-REALITY SIMULATIONS

• US 20150206329A1
• Filed: 01/23/2014
• Published: 07/23/2015
• Est. Priority Date: 01/23/2014
• Status: Active Grant

• Current Assignee: Derek A. Devries
• Original Assignee: Derek A. Devries
• Inventors: Devries, Derek A.
• Granted Patent: US 9,454,220 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation

Abstract

In one exemplary embodiment, a method includes the step of obtaining a digital image of an object with a digital camera. The object is identified. A user query is received. The method includes the step of matching, with at least one processor, the user query with at least one manipulatable portion of a virtual view of the digital image of the object. The method include the step of obtaining the at least one manipulatable portion from a database of manipulatable portions of the object. The method includes the step of integrating the at least one manipulatable portion with the virtual view of the digital image of the object, wherein a manipulatable portion comprises a region of an augmented-reality element integrated into the virtual view of the digital image of the object, and wherein the augmented-reality element comprises a hyper link to another augmented-reality element comprising a virtual interior view of the object and wherein the camera obtains the predefined user gesture.

Claims

1. A method comprising:

obtaining a digital image of an object with a digital camera;

identifying the object;

receiving a user query;

matching, with at least one processor, the user query with at least one manipulatable portion of a virtual view of the digital image of the object;

obtaining the at least one manipulatable portion from a database of manipulatable portions of the object;

integrating the at least one manipulatable portion with the virtual view of the digital image of the object, wherein a manipulatable portion comprises a region of an augmented-reality element integrated into the virtual view of the digital image of the object, and wherein the augmented-reality element comprises a hyper link to another augmented-reality element comprising a virtual interior view of the object, wherein a user access the other augmented-reality element by performing a predefined user gesture with respect to a location of the object that corresponds to the at least one manipulatable portion, and wherein the digital camera obtains the predefined user gesture; and

displaying, with a computer display, the at least one manipulatable portion in the virtual view of the digital image of the object and the augmented-reality element.

2. The method of claim 1, wherein the computer display comprises an augmented reality goggles display system.

3. The method of claim 2, wherein the object comprises a building.

4. The method of claim 3,

wherein the manipulatable portion of the building comprises a wall region of the building,

wherein the user gesture comprises a pointing motion with a hand of the user, and

wherein the other augmented reality portion comprises a digital image of a view of one or more interior rooms of the building, wherein the digital image is obtained from a database of digital images comprising interior rooms of the building.

5. The method of claim 1, wherein the computer display comprises a smart phone display system.

6. The method of claim 1 further comprising:

receiving another user query with respect to the object.

7. The method of claim 6 further comprising:

receiving a user gesture from the user with respect to the at least one manipulatable portion with the virtual view of the digital image of the object.

8. The method of claim 7 further comprising:

determining the other user is viewing object through another computer display.

9. The method of claim 8 further comprising:

displaying, with the other computer display, the at least one manipulatable portion in the virtual view of the digital image of the object and the augmented-reality element.

10. A computerized system comprising:

a processor configured to execute instructions;

a memory containing instructions when executed on the processor, causes the processor to perform operations that;

obtain a digital image of an object with a digital camera;

identify the object;

receive a user query;

match the user query with at least one manipulatable portion of a virtual view of the digital image of the object;

obtain the at least one manipulatable portion from a database of manipulatable portions of the object;

integrate the at least one manipulatable portion with the virtual view of the digital image of the object, wherein a manipulatable portion comprises a region of an augmented-reality element integrated into the virtual view of the digital image of the object, and wherein the augmented-reality element comprises a hyper link to another augmented-reality element comprising a virtual interior view of the object, wherein a user access the other augmented-reality element by performing a predefined user gesture with respect to a location of the object that corresponds to the at least one manipulatable portion, and wherein the digital camera obtains the predefined user gesture;

determine that the user has performed the predefined user gesture with respect to a location of the object that corresponds to the augmented-reality element; and

display, with a computer display, the at least one manipulatable portion in the virtual view of the digital image of the object and the augmented-reality element.

11. The method of claim 10, wherein the computer display comprises an augmented reality goggles display system.

12. The method of claim 11, wherein the object comprises an automobile.

13. The method of claim 12,

wherein the manipulatable portion of the automobile comprises a hood region of the automobile,

wherein the user gesture comprises a lifting motion with a hand of the user, and

wherein the other augmented reality portion comprises an digital image of a view of one or more automobile systems.

14. The method of claim 10, wherein the memory containing instructions when executed on the processor, causes the processor to perform operations that:

receive another user query with respect to the object.

15. The method of claim 14, wherein the memory containing instructions when executed on the processor, causes the processor to perform operations that:

receiving a user gesture from the user with respect to the at least one manipulatable portion with the virtual view of the digital image of the object.

16. The method of claim 15, wherein the memory containing instructions when executed on the processor, causes the processor to perform operations that:

determining the other user is viewing object through another computer display.

17. The method of claim 16, wherein the memory containing instructions when executed on the processor, causes the processor to perform operations that:

displaying, with the other computer display, the at least one manipulatable portion in the virtual view of the digital image of the object and the augmented-reality element.

18. A method of an augmented-reality simulation:

providing, with at least one processor, an augmented-reality view with a display of a computing device, wherein the augmented-reality comprises a physical object and at least one augmented-reality element;

obtaining a digital image of a symbolic input drawn by a user,

obtaining at least one digital image of a first user hand gesture with respect to the symbolic input drawn by the user;

providing a one or more set of manipulatable portions of the augmented reality element;

determining which of the one or more set of manipulatable portions are relevant to the symbolic input and the first user hand gesture;

automatically identifying the physical object with an image recognition system;

receiving a user query via a user input system of the computer device;

matching, with at least one processor, the user query with at least one manipulatable portion of the augmented-reality view;

obtaining the at least one manipulatable portion from a database of manipulatable portions associated with the physical object;

integrating the at least one manipulatable portion with the augmented-reality view, wherein a manipulatable portion comprises a region of the augmented-reality element integrated into the augmented-reality view, and wherein the augmented-reality element comprises a hyper link to another augmented-reality element comprising a virtual interior view of the physical object;

determining that the user performed a predefined user gesture with respect to a location of the physical object that corresponds to the at least one manipulatable portion; and

displaying, with a computer display, the at least one manipulatable portion in the augmented-reality view.

19. The method of claim 18,

wherein the user input system comprises at least one of a microphone, a touch screen or a digital camera,

wherein the hand gesture comprises a pointing gesture with a user'"'"'s finger,

wherein the symbolic input comprises a text input and a button image drawn on a surface by the user.

20. The method of claim 19,

wherein the computing system comprises an eye-tracking system and a head-mounted display system, and

wherein the predefined user gesture comprises a user pausing a user eye-tracking motion on the for a specified period of time on the manipulatable portion and later regressing back to the manipulatable portion while the user is holding and turning the physical object.