SENSOR-ENHANCED LOCALIZATION IN VIRTUAL AND PHYSICAL ENVIRONMENTS

• US 20130321391A1
• Filed: 06/01/2012
• Published: 12/05/2013
• Est. Priority Date: 06/01/2012
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Troy, James J., Puterbaugh, Kevin, Esposito, Christopher, Gass, John C.
• Granted Patent: US 9,058,681 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 2219/2004 Aligning objects, relative positioning of parts

Abstract

In one embodiment, a computer-based system comprises a measurement device, a display, a processor, and logic instructions stored in a tangible computer-readable medium coupled to the processor which, when executed by the processor, configure the processor to determine a position and orientation in a real three dimensional space of the measurement device relative to at least one real object in the three dimensional space and render on the display, a perspective view of a virtual image of a virtual object corresponding to the real object in a virtual three-dimensional space, wherein the perspective view of the virtual object corresponds to the perspective view of the real object from the position of the measurement device.

Claims

1. A computer-based method, comprising:

determining a position and orientation in a real three-dimensional space of a measurement device relative to at least one real object in the three-dimensional space; and

rendering on a display device coupled to the measurement device, a perspective view of a virtual image of a virtual object corresponding to the real object in a virtual three-dimensional space, wherein the perspective view of the virtual object corresponds to the perspective view of the real object from the position of the measurement device.

2. The method of claim 1, wherein determining a position and orientation in a real three-dimensional space of a measurement device relative to at least one object comprises:

receiving a signal indicating that the measurement device is positioned in a known physical location in the real three-dimensional space;

rendering, on the display device, a perspective view of a virtual image of at least one virtual object from the perspective of the known physical location; and

confirming a correspondence between the perspective view of the real object and the perspective view of the virtual object.

3. The method of claim 2, wherein the measurement device comprises an inertial measurement unit (IMU) and further comprising:

receiving a rotation input indicating a change in the orientation of the inertial measurement unit relative to the real object; and

rotating on the display device, the perspective view of the virtual object such that the perspective view of the virtual object aligns with the perspective view of the real object from the position and orientation derived from the data received from the measurement device.

4. The method of claim 3, wherein the rotation input comprises at least one of an input from the inertial measurement unit (IMU) or an input from an input device coupled to the IMU or the IMU data processing application and the display device.

5. The method of claim 3, wherein confirming a correspondence between the perspective view of the real object and the perspective view of the virtual object comprises at least one of:

receiving a translation input indicating a change in the position of the inertial measurement unit relative to the real object; and

updating on the display device, the perspective view of the virtual three dimensional space such that the perspective view of the virtual three dimensional space aligns with the perspective view of the real three-dimensional space from the position and orientation of the measurement device.

6. The method of claim 5, wherein the translation input comprises at least one of an input from the measurement device or an input from an input device coupled to the measurement device processing application and the display device.

7. The method of claim 1, further comprising

receiving an input in the display device to select one or more items of interest;

and

in response to the input;

displaying information about the one or more items of interest.

8. A computer-based system, comprising:

a measurement device;

a display;

a processor; and

logic instructions stored in a tangible computer-readable medium coupled to the processor which, when executed by the processor, configure the processor to;

determine a position and orientation in a real three-dimensional space of the measurement device relative to at least one real object in the three-dimensional space; and

render on the display, a perspective view of a virtual image of a virtual object corresponding to the real object in a virtual three-dimensional space, wherein the perspective view of the virtual object corresponds to the perspective view of the real object from the position of the measurement device.

9. The computer-based system of claim 8, wherein the logic instructions to determine a position and orientation in a real three-dimensional space further comprising logic instructions which, when executed by the processor, configure the processor to:

receive a signal indicating that the measurement device is positioned in a known physical location in the three-dimensional space;

render, on the display a perspective view of a virtual image of at least one virtual object from the perspective of the known physical location; and

confirm a correspondence between the perspective view of the real object and the perspective view of the virtual object.

10. The computer-based system of claim 9, further comprising logic instructions stored in a tangible computer-readable medium coupled to the processor which, when executed by the processor, configure the processor to:

receive a rotation input indicating a change in the orientation of the measurement device relative to the real object; and

rotate on the display device, the perspective view of the virtual object such that the perspective view of the virtual object aligns with the perspective view of the real object from the position and orientation of the measurement device.

11. The computer-based system of claim 10, wherein the rotation input comprises at least one of an input from the inertial measurement unit or an input from an an input device coupled to the measurement device processing application and the display device.

12. The computer-based system of claim 9, further comprising logic instructions stored in a tangible computer-readable medium coupled to the processor which, when executed by the processor, configure the processor to:

receive an input in the display device to select one or more items of interest; and

in response to the input, display information about the one or more items of interest.

13. The computer-based system of claim 12, wherein the input comprises at least one of an input from the measurement device or an input from an input device coupled to the measurement device processing application and the display device.

14. The computer-based system of claim 9, further comprising logic instructions stored in a tangible computer-readable medium coupled to the processor which, when executed by the processor, configure the processor to:

receive a confirmation input from an input device coupled to the measurement device processing application and the display device; and

receive a confirmation input from a matching process that determines a match between the perspective view of the virtual object and a real-time image of the real object.

15. A computer program product comprising logic instructions stored in a tangible computer-readable medium coupled to a processor which, when executed by the processor, configure the processor to:

determine a position and orientation in a real three-dimensional space of a measurement device relative to at least one real object in the three-dimensional space; and

render on a display device coupled to the measurement device, a perspective view of a virtual image of a virtual object corresponding to the real object in a virtual three-dimensional space, wherein the perspective view of the virtual object corresponds to the perspective view of the real object from the position of the measurement device.

16. The computer program product of claim 15, wherein the logic instructions to determine a position and orientation in a real three-dimensional space further comprising logic instructions which, when executed by the processor, configure the processor to:

receive a signal indicating that the measurement device is positioned in a known physical location in the real three-dimensional space;

render, on the display device, a perspective view of a virtual image of at least one virtual object from the perspective of the known physical location; and

confirm a correspondence between the perspective view of the real object and the perspective view of the virtual object.

17. The computer program product of claim 16, wherein the measurement device comprises an inertial measurement unit and further comprising further comprising logic instructions stored in a tangible computer-readable medium coupled to a processor which, when executed by the processor, configure the processor to:

receive a rotation input indicating a change in the orientation of the inertial measurement unit relative to the real object; and

rotate on the display device, the perspective view of the virtual object such that the perspective view of the virtual object aligns with the perspective view of the real object from the position and orientation of the measurement device.

18. The computer program product of claim 17, wherein the rotation input comprises at least one of an input from the inertial measurement unit or an input from an an input device coupled to the measurement device processing application and the display device.

19. The computer program product of claim 18, further comprising logic instructions stored in a tangible computer-readable medium coupled to the processor which, when executed by the processor, configure the processor to:

receive a translation input indicating a change in the position of the inertial measurement unit relative to the real object; and

update on the display device, the perspective view of the virtual three-dimensional space such that the perspective view of the virtual three-dimensional space aligns with the perspective view of the real three-dimensional space from the position and orientation of the measurement device

20. The computer program product of claim 19, wherein the translation input comprises at least one of an input from the inertial measurement unit or an input from an input device coupled to the measurement device processing application and the display device.

21. The computer program product of claim 15, further comprising logic instructions stored in a tangible computer-readable medium coupled to a processor which, when executed by the processor, configure the processor to:

receive an input in the display device to select one or more items of interest; and

in response to the input, display information about the one or more items of interest.