Viewpoint Control of a Display of a Virtual Product in a Virtual Environment

• US 20150371443A1
• Filed: 06/19/2014
• Published: 12/24/2015
• Est. Priority Date: 06/19/2014
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Troy, James J., Lea, Scott W., Wright, Daniel J.
• Granted Patent: US 9,355,498 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 15/50 Lighting effects
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A method, system, and apparatus for visually presenting a virtual environment relative to a physical workspace. An output device visually presents a view of the virtual environment to guide a human operator in performing a number of operations within the physical workspace. A mounting structure holds the output device and is movable with at least one degree of freedom relative to the physical workspace. A sensor system measures movement of the output device relative to the physical workspace to generate sensor data. A controller computes a transformation matrix and the set of scale factors to align the virtual environment and the physical workspace. The controller changes the view of virtual environment based on the sensor data to thereby change the view of the virtual environment in correspondence with the movement of the output device relative to the physical workspace.

Claims

1. An apparatus for visually presenting a virtual environment relative to a physical workspace, the apparatus comprising:

an output device that visually presents a view of the virtual environment to guide a human operator in performing a number of operations within the physical workspace;

a mounting structure that holds the output device and that is movable with at least one degree of freedom relative to the physical workspace;

a sensor system associated with at least one of the mounting structure or the output device, wherein the sensor system measures movement of the output device relative to the physical workspace to generate sensor data; and

a controller that computes a transformation matrix and a set of scale factors to align the virtual environment and the physical workspace, wherein the controller changes the view of the virtual environment based on the sensor data, the transformation matrix, and the set of scale factors;

whereby the controller changes the view of the virtual environment in correspondence with the movement of the output device relative to the physical workspace.

2. The apparatus of claim 1, wherein the sensor system comprises:

a sensor device that measures at least one of translational movement or rotational movement.

3. The apparatus of claim 2, wherein the mounting structure is an articulated structure and allows the output device to be moved relative to the physical workspace with the at least one degree of freedom up to six independent degrees of freedom.

4. The apparatus of claim 1 further comprising:

a support structure, wherein the mounting structure is movable relative to the support structure.

5. The apparatus of claim 1 further comprising:

an alignment system that enables an alignment of the physical workspace and the virtual environment by allowing a set of physical features in the physical workspace that correspond to a set of virtual features in the virtual environment to be located, wherein the set of physical features and the set of virtual features are preselected for the physical workspace and the virtual environment, respectively.

6. The apparatus of claim 5, wherein the controller uses the sensor data to compute a set of positions in three-dimensional coordinates for the set of physical features.

7. The apparatus of claim 6, wherein the controller establishes a coordinate system for the physical workspace based on the set of positions computed for the set of physical features and computes the transformation matrix for aligning the coordinate system for the physical workspace to a coordinate system for the virtual environment.

8. The apparatus of claim 5, wherein the alignment system comprises:

a laser device that emits a laser beam that allows the set of physical features to be located within the physical workspace; and

a lighting device associated with at least one of the mounting structure or the output device, wherein the lighting device illuminates a portion of the physical workspace for performing the number of operations within the physical workspace.

9. A system for visually presenting a virtual environment relative to a physical workspace, comprising:

an output device that visually presents a view of the virtual environment to guide a human operator in performing a number of operations within the physical workspace;

a support structure positioned relative to the physical workspace;

a mounting structure that holds the output device, is movable with at least one degree of freedom relative to the physical workspace, and is moveable relative to the support structure;

an alignment system that enables an alignment of the physical workspace and the virtual environment by allowing a set of physical features in the physical workspace that correspond to a set of virtual features in the virtual environment to be located, wherein the set of physical features and the set of virtual features are preselected for the physical workspace and the virtual environment, respectively;

a sensor system that measures movement of the output device relative to the physical workspace to generate sensor data; and

a controller that computes a set of positions for the set of physical features based on the sensor data and computes a transformation matrix and a set of scale factors to align the virtual environment and the physical workspace using the set of positions, wherein the controller changes the view of the virtual environment based on the sensor data, the transformation matrix, and the set of scale factors;

whereby the controller changes the view of the virtual environment in correspondence with the movement of the output device relative to the physical workspace.

10. A method for visually presenting a virtual environment relative to a physical workspace, the method comprising:

presenting, visually, a view of the virtual environment on an output device while the output device is positioned relative to the physical workspace;

computing a transformation matrix to perform an alignment between the virtual environment and the physical workspace and a set of scale factors to improve an accuracy of the alignment;

moving the output device relative to the physical workspace with respect to at least one degree of freedom using a mounting structure that holds the output device;

measuring movement of the output device relative to the physical workspace using a sensor system to generate sensor data; and

changing the view of the virtual environment based on the sensor data, the transformation matrix, and the set of scale factors whereby the view of the virtual environment is changed in correspondence with the movement of the output device relative to the physical workspace.

11. The method of claim 10 further comprising:

moving the output device to a preselected location relative to the physical workspace using the mounting structure.

12. The method of claim 10, wherein moving the mounting structure comprises:

moving the mounting structure at least one of translationally with up to three independent degrees of translational freedom or rotationally with up to three independent degrees of rotational freedom.

13. The method of claim 10, wherein measuring the movement of the mounting structure comprises:

measuring at least one of a translational movement or a rotational movement using a number of sensor devices in the sensor system.

14. The method of claim 10, wherein changing the view of the virtual environment comprises:

changing a viewpoint from which the virtual environment is visually presented in a display on the output device in correspondence with the movement of the output device relative to the physical workspace.

15. The method of claim 10 further comprising:

locating a set of physical features that has been preselected for the physical workspace using an alignment system.

16. The method of claim 15, wherein computing the transformation matrix and the set of scale factors to align the virtual environment and the physical workspace comprises:

computing a set of positions for the set of physical features using the sensor data;

establishing a coordinate system for the physical workspace using the set of positions computed; and

computing the transformation matrix that aligns the coordinate system for the physical workspace with a coordinate system for the virtual environment.

17. The method of claim 16, wherein computing the transformation matrix and the set of scale factors to align the virtual environment and the physical workspace further comprises:

computing a distance between positions of two physical features in the set of physical features; and

computing a ratio of the distance to a corresponding distance between two corresponding virtual features in the virtual environment as a scale factor in the set of scale factors.

18. The method of claim 10, wherein presenting, visually, the view of the virtual environment on the output device while the output device is positioned relative to the physical workspace comprises:

presenting, visually, a display on the output device that includes the view of the virtual environment.

19. The method of claim 18 further comprising:

determining whether the output device is in a preselected location relative to the physical workspace based on the sensor data; and

performing a number of actions in response to a determination that the output device is in the preselected location.

20. The method of claim 19, wherein performing the number of actions comprises:

performing at least one of adding instructions corresponding to the preselected location to the display, playing an audio recording of instructions or warnings, playing a video on another portion of the output device, visually presenting a new virtual element in the display, changing a state of existing virtual elements in the virtual environment in the display, sending a signal to another application implemented using a controller, sending data to another computer system over a communications link, or writing data to a file.