Augmented reality with direct user interaction
First Claim
1. An augmented reality system, comprising:
- an image sensor arranged to capture images of an object in a user-interaction region;
a display device which is at least partially transparent and arranged to combine a displayed virtual environment with a view of the user-interaction region, such that both the virtual environment and the object are concurrently visible to a viewing user using the display device;
a processor arranged to receive the images, calculate a corresponding position within the virtual environment that is visually aligned with position of the object when viewed by the viewing user using the display device, and control the display device to update the virtual environment based on the corresponding position to enable an interaction with the virtual environment by a touch or gesture of the user, the processor being further arranged to selectively control the display device to render black pixels at locations in the virtual environment that are coincident with at least one hand of the user.
2 Assignments
0 Petitions
Accused Products
Abstract
Augmented reality with direct user interaction is described. In one example, an augmented reality system comprises a user-interaction region, a camera that captures images of an object in the user-interaction region, and a partially transparent display device which combines a virtual environment with a view of the user-interaction region, so that both are visible at the same time to a user. A processor receives the images, tracks the object'"'"'s movement, calculates a corresponding movement within the virtual environment, and updates the virtual environment based on the corresponding movement. In another example, a method of direct interaction in an augmented reality system comprises generating a virtual representation of the object having the corresponding movement, and updating the virtual environment so that the virtual representation interacts with virtual objects in the virtual environment. From the user'"'"'s perspective, the object directly interacts with the virtual objects.
-
Citations
20 Claims
-
1. An augmented reality system, comprising:
-
an image sensor arranged to capture images of an object in a user-interaction region; a display device which is at least partially transparent and arranged to combine a displayed virtual environment with a view of the user-interaction region, such that both the virtual environment and the object are concurrently visible to a viewing user using the display device; a processor arranged to receive the images, calculate a corresponding position within the virtual environment that is visually aligned with position of the object when viewed by the viewing user using the display device, and control the display device to update the virtual environment based on the corresponding position to enable an interaction with the virtual environment by a touch or gesture of the user, the processor being further arranged to selectively control the display device to render black pixels at locations in the virtual environment that are coincident with at least one hand of the user. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13)
-
-
14. A computer-implemented method of direct user-interaction in an augmented reality system, comprising:
-
controlling, using a processor, an at least partially transparent display device to display a virtual environment combined with a view of a user-interaction region, such that both the virtual environment and an object are concurrently visible and visually aligned to a viewing user using the display device; detecting movement of the object located in the user-interaction region using an image sensor; generating a virtual representation of the object having a corresponding movement in the virtual environment that is visually aligned with the movement of the object when viewed by the viewing user using the display device; updating the virtual environment using the display device such that the virtual representation interacts with one or more virtual objects in the virtual environment, such that, from the perspective of the user, the movement of the object viewed through the display device directly interacts with the one or more virtual objects in the virtual environment to enable an interaction with the virtual environment by allowing a touch or gesture of the user to manipulate the position of the one or more virtual objects; and selectively controlling the at least partially transparent display device to render black pixels at locations in the virtual environment that are coincident with at least one hand of the user. - View Dependent Claims (15, 16, 17, 18, 19)
-
-
20. An augmented reality system, comprising:
-
a display screen arranged to display a three-dimensional virtual environment comprising one or more virtual objects; a depth camera arranged to capture depth images of a user-controlled object located within a user-interaction region; a half-silvered mirror positioned to reflect light from the display screen and transmit light from within the user-interaction region, such that both the virtual environment and the user-controlled object are concurrently visible to the user using a surface of the half-silvered mirror; a processor arranged to;
control the display screen to display the virtual environment such that the virtual environment is visually aligned with the user-interaction region on a surface of the half-silvered mirror from the perspective of the user;
receive the depth images and track movement of the user-controlled object in three dimensions in the user-interaction region;
generate a virtual representation of the user-controlled object having a corresponding movement in the virtual environment that is visually aligned with the movement of the user-controlled object when viewed by the user on the half-silvered mirror; and
update the virtual environment on the display screen such that the virtual representation interacts with the one or more virtual objects in the virtual environment, such that, from the perspective of the user, the movement of the user-controlled object viewed through the half-silvered mirror directly interacts with the virtual objects in the virtual environment to enable an interaction with the virtual environment by allowing a touch or gesture of the user to manipulate the position of the one or more virtual objects, the processor being further arranged to selectively control the display screen to render black pixels at locations in the virtual environment that are coincident with at least one hand of the user.
-
Specification