Systems and methods of rerendering image hands to create a realistic grab experience in virtual reality/augmented reality environments
First Claim
1. A system of realistic displacement of a virtual object to render a realistic representation of a hand in a three-dimensional (3D) sensory space as a virtual object in a virtual space, the system including:
- a head mounted device depicting at least a visual presentation to a wearer;
a hand-gesture sensor attached to the head mounted device; and
a processor and a computer readable storage medium storing computer instructions configured for performing;
determining an offset between expected positions of one or more eyes of a wearer of a head mounted device and an actual position of a hand-gesture sensor attached to the head mounted device;
using the hand-gesture sensor, obtaining a sequence of images correlated to construct a 3-D solid model of the hand, including its position and shape, and using the 3-D solid model of the hand computing a sensed position of hand in a three-dimensional (3D) sensory space, wherein the computing includes determining positions of fingers, thumb and palm of the hand;
calculating a transformation matrix using the determined offset, such that the transformation matrix, when applied to the sensed position of the hand, decreases a rendered size of the hand in accordance with the determined offset;
applying the transformation matrix to the sensed position of the hand to transform the sensed position of the hand into a re-rendered position of the hand, the re-rendered position of the hand placing in the 3D sensory space the positions of the fingers, thumb and palm of the re-rendered position of the hand at the offset with respect to the sensed position of the hand such that the wearer of the head mounted device perceives the re-rendered position of the hand situated in the 3D sensory space as expected by the wearer when the wearer is looking at the actual hand in the 3D sensory space; and
depicting to the wearer of the head mounted device a re-rendered image hand using the re-rendered position of the hand.
9 Assignments
0 Petitions
Accused Products
Abstract
The technology disclosed relates to a method of realistic rendering of a real object as a virtual object in a virtual space using an offset in the position of the hand in a three-dimensional (3D) sensory space. An offset between expected positions of the eye(s) of a wearer of a head mounted device and a sensor attached to the head mounted device for sensing a position of at least one hand in a three-dimensional (3D) sensory space is determined. A position of the hand in the three-dimensional (3D) sensory space can be sensed using a sensor. The sensed position of the hand can be transformed by the offset into a re-rendered position of the hand as would appear to the wearer of the head mounted device if the wearer were looking at the actual hand. The re-rendered hand can be depicted to the wearer of the head mounted device.
27 Citations
22 Claims
-
1. A system of realistic displacement of a virtual object to render a realistic representation of a hand in a three-dimensional (3D) sensory space as a virtual object in a virtual space, the system including:
-
a head mounted device depicting at least a visual presentation to a wearer; a hand-gesture sensor attached to the head mounted device; and a processor and a computer readable storage medium storing computer instructions configured for performing; determining an offset between expected positions of one or more eyes of a wearer of a head mounted device and an actual position of a hand-gesture sensor attached to the head mounted device; using the hand-gesture sensor, obtaining a sequence of images correlated to construct a 3-D solid model of the hand, including its position and shape, and using the 3-D solid model of the hand computing a sensed position of hand in a three-dimensional (3D) sensory space, wherein the computing includes determining positions of fingers, thumb and palm of the hand; calculating a transformation matrix using the determined offset, such that the transformation matrix, when applied to the sensed position of the hand, decreases a rendered size of the hand in accordance with the determined offset; applying the transformation matrix to the sensed position of the hand to transform the sensed position of the hand into a re-rendered position of the hand, the re-rendered position of the hand placing in the 3D sensory space the positions of the fingers, thumb and palm of the re-rendered position of the hand at the offset with respect to the sensed position of the hand such that the wearer of the head mounted device perceives the re-rendered position of the hand situated in the 3D sensory space as expected by the wearer when the wearer is looking at the actual hand in the 3D sensory space; and depicting to the wearer of the head mounted device a re-rendered image hand using the re-rendered position of the hand. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. A method of realistic displacement of a virtual object to render a realistic representation of a hand in a three-dimensional (3D) sensory space as a virtual object in a virtual space, the method including:
-
determining an offset between expected positions of one or more eyes of a wearer of a head mounted device and an actual position of a hand-gesture sensor attached to the head mounted device; using the hand-gesture sensor, obtaining a sequence of images correlated to construct a 3-D solid model of the hand, including its position and shape, and using the 3-D solid model of the hand computing a sensed position of at least one hand in a three-dimensional (3D) sensory space, wherein the computing includes determining positions of fingers, thumb and palm of the hand; calculating a transformation matrix using the determined offset, such that the transformation matrix, when applied to the sensed position of the hand, decreases a rendered size of the hand in accordance with the determined offset; applying the transformation matrix to the sensed position of the hand to transform the sensed position of the hand into a re-rendered position of the hand, the re-rendered position of the hand placing in the 3D sensory space the positions of the fingers, thumb and palm of the re-rendered position of the hand at the offset with respect to the sensed position of the hand such that the wearer of the head mounted device perceives the re-rendered position of the hand situated in the 3D sensory space as expected by the wearer when the wearer is looking at the actual hand in the 3D sensory space; and depicting to the wearer of the head mounted device a re-rendered image hand using the re-rendered position of the hand. - View Dependent Claims (9, 10, 11, 12, 13)
-
-
14. One or more non-transitory computer readable media having instructions stored thereon for performing realistic displacement of a virtual object to render a realistic representation of a hand in a three-dimensional (3D) sensory space as a virtual object in a virtual space, the performing including:
-
determining an offset between expected positions of one or more eyes of a wearer of a head mounted device and an actual position of a hand-gesture sensor attached to the head mounted device; using the hand-gesture sensor, obtaining a sequence of images correlated to construct a 3-D solid model of the hand, including its position and shape, and using the 3-D solid model of the hand computing a sensed position of at least one hand in a three-dimensional (3D) sensory space, wherein the computing includes determining positions of fingers, thumb and palm of the hand; calculating a transformation matrix using the determined offset, such that the transformation matrix, when applied to the sensed position of the hand, decreases a rendered size of the hand in accordance with the determined offset; applying the transformation matrix to the sensed position of the hand to transform the sensed position of the hand into a re-rendered position of the hand, the re-rendered position of the hand placing in the 3D sensory space the positions of the fingers, thumb and palm of the re-rendered position of the hand at the offset with respect to the sensed position of the hand such that the wearer of the head mounted device perceives the re-rendered position of the hand situated in the 3D sensory space as expected by the wearer when the wearer is looking at the actual hand in the 3D sensory space; and depicting to the wearer of the head mounted device a re-rendered image hand using the re-rendered position of the hand. - View Dependent Claims (15, 16, 17, 18, 19, 20, 21, 22)
-
Specification