3D skeletonization using truncated epipolar lines
First Claim
1. A computing device, comprising:
- a processing unit; and
a memory;
the computing device programmed, via computer-executable instructions, to perform operations for performing three-dimensional skeletonization, the operations comprising;
receiving multiple two-dimensional images of a target object, wherein the multiple two-dimensional images depict the target object from different perspectives;
identifying corresponding keypoints of the target object in the multiple two-dimensional images, wherein the identifying comprises projecting one or more truncated epipolar lines onto one or more of the multiple two-dimensional images using one or more keypoints in another one or more of the multiple two-dimensional images, wherein creating a truncated epipolar line comprises subtracting a truncation distance from a distance value associated with one of the keypoints, wherein the distance value is retrieved from a depth map associated with one of the multiple two-dimensional images;
mapping the corresponding keypoints to three-dimensional coordinates in a three-dimensional space; and
generating a three-dimensional skeleton of the target object using the three-dimensional coordinates.
1 Assignment
0 Petitions
Accused Products
Abstract
Technologies are provided for generating three-dimensional (3D) skeletons of target objects using images of the target objects captured from different viewpoints. Images of an object (such as a person) can be captured from different camera angles. Feature keypoints of the object can be identified in the captured images. Keypoints that identify a same feature in separate images can be correlated using truncated epipolar lines. For example, depth information for a keypoint can be used to truncate an epipolar line that is created using the keypoint. The correlated feature keypoints can be used to create 3D feature coordinates for the associated features of the object. A 3D skeleton can be generated using the 3D feature coordinates. One or more 3D models can be mapped to the 3D skeleton and rendered. The rendered one or more 3D models can be displayed on one or more display devices.
-
Citations
20 Claims
-
1. A computing device, comprising:
-
a processing unit; and a memory; the computing device programmed, via computer-executable instructions, to perform operations for performing three-dimensional skeletonization, the operations comprising; receiving multiple two-dimensional images of a target object, wherein the multiple two-dimensional images depict the target object from different perspectives; identifying corresponding keypoints of the target object in the multiple two-dimensional images, wherein the identifying comprises projecting one or more truncated epipolar lines onto one or more of the multiple two-dimensional images using one or more keypoints in another one or more of the multiple two-dimensional images, wherein creating a truncated epipolar line comprises subtracting a truncation distance from a distance value associated with one of the keypoints, wherein the distance value is retrieved from a depth map associated with one of the multiple two-dimensional images; mapping the corresponding keypoints to three-dimensional coordinates in a three-dimensional space; and generating a three-dimensional skeleton of the target object using the three-dimensional coordinates. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. A method, implemented by a computing device comprising a processing unit and a memory, the method comprising:
-
receiving images depicting a person from multiple viewpoints; identifying features of the person in the images; correlating the identified features using one or more truncated epipolar lines, wherein creating a truncated epipolar line comprises subtracting a truncation distance from a distance value associated with one of the features, wherein the distance value is retrieved from a depth map associated with one of the images; constructing three-dimensional feature coordinates for the identified features using the correlated identified features; and generating a three-dimensional skeleton of the person using the three-dimensional feature coordinates. - View Dependent Claims (12, 13, 14, 15, 16)
-
-
17. A system for generating a three-dimensional skeleton of a human, the system comprising:
-
multiple camera devices configured to capture images of a human from different viewpoints; and a computing device comprising a processor and a storage medium storing executable instructions that, when executed by the processor, program the computing device to perform operations, the operations comprising; receiving the captured images from the multiple camera devices; detecting keypoints in the captured images associated with body parts of the human; correlating the detected keypoints, wherein the correlating comprises projecting one or more truncated epipolar lines onto one or more of the captured images using one or more keypoints detected in another one or more of the captured images to determine that keypoints in separate images, of the captured images, are associated with a same body part depicted from different viewpoints, wherein creating a truncated epipolar line comprises subtracting a truncation distance from a distance value associated with one of the keypoints, wherein the distance value is retrieved from a depth map associated with one of the captured images; triangulating three-dimensional coordinates for the body parts using the correlated keypoints associated with the body parts; and generating a three-dimensional skeleton comprising the three-dimensional coordinates. - View Dependent Claims (18, 19, 20)
-
Specification