INFORMATION PROCESSING APPARATUS, INFORMATION PROCESSING METHOD, AND STORAGE MEDIUM

• US 20190066306A1
• Filed: 08/28/2018
• Published: 02/28/2019
• Est. Priority Date: 08/30/2017
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Masuda, Keita
• Granted Patent: US 10,593,044 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30088 Skin; Dermal
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/507 from shading G06T7/586 takes precedence
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/46 Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features colour feature extraction G06V10/56
• CPC Class Codes: G06V 10/476 using statistical shape modelling, e.g. point distribution models
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 40/107 Static hand or arm
• CPC Class Codes: G06V 40/11 Hand-related biometrics; Hand pose recognition

Abstract

An information processing apparatus includes a depth image acquisition unit configured to acquire a depth image from a measurement apparatus that has measured a distance to an object, an image acquisition unit configured to acquire a captured image from an image capturing apparatus that has captured an image of the object, and an estimation unit configured to estimate a shape of the object based on the depth image and the captured image. The estimation unit acquires information about a contour of the object from the captured image, corrects the information about the contour based on the depth image, and estimates the shape of the object based on the corrected information about the contour.

Claims

1. An information processing apparatus comprising:

a depth image acquisition unit configured to acquire a depth image from a measurement apparatus that has measured a distance to an object;

an image acquisition unit configured to acquire a captured image from an image capturing apparatus that has captured an image of the object; and

an estimation unit configured to estimate a shape of the object based on the depth image and the captured image,

wherein the estimation unit acquires information about a contour of the object from the captured image, corrects the information about the contour based on the depth image, and estimates the shape of the object based on the corrected information about the contour.

2. The information processing apparatus according to claim 1, wherein the object is a hand.

3. The information processing apparatus according to claim 1, wherein the estimation unit estimates the shape of the object by projecting a polygon model representing an area of the object on the depth image.

4. The information processing apparatus according to claim 1, wherein the estimation unit corrects the information about the contour of the object based on a gradient of a depth of the depth image.

5. The information processing apparatus according to claim 1, wherein the estimation unit corrects the information about the contour of the object based on a gradient of a color of the captured image.

6. The information processing apparatus according to claim 1, wherein the estimation unit corrects the information about the contour of the object based on an evaluation function for prioritizing each of a gradient of a depth of the depth image and a gradient of a color of the captured image.

7. The information processing apparatus according to claim 1, wherein the estimation unit executes no correction on the information about the contour of the object in an area in which the depth image is not present.

8. The information processing apparatus according to claim 1, further comprising a display unit configured to display the estimated shape of the object.

9. The information processing apparatus according to claim 2, wherein the estimation unit acquires the information about the contour of the object based on information about a skin color of the captured image.

10. The information processing apparatus according to claim 8, wherein the display unit causes a head mounted display (HMD) to display the estimated shape of the object.

11. An information processing method comprising:

acquiring a depth image from a measurement apparatus that has measured a distance to an object;

acquiring a captured image from an image capturing apparatus that has captured an image of the object; and

estimating a shape of the object based on the depth image and the captured image,

wherein, in the estimating, information about a contour of the object is acquired from the captured image, the information about the contour is corrected based on the depth image, and the shape of the object is estimated based on the corrected information about the contour.

12. The information processing method according to claim 11, wherein the object is a hand.

13. The information processing method according to claim 11, wherein, in the estimating, the shape of the object is estimated by projecting a polygon model representing an area of the object on the depth image.

14. The information processing method according to claim 11, wherein, in the estimating, the information about the contour of the object is corrected based on a gradient of a depth of the depth image.

15. The information processing method according to claim 11, wherein, in the estimating, the information about the contour of the object is corrected based on a gradient of a color of the captured image.

16. The information processing method according to claim 11, wherein, in the estimating, the information about the contour of the object is corrected based on an evaluation function for prioritizing each of a gradient of a depth of the depth image and a gradient of a color of the captured image.

17. The information processing method according to claim 11, wherein, in the estimating, no correction is executed on the information about the contour of the object in an area in which the depth image is not present.

18. The information processing method according to claim 11, wherein the estimated shape of the object is displayed.

19. The information processing method according to claim 12, wherein, in the estimating, the information about the contour of the object is acquired based on information about a skin color of the captured image.

20. A non-transitory storage medium storing a program causing a computer to execute an information processing method for estimating a shape of an object, the information processing method comprising:

acquiring a depth image from a measurement apparatus that has measured a distance to the object;

acquiring a captured image from an image capturing apparatus that has captured an image of the object; and

estimating the shape of the object based on the depth image and the captured image,

wherein, in the estimating, information about a contour of the object is acquired from the captured image, the information about the contour is corrected based on the depth image, and the shape of the object is estimated based on the corrected information about the contour.