Method, apparatus, and terminal for obtaining vital sign data of target object

• US 9,984,461 B2
• Filed: 12/27/2016
• Issued: 05/29/2018
• Est. Priority Date: 06/27/2014
• Status: Active Grant

• Current Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Inventors: Tang, Weidong
• Primary Examiner(s): Tabatabai, Abolfazl
• Application Number:
  US15/391,068
• Publication Number:
  US 20170109885A1
• Time in Patent Office: 518 Days
• Field of Search: 382128, 382129, 382130, 382131, 382132, 382133, 382134, 382154, 348169, 345419
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20072 Graph-based image processing
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/62 of area, perimeter, diameter or volume
• CPC Class Codes: G06V 10/145 Illumination specially adapted for pattern recognition, e.g. using gratings
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 10/273 removing elements interfering with the pattern to be recognised
• CPC Class Codes: G06V 10/30 Noise filtering
• CPC Class Codes: G06V 10/426 Graphical representations
• CPC Class Codes: G06V 10/431 Frequency domain transformation; Autocorrelation
• CPC Class Codes: G06V 10/46 Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features colour feature extraction G06V10/56
• CPC Class Codes: G06V 10/60 relating to illumination properties, e.g. using a reflectance or lighting model
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects
• CPC Class Codes: G06V 20/653 by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces
• CPC Class Codes: G06V 2201/121 using special illumination
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

Embodiments of the present invention provide a method for obtaining vital sign data of a target object, including: obtaining a 3D depth image of a target object; obtaining, according to depth values of pixels in the 3D depth image of the target object, framework parameters of the target object and a graphic contour of the target object, where the depth value, is obtained according to the distance information, indicates a distance between a point on the target object and the imaging device; retrieving a 3D model matching the framework parameters of the target object and the graphic contour of the target object from a 3D model library, and obtaining a parameter ratio of the 3D model; obtaining at least one real size of the target object; and obtaining vital sign data of the target object according to the parameter ratio of the 3D model and the at least one real size.

Claims

1. A method for obtaining vital sign data of a target object, comprising:

obtaining a 3D depth image of a target object, wherein the 3D depth image is a two-dimensional image with distance information, and the distance information indicates a distance between the target object and an imaging device;

obtaining, according to depth values of pixels in the 3D depth image of the target object, framework parameters of the target object and a graphic contour of the target object, wherein the depth value indicates a distance, obtained according to the distance information, between a point on the target object and the imaging device;

retrieving a 3D model matching the framework parameters of the target object and the graphic contour of the target object from a 3D model library, and obtaining a parameter ratio of the 3D model;

obtaining at least one real size of the target object; and

obtaining vital sign data of the target object according to the parameter ratio of the 3D model and the at least one real size.

2. The method according to claim 1, wherein obtaining the 3D depth image of a target object comprises:

transmitting a reference pattern to the target object;

receiving a secondary pattern obtained from the reference pattern being reflected by the target object;

calculating an offset value of the secondary pattern relative to the reference pattern; and

obtaining the distance information by performing Fourier transformation on the offset value, and obtaining the 3D depth image by using the distance information.

3. The method according to claim 1, wherein obtaining the graphic contour of the target object according to depth values of pixels in the 3D depth image of the target object comprises:

performing difference calculation on the depth values of the pixels in the 3D depth image, to obtain the graphic contour of the target object, wherein performing the difference calculation on the depth values of the pixels in the 3D depth image comprises;

calculating a depth value difference between a depth value of a first pixel in the 3D depth image and a depth value of each of four neighboring pixels connected to the first pixel, to obtain four first depth difference values;

when at least one first depth difference value in the four first depth difference values is greater than a first difference threshold, marking a neighboring pixel corresponding to the at least one first depth difference value as a contour location;

querying whether a pixel marked as a contour location exists in eight neighboring pixels connected to a second pixel in the 3D depth image;

if the pixel marked as a contour location exists in the eight neighboring pixels connected to the second pixel in the 3D depth image, performing difference calculation between a depth value of the second pixel and a depth value of a pixel that is in the eight neighboring pixels connected to the second pixel and that is a non-contour location, to obtain a second depth difference value;

when at least one of the second depth difference value is greater than a second difference threshold, marking the second pixel as a contour location; and

obtaining the graphic contour of the target object according to pixels marked as contour locations.

4. The method according to claim 1, wherein after obtaining the 3D depth image of the target object, the method further comprises:

performing background noise reduction processing on the 3D depth image, to obtain a first 3D target depth image; and

, wherein

performing the difference calculation on the depth values of the pixels in the 3D depth image, to obtain the graphic contour of the target object comprises;

performing the difference calculation on depth values of pixels in the first 3D target depth image, to obtain the graphic contour of the target object.

5. The method according to claim 4, wherein performing the background noise reduction processing on the 3D depth image, to obtain a first 3D target depth image comprises:

setting a depth threshold;

comparing a depth value of each pixel in the 3D depth image with the depth threshold;

filtering out a pixel in the 3D depth image that has a depth value greater than the depth threshold; and

obtaining remaining pixels to form the first 3D target depth image.

6. The method according to claim 5, wherein after the obtaining the first 3D target depth image, the method further comprises:

performing edge noise reduction processing on the first 3D target depth image, to obtain a second 3D target depth image; and

, wherein

performing the difference calculation on the depth values of the pixels in the 3D depth image, to obtain the graphic contour of the target object comprises;

performing difference calculation on depth values of pixels in the second 3D target depth image, to obtain the graphic contour of the target object.

7. The method according to claim 6, wherein the performing edge noise reduction processing on the first 3D target depth image, to obtain a second 3D target depth image comprises:

segmenting the first 3D target depth image into multiple pixel blocks;

setting a pixel depth segment interval;

performing average processing on depth values of all pixels in each of the pixel blocks, to obtain a pixel average value of each of the pixel blocks; and

mapping the pixel average value to a corresponding interval in the pixel depth segment interval, and combining pixel blocks corresponding to all pixel average values in a same interval, to obtain the second 3D target depth image.

8. The method according to claim 1, wherein obtaining the framework parameters of the target object according to depth values of pixels in the 3D depth image of the target object comprises:

obtaining, by using a linear least square method, a central axis of the target object according to depth values of all pixels in the 3D depth image of the target object;

calculating transverse thicknesses of the graphic contour of the target object along multiple first lines perpendicular to the central axis;

calculating longitudinal thicknesses of the graphic contour of the target object along multiple second lines parallel to the central axis; and

constituting a framework of the target object by using areas limited by the first lines and the second lines, wherein the corresponding transverse thicknesses and longitudinal thicknesses are the framework parameters of the target object.

9. The method according to claim 8, wherein the retrieving a 3D model matching the framework parameters of the target object and the graphic contour of the target object from a 3D model library, and obtaining a parameter ratio of the 3D model comprises:

performing matching between the graphic contour of the target object and a graphic contour of a 3D model in the 3D model library, to obtain a graphic contour of a 3D model with a highest matching degree;

when the graphic contour of the 3D model is not a front-view graphic contour of the 3D model, obtaining the front-view graphic contour of the 3D model according to the graphic contour of the 3D model;

calculating a visual angle parameter of the 3D model according to the graphic contour of the 3D model and the front-view graphic contour of the 3D model, wherein the visual angle parameter is a visual angle that is of the graphic contour of the 3D model based on the front-view graphic contour of the 3D model;

rotating the front-view graphic contour of the 3D model based on the visual angle parameter, to obtain framework parameters of the 3D model;

obtaining, by means of comparison, a similarity between the framework parameters of the target object and the framework parameters of the 3D model, wherein when the similarity is less than a preset value, the 3D model is the 3D model matching the framework parameters of the target object and the graphic contour of the target object; and

obtaining the parameter ratio of the 3D model by using the 3D model.

10. The method according to claim 9, wherein the 3D model library comprises graphic contours of all visual angles of the 3D model, and comprises at least the front-view graphic contour of the 3D model.

11. The method according to claim 10, wherein performing the matching between the graphic contour of the target object and a graphic contour of a 3D model in the 3D model library, to obtain a graphic contour of a 3D model with a highest matching degree comprises:

describing the graphic contour of the target object by using a Zernike moment descriptor and a Fourier descriptor, to obtain first description information;

describing the graphic contour of the 3D model in the 3D model library by using the Zernike moment descriptor and the Fourier descriptor, to obtain second description information; and

comparing the first description information and the second description information, and using a graphic contour of a 3D model corresponding to second description information that differs from the first description information by a preset threshold as the graphic contour of the 3D model with the highest matching degree.

12. The method according to claim 1, wherein the obtaining at least one real size of the target object comprises:

transmitting a sound wave signal to the target object;

receiving a sound wave signal reflected by the target object;

obtaining transmission time of the sound wave signal, wherein the transmission time is a difference between time of transmitting the sound wave signal and time of receiving the sound wave signal;

calculating a distance between a surface of the target object and the imaging device by using the transmission time and a propagation velocity of the sound wave signal; and

calculating the at least one real size of the target object by using the distance and an image distance of the imaging device.

13. A terminal for obtaining vital sign data of a target object, comprising:

a 3D sensor, configured to obtain a 3D depth image of a target object, wherein the 3D depth image is a two-dimensional image with distance information, and the distance information indicates a distance between the target object and an imaging device; and

a processor, configured to obtain, according to depth values of pixels in the 3D depth image of the target object, framework parameters of the target object and a graphic contour of the target object, wherein the depth value indicates a distance, obtained according to the distance information, between a point on the target object and the imaging device, the processor is further configured to retrieve a 3D model matching the framework parameters of the target object and the graphic contour of the target object from a 3D model library, and obtain a parameter ratio of the 3D model, and the processor is further configured to obtain at least one real size of the target object and obtain vital sign data of the target object according to the parameter ratio of the 3D model and the at least one real size.

14. The terminal according to claim 13, wherein the 3D sensor is specifically configured to:

transmit a reference pattern to the target object, receive a secondary pattern obtained from the reference pattern being reflected by the target object, calculate an offset value of the secondary pattern relative to the reference pattern, obtain the distance information by performing Fourier transformation on the offset value, and obtain the 3D depth image by using the distance information.

15. The terminal according to claim 13, wherein the processor is further configured to perform difference calculation on the depth values of the pixels in the 3D depth image, to obtain the graphic contour of the target object;

and, wherein

performing the difference calculation on the depth values of the pixels in the 3D depth image comprises;

calculating a depth value difference between a depth value of a first pixel in the 3D depth image and a depth value of each of four neighboring pixels connected to the first pixel, to obtain four first depth difference values;

when at least one first depth difference value in the four first depth difference values is greater than a first difference threshold, marking a neighboring pixel corresponding to the at least one first depth difference value as a contour location;

querying whether a pixel marked as a contour location exists in eight neighboring pixels connected to a second pixel in the 3D depth image;

if the pixel marked as a contour location exists in the eight neighboring pixels connected to the second pixel in the 3D depth image, separately performing difference calculation between a depth value of the second pixel and a depth value of a pixel that is in the eight neighboring pixels connected to the second pixel and that is a non-contour location, to obtain a second depth difference value;

when at least one of the second depth difference value is greater than a second difference threshold, marking the second pixel as a contour location; and

obtaining the graphic contour of the target object according to pixels marked as contour locations.

16. The terminal according to claim 13, wherein the processor is further configured to perform background noise reduction processing on the 3D depth image, to obtain a first 3D target depth image;

and, wherein

performing the difference calculation on the depth values of the pixels in the 3D depth image, to obtain the graphic contour of the target object comprises;

performing the difference calculation on depth values of pixels in the first 3D target depth image, to obtain the graphic contour of the target object.

17. The terminal according to claim 16, wherein the processor is further configured to:

set a depth threshold; and

compare a depth value of each pixel in the 3D depth image with the depth threshold,

filter out a pixel in the 3D depth image that has a pixel depth value greater than the depth threshold, and

obtain remaining pixels to form the first 3D target depth image.

18. The terminal according to claim 17, wherein the processor is further configured to:

perform edge noise reduction processing on the first 3D target depth image, to obtain a second 3D target depth image; and

, wherein

performing the difference calculation on the depth values of the pixels in the 3D depth image, to obtain the graphic contour of the target object comprises;

performing the difference calculation on depth values of pixels in the second 3D target depth image, to obtain the graphic contour of the target object.

19. The terminal according to claim 18, wherein the processor is specifically configured to:

segment the first 3D target depth image into multiple pixel blocks;

set a pixel depth segment interval;

perform average processing on pixel depth values of all pixels in each of the pixel blocks, to obtain a pixel average value of each of the pixel blocks; and

map the pixel average value to a corresponding interval in the pixel depth segment interval, and combine pixel blocks corresponding to all pixel average values in a same interval, to obtain the second 3D target depth image.

20. The terminal according to claim 13, wherein the processor is specifically configured to:

obtain, by using a linear least square method, a central axis of the target object according to pixel depth values of all pixels in the 3D depth image of the target object;

calculate transverse thicknesses of the graphic contour of the target object along multiple first lines perpendicular to the central axis;

calculate longitudinal thicknesses of the graphic contour of the target object along multiple second lines parallel to the central axis; and

constitute a framework of the target object by using areas limited by the first lines and the second lines, wherein the corresponding transverse thicknesses and longitudinal thicknesses are the framework parameters of the target object.