METHOD AND DEVICE FOR ACQUIRING TARGET OBJECT, AND ROBOT

• US 20190311491A1
• Filed: 06/24/2019
• Published: 10/10/2019
• Est. Priority Date: 12/22/2016
• Status: Active Grant

• Current Assignee: Shenzhen Kuang-Chi Hezhong Technology, Ltd.
• Original Assignee: Shenzhen Kuang-Chi Hezhong Technology, Ltd.
• Inventors: LIU, Ruopeng, OUYANG, Yicun
• Granted Patent: US 11,127,151 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20072 Graph-based image processing
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/20216 Image averaging
• CPC Class Codes: G06T 2207/20224 Image subtraction
• CPC Class Codes: G06T 7/10 Segmentation; Edge detection motion-based segmentation G06T7/215
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/162 involving graph-based methods
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

A method and device for acquiring a target object, and a robot are provided. The method includes: acquiring a plurality of frames of continuous images in video images, where each frame of image includes a first target object; acquiring, based on the continuous images including the first target object, a first target image from which a background has been removed; acquiring, based on a distance from a first center point of the first target image to a common boundary, a location of a second target object in the first target image, where the first center point is determined based on a color of the second target object, the common boundary is an overlapping boundary of the second target object and the first target object, and the second target object is included in the first target object; and extracting the second target object based on the location of the second target object in the first target image.

Claims

1. A method for acquiring a target object, comprising:

acquiring a plurality of frames of continuous images in video images, wherein each frame of image comprises a first target object;

acquiring, based on the continuous images comprising the first target object, a first target image from which a background has been removed;

acquiring, based on a distance from a first center point of the first target image to a common boundary, a location of a second target object in the first target image, wherein the first center point is determined based on a color of the second target object, the common boundary is an overlapping boundary of the second target object and the first target object, and the second target object is comprised in the first target object; and

extracting the second target object based on the location of the second target object in the first target image.

2. The method according to claim 1, wherein the acquiring, based on the continuous images comprising the first target object, a first target image from which a background has been removed comprises:

obtaining a weight distribution graph of a current frame of image based on weights of a color value of the first target image in the current frame of image among the plurality of frames of continuous images, wherein the weights are used to represent weights of the color value of the first target object in all color values of the current frame of image;

determining, in the weight distribution graph, a square comprising the first target object; and

acquiring, based on the square of the first target object, the first target image from which the background has been removed.

3. The method according to claim 2, wherein the obtaining a weight distribution graph of a current frame of image based on weights of a color value of the first target image in the current frame of image among the plurality of frames of continuous images comprises:

acquiring the weights of the color value of the first target object in the plurality of frames of images; and

mixing and filtering the weights of the color value of the first target object in the plurality of frames of images, to obtain the weight distribution graph of the current frame of image.

4. The method according to claim 3, wherein the acquiring the weights of the color value of the first target object in the plurality of frames of images comprises:

storing a first acquired image comprising the first target object, and acquiring a background image and a gradient background image of the first target object based on the first image comprising the first target object;

subtracting a background image of a previous frame of image from the color value of the current frame of image, to obtain a first weight;

subtracting a gradient background image of the previous frame of image from the color value of the current frame of image, to obtain a second weight;

subtracting the background image of the previous frame of image from a color gradient of the current frame of image, to obtain a third weight;

subtracting a preset gradient in gradients of the background image of the previous frame of image from the color value of the current frame of image, to obtain a fourth weight; and

subtracting a main color value of the first target object from the color value of the current frame of image, to obtain a fifth weight.

5. The method according to claim 4, wherein the mixing and filtering the weights of the color value of the first target object in the plurality of frames of images, to obtain the weight distribution graph of the current frame of image comprises:

performing weighting on the first weight, the second weight, the third weight, the fourth weight, and the fifth weight based on a preset ratio, to obtain a mixed weight;

performing weight processing on the mixed weight, wherein a weight used to perform the weight processing tends to decrease from a center of the current frame of image to a boundary of the current frame of image;

adding an image obtained after the weight processing to weights in a weight distribution graph of the previous frame of image, and performing averaging processing; and

extracting a preset quantity of weights from weights obtained after the averaging processing, and subtracting the preset quantity of weights from the weights obtained after the averaging processing, to obtain the weight distribution graph of the current frame of image.

6. The method according to claim 2, wherein the determining, in the weight distribution graph, a square comprising the first target object comprises:

determining a plurality of squares based on color values in the weight distribution graph, and selecting, from the plurality of squares, a square satisfying a preset condition, wherein the preset condition comprises;

a weight in the square is higher than a preset target value;

an area of the square is within a first preset range;

a length of the square in a first dimension and a length of the square in a second dimension are within a second preset range, wherein the first dimension is perpendicular to the second dimension;

or

a difference between a total weight of squares of the current frame of image and an average total weight of a preset quantity of previous images is within a third preset range.

7. The method according to claim 6, wherein the determining a plurality of squares based on color values in the weight distribution graph comprises:

step A;

determining a point with a maximum weight in the weight distribution graph;

step B;

using the point with the maximum weight as a second center point, and expanding a square to a fourth preset range;

step C;

after the square is expanded to the fourth preset range, shrinking the square, until an average value of color values at edges of the square is less than an average value of a preset ratio and a sum of the color values of the image;

step D;

clearing color values in a range determined by the square; and

step E;

repeating step A to step D, to obtain the plurality of squares.

8. The method according to claim 7, wherein the acquiring, based on the square of the first target object, the first target image from which the background has been removed comprises:

determining a main color of the first target object, and determining a similar color of the main color, wherein a color difference between the similar color and the main color is within a preset color difference range;

determining the first center point in coordinates of the similar color, wherein coordinates of the first center point are closest to the second center point among coordinates of all similar colors; and

determining a preset quantity of points outward in a preset direction by using the first center point as a center, and if any point indicates a similar color of the main color, connecting points that indicate the similar color of the main color, and performing expansion outward and re-determining points, until determined points do not comprise the similar color.

9. The method according to claim 1, wherein the acquiring, based on a distance from a first center point of the first target image to a common boundary, a location of a second target object in the first target image comprises:

determining an upper limit and a lower limit of the second target object based on a largest distance from the first center point to the common boundary; and

removing points above the upper limit and points below the lower limit from the location, to obtain the second target object.

10. The method according to claim 9, wherein after the second target object is acquired, the method further comprises:

acquiring image information of the second target object, and storing the image information in a preset format, wherein the image information comprises coordinates of a square and a size of the second target object.

11. A device for acquiring a target object, comprising:

a first acquisition module, configured to acquire a plurality of frames of continuous images in video images, wherein each frame of image comprises a first target object;

a second acquisition module, configured to acquire, based on the continuous images comprising the first target object, a first target image from which a background has been removed;

a third acquisition module, configured to acquire, based on a distance from a first center point of the first target image to a common boundary, a location of a second target object in the first target image, wherein the first center point is determined based on a color of the second target object, the common boundary is an overlapping boundary of the second target object and the first target object, and the second target object is comprised in the first target object; and

an extraction module, configured to extract the second target object based on the location of the second target object in the first target image.

12. The device according to claim 11, wherein the second acquisition module comprises:

a first acquisition submodule, configured to obtain a weight distribution graph of a current frame of image based on weights of a color value of the first target image in the current frame of image among the plurality of frames of continuous images, wherein the weights are used to represent weights of the color value of the first target object in all color values of the current frame of image;

a first determining submodule, configured to determine, in the weight distribution graph, a square comprising the first target object; and

a second acquisition submodule, configured to acquire, based on the square of the first target object, the first target image from which the background has been removed.

13. The device according to claim 12, wherein the first acquisition submodule comprises:

an acquisition unit, configured to acquire the weights of the color value of the first target object in the plurality of frames of images; and

a mixing and filtering unit, configured to mix and filter the weights of the color value of the first target object in the plurality of frames of images, to obtain the weight distribution graph of the current frame of image.

14. The device according to claim 11, wherein the third acquisition module comprises:

a second determining submodule, configured to determine an upper limit and a lower limit of the second target object based on a largest distance from the first center point to the common boundary; and

a removal submodule, configured to remove points above the upper limit and points below the lower limit from the location, to obtain the second target object.

15. The device according to claim 14, wherein the device further comprises:

a storage module, configured to acquire image information of the second target object, and store the image information in a preset format, wherein the image information comprises coordinates of a square and a size of the second target object.

16. A robot, comprising the device for acquiring a target object according to claim 11.

17. The device according to claim 13, wherein the acquisition unit is configured to:

store a first acquired image comprising the first target object, and acquire a background image and a gradient background image of the first target object based on the first image comprising the first target object;

subtract a background image of a previous frame of image from the color value of the current frame of image, to obtain a first weight;

subtract a gradient background image of the previous frame of image from the color value of the current frame of image, to obtain a second weight;

subtract the background image of the previous frame of image from a color gradient of the current frame of image, to obtain a third weight;

subtract a preset gradient in gradients of the background image of the previous frame of image from the color value of the current frame of image, to obtain a fourth weight; and

subtract a main color value of the first target object from the color value of the current frame of image, to obtain a fifth weight.

18. The device according to claim 17, wherein the mixing and filtering unit is configured to:

perform weighting on the first weight, the second weight, the third weight, the fourth weight, and the fifth weight based on a preset ratio, to obtain a mixed weight;

perform weight processing on the mixed weight, wherein a weight used to perform the weight processing tends to decrease from a center of the current frame of image to a boundary of the current frame of image;

add an image obtained after the weight processing to weights in a weight distribution graph of the previous frame of image, and perform averaging processing; and

extract a preset quantity of weights from weights obtained after the averaging processing, and subtract the preset quantity of weights from the weights obtained after the averaging processing, to obtain the weight distribution graph of the current frame of image.

19. The device according to claim 12, wherein the first determining submodule is configured to:

determine a plurality of squares based on color values in the weight distribution graph, and select, from the plurality of squares, a square satisfying a preset condition, wherein the preset condition comprises;

a weight in the square is higher than a preset target value;

an area of the square is within a first preset range;

a length of the square in a first dimension and a length of the square in a second dimension are within a second preset range, wherein the first dimension is perpendicular to the second dimension;

or

a difference between a total weight of squares of the current frame of image and an average total weight of a preset quantity of previous images is within a third preset range.

20. The device according to claim 19, wherein the first determining submodule is configured to determine the plurality of squares based on the color values in the weight distribution graph through the following steps:

step A;

determining a point with a maximum weight in the weight distribution graph;

step B;

using the point with the maximum weight as a second center point, and expanding a square to a fourth preset range;

step C;

after the square is expanded to the fourth preset range, shrinking the square, until an average value of color values at edges of the square is less than an average value of a preset ratio and a sum of the color values of the image;

step D;

clearing color values in a range determined by the square; and

step E;

repeating step A to step D, to obtain the plurality of squares.