METHOD AND APPARATUS FOR DISPLAYING BUSINESS OBJECT IN VIDEO IMAGE AND ELECTRONIC DEVICE

• US 20180108165A1
• Filed: 12/19/2017
• Published: 04/19/2018
• Est. Priority Date: 08/19/2016
• Status: Active Grant

• Current Assignee: Beijing SenseTime Technology Development Co., Ltd. (SenseTime group Limited)
• Original Assignee: Beijing SenseTime Technology Development Co., Ltd. (SenseTime group Limited)
• Inventors: SHI, Jianping, LUAN, Qing, XU, Qinqin, WANG, Lei
• Granted Patent: US 11,037,348 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06Q 30/0251 Targeted advertisements
• CPC Class Codes: G06Q 30/0277 Online advertisement
• CPC Class Codes: G06T 11/20 Drawing from basic elements, e.g. lines or circles
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/225 based on a marking or identifier characterising the area
• CPC Class Codes: G06V 10/23 based on positionally close patterns or neighbourhood relationships
• CPC Class Codes: G06V 2201/07 Target detection

Abstract

Embodiments of the present disclosure provide a method and an apparatus for displaying a business object in a video image and an electronic device. The method for displaying a business object in a video image includes: detecting at least one target object from a video image, and determining a feature point of the at least one target object; determining a display position of a to-be-displayed business object in the video image according to the feature point of the at least one target object; and drawing the business object at the display position by using computer graphics. According to the embodiments of the present disclosure, the method and apparatus are conductive to saving network resources and system resources of a client.

Claims

1. A method for displaying a business object in a video image, comprising:

detecting at least one target object from a video image, and determining a feature point of the at least one target object;

determining a display position of a to-be-displayed business object in the video image according to the feature point of the at least one target object; and

drawing the business object at the display position by using computer graphics.

2. The method according to claim 1, wherein the determining a display position of a to-be-displayed business object in the video image according to the feature point of the at least one target object comprises:

determining the display position of the to-be-displayed business object in the video image according to the feature point of the at least one target object by using a convolutional network model, the convolutional network model being pre-trained and used to determine a display position of a business object in a video image.

3. The method according to claim 2, wherein pre-training of the convolutional network model comprises:

acquiring a feature vector of a business object sample image to be trained by using the convolutional network model, wherein the feature vector includes information of the target object and position information and/or confidence information of the business object in the business object sample image to be trained;

performing convolution processing on the feature vector by using the convolutional network model to acquire a convolution result of the feature vector;

determining respectively whether corresponding information of the target object and corresponding position information and/or confidence information of the business object in the convolution result of the feature vector meet a convergence condition;

completing training of the convolutional network model if the convergence condition is met; and

if the convergence condition is not met, adjusting a parameter of the convolutional network model according to the convolution result of the feature vector if the convergence condition is not met, and performing an iterative training of the convolutional network model according to the adjusted parameter of the convolutional network model until the convolution result of the feature vector after the iterative training meets the convergence condition.

4. The method according to claim 3, wherein the determining respectively whether corresponding position information and/or confidence information of the business object in the convolution result of the feature vector meet a convergence condition comprises:

acquiring the corresponding position information of the business object in the convolution result of the feature vector;

calculating a first distance between a position indicated by the corresponding position information of the business object and a preset standard position using a first loss function; and

determining whether the corresponding position information of the business object meets the convergence condition according to the first distance; and

/or

acquiring the corresponding confidence information of the business object in the convolution result of the feature vector;

calculating a second distance between a confidence indicated by the corresponding confidence information of the business object and a preset standard confidence using a second loss function; and

determining whether the corresponding confidence information of the business object meets the convergence condition according to the second distance.

5. The method according to claim 1, wherein the determining a display position of a to-be-displayed business object in the video image according to the feature point of the at least one target object comprises:

determining a type of the at least one target object according to the feature point of the at least one target object;

determining a display area of the to-be-displayed business object according to the type of the at least one target object; and

determining the display position of the to-be-displayed business object in the video image according to the display area.

6. The method according to claim 1, wherein the determining a display position of a to-be-displayed business object in the video image according to the feature point of the at least one target object comprises:

determining the display position of the to-be-displayed business object in the video image according to the feature point of the at least one target object and a type of the to-be-displayed business object.

7. The method according to claim 6, wherein the determining the display position of the to-be-displayed business object in the video image according to the feature point of the at least one target object and a type of the to-be-displayed business object comprises:

acquiring a plurality of display positions of the to-be-displayed business object in the video image according to the feature point of the at least one target object and the type of the to-be-displayed business object; and

selecting at least one display position from the plurality of display positions as the display position of the to-be-displayed business object in the video image.

8. The method according to claim 1, wherein the determining a display position of a to-be-displayed business object in the video image according to the feature point of the at least one target object comprises:

acquiring a target display position corresponding to the feature point of the at least one target object from a pre-stored corresponding relationship between a feature point of a target object and a display position; and

determining the acquired target display position as the display position of the to-be-displayed business object in the video image.

9. The method according to claim 1, wherein the business object comprises a plurality of associated business objects;

the determining a display position of a to-be-displayed business object in the video image comprises determining corresponding display positions of a plurality of to-be-displayed associated business objects in the video image; and

the drawing the business object at the display position by using computer graphics comprises drawing the plurality of associated business objects at the corresponding display positions by using computer graphics, respectively.

10. The method according to claim 9, wherein the plurality of associated business objects comprise any one or more of:

multiple special effects containing semantic information that are used for displaying a same business object theme, multiple display portions of a same special effect containing semantic information, and multiple special effects containing semantic information that are provided by a same business object provider.

20. A computer program product, comprising a computer readable code, wherein when the computer readable code is run on a device, a processor in the device executes an instruction for realizing the method for displaying a business object in a video image according to claim 1.

11. An apparatus for displaying a business object in a video image, comprising:

a processor; and

a memory storing instructions, the instructions when executed by the processor, cause the processor to perform operations, the operations comprising;

detecting at least one target object from a video image, and determine a feature point of the at least one target object;

determining a display position of a to-be-displayed business object in the video image according to the feature point of the at least one target object; and

drawing the business object at the display position by using computer graphics.

12. The apparatus according to claim 11, wherein the determining a display position of a to-be-displayed business object in the video image according to the feature point of the at least one target object is implemented by a pre-trained convolutional network model.

13. The apparatus according to claim 12, wherein the operations further comprises pre-training the convolutional network model,

wherein the pre-training the convolutional network model comprises:

acquiring a feature vector of a business object sample image to be trained, wherein the feature vector includes information of the target object and position information and/or confidence information of the business object in the business object sample image to be trained;

performing convolution processing on the feature vector to acquire a convolution result of the feature vector;

determining whether corresponding information of the target object and corresponding position information and/or confidence information of the business object in the convolution result of the feature vector meet a convergence condition, respectively; and

completing training of the convolutional network model if the determining result of the determining module is that the convergence condition is met; and

adjusting a parameter of the convolutional network model according to the convolution result of the feature vector if the determining result of the determining module is that the convergence condition is not met, so that the training module performs an iterative training of the convolutional network model according to the adjusted parameter of the convolutional network model until the convolution result of the feature vector after the iterative training meets the convergence condition.

14. The apparatus according to claim 13, wherein the determining whether corresponding information of the target object and corresponding position information and/or confidence information of the business object in the convolution result of the feature vector meet a convergence condition, respectively, comprises:

acquiring the corresponding position information of the business object in the convolution result of the feature vector;

calculating a first distance between a position indicated by the corresponding position information of the business object and a preset standard position using a first loss function; and

determining whether the corresponding position information of the business object meets the convergence condition according to the first distance; and

/or

acquiring the corresponding confidence information of the business object in the convolution result of the feature vector;

calculating a second distance between a confidence indicated by the corresponding confidence information of the business object and a preset standard confidence using a second loss function; and

determining whether the corresponding confidence information of the business object meets the convergence condition according to the second distance.

15. The apparatus according to claim 11, wherein the determining a display position of a to-be-displayed business object in the video image according to the feature point of the at least one target object comprises:

determining a type of the at least one target object according to information of the feature point of the at least one target object;

determining a display area of the to-be-displayed business object according to the type of the at least one target object; and

determining the display position of the to-be-displayed business object in the video image according to the display area.

16. The apparatus according to claim 11, wherein the determining a display position of a to-be-displayed business object in the video image according to the feature point of the at least one target object comprises determining the display position of the to-be-displayed business object in the video image according to the feature point of the at least one target object and a type of the to-be-displayed business object.

17. The apparatus according to claim 16, wherein the determining a display position of a to-be-displayed business object in the video image according to the feature point of the at least one target object comprises acquiring a plurality of display positions of the to-be-displayed business object in the video image, according to the feature point of the at least one target object and the type of the to-be-displayed business object;

and selecting at least one display position from the plurality of display positions as the display position of the to-be-displayed business object in the video image.

18. The apparatus according to claim 11, wherein the determining a display position of a to-be-displayed business object in the video image according to the feature point of the at least one target object comprises acquiring a target display position corresponding to the feature point of the at least one target object from a pre-stored corresponding relationship between a feature point of a target object and a display position;

and determining the acquired target display position as the display position of the to-be-displayed business object in the video image.

19. The apparatus according to claim 11, wherein the business object comprises a plurality of associated business objects;

the determining a display position of a to-be-displayed business object in the video image according to the feature point of the at least one target object comprises;

determining corresponding display positions of a plurality of to-be-displayed associated business objects in the video image according to the feature point of the at least one target object; and

the drawing the business object at the display position by using computer graphics comprises;

drawing the plurality of associated business objects at the corresponding display positions by using computer graphics, respectively.