METHOD, APPARATUS, AND SYSTEM FOR IDENTIFYING OBJECTS IN VIDEO IMAGES AND DISPLAYING INFORMATION OF SAME

• US 20170255830A1
• Filed: 08/11/2015
• Published: 09/07/2017
• Est. Priority Date: 08/27/2014
• Status: Active Grant

• Current Assignee: Alibaba Group Holding Ltd.
• Original Assignee: Alibaba Group Holding Ltd.
• Inventors: CHEN, Xiao
• Granted Patent: US 10,395,120 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06Q 30/0277 Online advertisement
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/48 by mapping characteristic values of the pattern into a parameter space, e.g. Hough transformation
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G11B 27/036 Insert-editing
• CPC Class Codes: G11B 27/102 Programmed access in sequence to addressed parts of tracks of operating record carriers access by moving the head G11B3/08, G11B5/54, G11B7/085, G11B21/022; by moving the record carrier G11B15/005, G11B17/005, by driving of both record carrier and head G11B15/1816

Abstract

The disclosure discloses a method and apparatus for identifying objects in paused video images, a method and apparatus for displaying object information on paused video images, and a system for displaying object information on paused video images. The method of identifying objects in paused video images comprises: receiving, from a client, information associated with the paused video images; obtaining, based on the received information associated with the paused video images, the at least one static image corresponding to a paused video of the client; identifying the objects from the at least one static image and obtaining object information; and sending, to the client, identified position information and the object information associated with the objects in the at least one static image. With the solutions provided in the disclosure and by combining client ends and servers, the disclosure achieves the functions of automatically identifying object in paused video images and displaying object information, thereby effectively improving the efficiency and accuracy of advertising in a video play process.

Claims

1. A method comprising:

receiving, from a client, information associated with a paused video;

obtaining, based on the information associated with the paused video, at least one static image corresponding to the paused video;

identifying an object present in the at least one static image and obtaining object information associated with the object; and

sending, to the client, the object information and an identified position of the object within the at least one static image.

2. The method of claim 1, wherein obtaining the at least one static image corresponding to the paused video comprises one of:

extracting the at least one static image from the received information;

or retrieving the at least one static image from a video server based on a pause time point included in the received information.

4. The method of claim 2, wherein the at least one static image includes a plurality of frames and wherein the method further comprises:

calculating a clarity value for each frame by identifying image edge points or calculating an image sharpness value for each frame; and

selecting, as the at least one static image, a frame from the plurality of frames that has a clarity value meeting a predetermined threshold.

5. The method of claim 1, wherein prior to identifying the object from the at least one static image and obtaining object information, the method further comprises:

obtaining, from the at least one static image, an area including an image of an object, and using the area as an object image area; and

identifying the object present in the object image area and obtaining the object information associated with the object.

6. The method of claim 5, wherein obtaining, from the at least one static image, an area including an image of an object comprises:

partitioning the at least one static image into multiple image areas using an image segmentation method;

obtaining, for each image area, a feature value used to describe characteristics of the image area; and

determining, for each image area, whether the feature value of the image area meets a preset feature of the object image area, and if so, identifying the image area as an object image area, and a position of the image area as a position of an object shown in the image area.

7. The method of claim 6, wherein the image segmentation method comprises:

an area-based segmentation method or an edge-based segmentation method.

8. The method of claims 5, wherein identifying the object present in the object image area and obtaining the object information associated with the object comprises:

extracting an image feature from the object image area, and using the image feature as a reference feature;

searching, in a pre-generated object candidate feature library, for an object candidate feature matching with the reference feature, and obtaining an object index corresponding to the matching object candidate feature; and

obtaining, based on the object index, information associated with the identified object.

9. The method of claim 8, wherein the image feature comprises at least one of the following features:

a color feature, a shape feature, a texture feature, or a semantic feature.

10. The method of claim 8, wherein extracting an image feature from the object image area is performed using a scale-invariant feature transform.

11. The method of claim 8, wherein searching, in a pre-generated object candidate feature library, for an object candidate feature matching with the reference feature comprises:

calculating a feature value difference between the reference feature and each object candidate feature in the object candidate feature library; and

using an object candidate feature with a feature value difference meeting a set requirement as the object candidate feature matching the reference feature.

22. The method of claim 1 wherein receiving, from a client, information associated with a paused video comprises receiving a pause instruction for a video currently being played at a client device and pausing the video;

and wherein the method further comprises displaying, based on the position information, the object information at a corresponding position in the paused video.

3. (canceled)

12-21. -21. (canceled)

23. (canceled)

24. The method of claim 21, wherein the object information includes object link information providing address information for accessing an object details page or a page for purchasing the object associated with the object information, and wherein the method further comprises:

adding a ‘

click to redirect’

area corresponding to the object link at the corresponding position;

receiving a click or touch operation performed on the paused video image; and

determining whether an area receiving the click or touch operation is the ‘

click to redirect’

area, and

if so, obtaining and displaying a corresponding page based on the object link information corresponding to the ‘

click to redirect’

area, and

if not, clearing the displayed object information, removing the ‘

click to redirect’

area, and continuing to play the paused video.

24-30. -30. (canceled)

31. An apparatus comprising:

one or processors; and

a non-transitory memory storing computer-executable instructions therein that, when executed by the processors, cause the apparatus to perform the operations of;

receiving, from a client, information associated with a paused video;

obtaining, based on the information associated with the paused video, at least one static image corresponding to the paused video;

identifying an object present in the at least one static image and obtaining object information associated with the object; and

sending, to the client, the object information and an identified position of the object within the at least one static image.

32. The apparatus of claim 31, wherein the operation of obtaining the at least one static image corresponding to the paused video comprises one of:

extracting the at least one static image from the received information;

or retrieving the at least one static image from a video server based on a pause time point included in the received information.

33. The apparatus of claim 32, wherein the at least one static image includes a plurality of frames and wherein the operations further include:

calculating a clarity value for each frame by identifying image edge points or calculating an image sharpness value for each frame; and

selecting, as the at least one static image, a frame from the plurality of frames that has a clarity value meeting a predetermined threshold.

34. The apparatus of claim 31, wherein prior to the operation of identifying the object from the at least one static image and obtaining object information, the operations further include:

obtaining, from the at least one static image, an area including an image of an object, and using the area as an object image area; and

identifying the object present in the object image area and obtaining the object information associated with the object.

35. The apparatus of claim 34, wherein the operation of obtaining, from the at least one static image, an area including an image of an object includes the operations of:

partitioning the at least one static image into multiple image areas using an image segmentation method;

obtaining, for each image area, a feature value used to describe characteristics of the image area; and

determining, for each image area, whether the feature value of the image area meets a preset feature of the object image area, and if so, identifying the image area as an object image area, and a position of the image area as a position of an object shown in the image area.

36. The apparatus of claim 35, wherein the image segmentation method comprises:

an area-based segmentation method or an edge-based segmentation method.

37. The apparatus of claim 34, wherein the operation of identifying the object present in the object image area and obtaining the object information associated with the object item includes the operations of:

extracting an image feature from the object image area, and using the image feature as a reference feature;

searching, in a pre-generated object candidate feature library, for an object candidate feature matching with the reference feature, and obtaining an object index corresponding to the matching object candidate feature; and

obtaining, based on the object index, information associated with the identified object.

38. The apparatus of claim 37, wherein the image feature comprises at least one of the following features:

a color feature, a shape feature, a texture feature, or a semantic feature.

39. The apparatus of claim 37, wherein the operation of extracting an image feature from the object image area is performed using a scale-invariant feature transform.

40. The apparatus of claim 37, wherein the operation of searching, in a pre-generated object candidate feature library, for a object candidate feature matching with the reference feature includes the operations of:

calculating a feature value difference between the reference feature and each object candidate feature in the object candidate feature library; and

using an object candidate feature with a feature value difference meeting a set requirement as the object candidate feature matching the reference feature.