Object-centric video redaction

• US 10,275,608 B2
• Filed: 10/15/2016
• Issued: 04/30/2019
• Est. Priority Date: 10/15/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bobbitt, Russell P, Brobst, Curtis H, Feris, Rogerio S, Zhai, Yun
• Primary Examiner(s): King, John B
• Application Number:
  US15/294,694
• Publication Number:
  US 20180107830A1
• Time in Patent Office: 927 Days
• CPC Class Codes: G06F 18/41 Interactive pattern learning with a human teacher
• CPC Class Codes: G06F 21/6209 to a single file or object, e.g. in a secure envelope, encrypted and accessed using a key, or with access control rules appended to the object itself
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 10/7557 based on appearance, e.g. active appearance models [AAM]
• CPC Class Codes: G06V 10/7788 the supervisor being a human, e.g. interactive learning with a human teacher
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D

Abstract

Techniques are described for video redaction. In one aspect, techniques include receiving a video for redaction; analyzing the video to generate an appearance model for the video, providing a user interface (UI) allowing a user to modify the appearance model, and responsive to a user selecting an object from the appearance model, extending and completing a trajectory of the selected object with enhanced marking based on the user input.

Claims

1. A computer-implemented method for video redaction, the method comprising:

receiving a video for redaction having a plurality of frames;

analyzing the video to generate an appearance model for the video;

providing a user interface (UI) allowing a user to modify the appearance model with a plurality of user-placed marks;

receiving the plurality of user-placed marks corresponding with a plurality of sample frames selected from the plurality of frames;

interpolating a bounding box on the plurality of sample frames based on the received plurality of user-placed marks;

responsive to a user selecting an object from the appearance model based on the interpolated bounding box, extending and completing a trajectory of the selected object within a plurality of remainder frames from the plurality of frames, and placing automatically generated marks to enclose the selected object within each remainder frame of the plurality of remainder frames based on the received plurality of user-placed marks and the extended and completed trajectory of the selected object; and

redacting the plurality of frames of the video based on the modified appearance model.

2. The method of claim 1, further comprising:

designating weightings for trajectory observations in the appearance model.

3. The method of claim 2, further comprising:

designating a higher weighting for the trajectory observations of the selected object than the weighting for observations that have not been selected.

4. The method of claim 1, further comprising:

receiving user input selecting a boundary for identify an object in the appearance model.

5. The method of claim 1, further comprising:

receiving further user input and making further modifications to the appearance model iteratively.

6. The method of claim 1, wherein the video is a video used for law enforcement.

7. A computer system comprising:

one or more processors, one or more computer-readable memories, and one or more computer-readable, tangible storage devices;

program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories to perform operations comprising;

receiving a video for redaction having a plurality of frames;

analyzing the video to generate an appearance model for the video;

providing a user interface (UI) allowing a user to modify the appearance model with a plurality of user-placed marks;

receiving the plurality of user-placed marks corresponding with a plurality of sample frames selected from the plurality of frames;

interpolating a bounding box on the plurality of sample frames based on the received plurality of user-placed marks;

responsive to a user selecting an object from the appearance model based on the interpolated bounding box, extending and completing a trajectory of the selected object within a plurality of remainder frames from the plurality of frames, and placing automatically generated marks to enclose the selected object within each remainder frame of the plurality of remainder frames based on the received plurality of user-placed marks and the extended and completed trajectory of the selected object; and

redacting the plurality of frames of the video based on the modified appearance model.

8. The system of claim 7, wherein the operations further comprise:

designating weightings for trajectory observations in the appearance model.

9. The system of claim 8, wherein the operations further comprise:

designating a higher weighting for the trajectory observations of the selected object than the weighting for observations that have not been selected.

10. The system of claim 7, wherein the operations further comprise:

receiving user input selecting a boundary for identify an object in the appearance model.

11. The system of claim 7, wherein the operations further comprise:

receiving further user input and making further modifications to the appearance model iteratively.

12. A computer program product comprising a non-transitory computer-readable storage medium having program code embodied therewith, the program code executable by a computing device to perform operations comprising:

receiving a video for redaction having a plurality of frames;

analyzing the video to generate an appearance model for the video;

providing a user interface (UI) allowing a user to modify the appearance model with a plurality of user-placed marks;

receiving the plurality of user-placed marks corresponding with a plurality of sample frames selected from the plurality of frames;

interpolating a bounding box on the plurality of sample frames based on the received plurality of user-placed marks;

responsive to a user selecting an object from the appearance model based on the interpolated bounding box, extending and completing a trajectory of the selected object within a plurality of remainder frames from the plurality of frames, and placing automatically generated marks to enclose the selected object within each remainder frame of the plurality of remainder frames based on the received plurality of user-placed marks and the extended and completed trajectory of the selected object; and

redacting the plurality of frames of the video based on the modified appearance model.

13. The program product of claim 12, wherein the operations further comprise:

designating weightings for trajectory observations in the appearance model.

14. The program product of claim 13, wherein the operations further comprise:

designating a higher weighting for the trajectory observations of the selected object than the weighting for observations that have not been selected.

15. The program product of claim 12, wherein the operations further comprise:

receiving user input selecting a boundary for identify an object in the appearance model.

16. The program product of claim 12, wherein the operations further comprise:

receiving further user input and making further modifications to the appearance model iteratively.

17. The program product of claim 12, wherein the video is a video used for law enforcement.