Adaptive overlay element placement in video

• US 6,778,224 B2
• Filed: 06/25/2001
• Issued: 08/17/2004
• Est. Priority Date: 06/25/2001
• Status: Expired due to Fees

• Current Assignee: Fox Digital, Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Dagtas, Serhan, Abdel-Mottaleb, Mohamed
• Primary Examiner(s): Lee, Michael H.
• Assistant Examiner(s): Tran, Trang U.
• Application Number:
  US09/888,804
• Publication Number:
  US 20020196370A1
• Time in Patent Office: 1,149 Days
• Field of Search: H04/N.974, 348/586, 348/588, 348/589, 348/564, 348/596, 348/565, 348/600, 348/584, 345/723, 345/641, 345/640, 345/638, 345/629
• US Class Current: 348/586
• CPC Class Codes: H04N 19/467 characterised by the embedded information being invisible, e.g. watermarking
• CPC Class Codes: H04N 21/434 Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream demultiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4884 for displaying subtitles

Abstract

A method for placing overlay elements in images of a digital video so that desirable viewing areas of the images are not occluded by the overlay element, the method includes extracting frames from the video, each of the frames defining an image which may have one or more desirable viewing areas. For each frame, an area is selected in the image for placing an overlay element and the percentage the overlay element overlaps the desirable viewing areas is determined for the selected area. The overlay element is placed in the selected area if the percentage is below a maximum allowed predetermined percentage of overlapping. If it is not, one or more other areas in the image are selected and/or the size, shape, aspect ratio of the overlay element is changed until a position and/or size, shape, aspect ratio is found which minimizes occlusion of the desirable viewing areas in the image. Further, a device, such as a video receiver or video transmitter, for placing an overlay element in images of a digital video so that desirable viewing areas of the images are not occluded by the overlay element. In such a device, a frame extractor extracts frames from the video, each of the frames defining an image having at least one desirable viewing area. An image processor analyzes the images of the frames to identify desirable viewing areas in the images of the frames. An overlay element locator places the overlay element in the images of the frames in a manner which minimizes the occlusion of the desirable viewing areas.

Claims

1. A method for placing an overlay element in images of a digital video so that desirable viewing areas of the images are not occluded by the overlay element, the method comprising the steps of:

a) extracting a frame from the video, the frame defining an image having at least one desirable viewing area;

b) selecting an area in the image for placing an overlay element;

c) determining for the selected area what percentage the overlay element overlaps the at least one desirable viewing area;

d) selecting another area in the image for placing the overlay element if the percentage is above the maximum allowed predetermined percentage of overlapping and continuing on from step c).

2. The method according to claim 1, further comprising the step of placing the overlay element in the selected area if the percentage is below a maximum allowed predetermined percentage of overlapping.

3. The method according to claim 2, further comprising the step of extracting another frame from the video defining an image having at least one desirable viewing area and continuing on from step b).

4. The method according to claim 1, wherein the at least one desirable viewing area includes a face area.

5. The method according to claim 1, wherein the at least one desirable viewing area includes a text area.

6. The method according to claim 1, wherein the at least one desirable viewing area includes high motion activity areas.

7. The method according to claim 1, wherein the overlay element is selected from the group consisting of closed captioning, on screen commands, channel identification information, captions for displayed persons, scoreboards, stock and news tickers, logos and other images not originally in the video.

8. The method according to claim 1, wherein between steps b) and c) further comprising the step of obtaining location data about the at least one desirable viewing area in the image of the frame and using the data in step c) for determining the percentage the overlay element overlaps the at least one desirable viewing area.

9. A device for placing an overlay element in images of a digital video so that desirable viewing areas of the images are not occluded by the overlay element, the device comprising:

a frame extractor for extracting frames from the video, each of the frames defining an image having at least one desirable viewing area;

an image processor for analyzing the images of the frames to identify desirable viewing areas in the images of the frames; and

an overlay element locator for selecting an area in the image for placing the overlay and if the area occludes a threshold amount of desirable viewing areas then selecting another area of the image which minimizes the occlusion of the desirable viewing areas.

10. The device according to claim 9, wherein the at least one desirable viewing area includes a face area.

11. The device according to claim 9, wherein the at least one desirable viewing area includes a text area.

12. The device according to claim 9, wherein the at least one desirable viewing area includes high motion activity areas.

13. The device according to claim 9, wherein the image processor measures motion activity in different areas of the frame.

14. The device according to claim 9, wherein the overlay element is selected from the group consisting of closed captioning, on screen commands, channel identification information, captions for displayed persons, scoreboards, stock and news tickers, logos and other images not originally in the video.

15. The device according to claim 9, wherein the locator includes an area selector for selecting an area in the image of each of the frames for placing an overlay element and an occlusion calculator for determining for the selected area what percentage the overlay element overlaps the at least one desirable viewing area, the locator placing the overlay element in the selected area of the image if the percentage is below a maximum allowed predetermined percentage of overlapping, the locator selecting another area in the image for placing the overlay element if the percentage is above the maximum allowed predetermined percentage of overlapping.

16. The device according to claim 15, wherein the occlusion calculator determines for the selected area what percentage the overlay element overlaps the at least one desirable viewing area using the equation:

17. A receiver device for placing an overlay element in images of a digital video so that desirable viewing areas of the images are not occluded by the overlay element, the receiver device comprising:

a signal isolator for isolating a video signal having a video portion and an overlay element portion from a received video signal transmission;

a frame extractor for extracting frames from the video portion of the isolated video signal, each of the frames defining an image having at least one desirable viewing area;

an image processor for analyzing the images of the frames to identify desirable viewing areas in the images of the frames; and

an overlay element locator for selecting an area in the image for placing the overlay and if the area occludes a threshold amount of desirable viewing areas then selecting another area of the image which minimizes the occlusion of the desirable viewing areas.

18. The receiver device according to claim 17, wherein the at least one desirable viewing area includes a face area.

19. The receiver device according to claim 17, wherein the at least one desirable viewing area includes a text area.

20. The receiver device according to claim 17, wherein the at least one desirable viewing area includes high motion activity areas.

21. The receiver device according to claim 17, wherein the image processor measures motion activity in different areas of the frame.

22. The receiver device according to claim 17, wherein the overlay element is selected from the group consisting of closed captioning, on screen commands, channel identification information, captions for displayed persons, scoreboards, stock and news tickers, logos and other images not originally in the video.

23. The receiver device according to claim 17, wherein the locator includes an area selector for selecting an area in the image of each of the frames for placing an overlay element and an occlusion calculator for determining for the selected area what percentage the overlay element overlaps the at least one desirable viewing area, the locator placing the overlay element in the selected area of the image if the percentage is below a maximum allowed predetermined percentage of overlapping, the locator selecting another area in the image for placing the overlay element if the percentage is above the maximum allowed predetermined percentage of overlapping.

24. The receiver device according to claim 23, wherein the occlusion calculator determines for the selected area what percentage the overlay element overlaps the at least one desirable viewing area using the equation:

25. The receiver device according to claim 17, wherein the receiver device comprises one of a television receiver and set-top box.

26. A transmitter device for placing an overlay element in images of a digital video so that desirable viewing areas of the images are not occluded by the overlay element, the transmitter device comprising:

a video generator for generating a digital video;

a frame extractor for extracting frames from the video, each of the frames defining an image having at least one desirable viewing area;

an image processor for analyzing the images of the frames to identify desirable viewing areas in the images of the frames;

an overlay element generator for generating an overlay element;

an overlay element locator for selecting an area in the image for placing the overlay and if the area occludes a threshold amount of desirable viewing areas then selecting another area of the image which minimizes the occlusion of the desirable viewing areas; and

a transmitter for transmitting a video signal of the video with the overlay element positioning information.

27. The transmitter device according to claim 26, wherein the at least one desirable viewing area includes a face area.

28. The transmitter device according to claim 26, wherein the at least one desirable viewing area includes a text area.

29. The transmitter device according to claim 26, wherein the at least one desirable viewing area includes high motion activity areas.

30. The transmitter device according to claim 26, wherein the image processor measures motion activity in different areas of the frame.

31. The transmitter device according to claim 26, wherein the overlay element is selected from the group consisting of closed captioning, on screen commands, channel identification information, captions for displayed persons, scoreboards, stock and news tickers, logos and other images not originally in the video.

32. The transmitter device according to claim 26, wherein the locator includes an area selector for selecting an area in the image of each of the frames for placing an overlay element and an occlusion calculator for determining for the selected area what percentage the overlay element overlaps the at least one desirable viewing area, the locator placing the overlay element in the selected area of the image if the percentage is below a maximum allowed predetermined percentage of overlapping, the locator selecting another area in the image for placing the overlay element if the percentage is above the maximum allowed predetermined percentage of overlapping.

33. The transmitter device according to claim 32, wherein the occlusion calculator determines for the selected area what percentage the overlay element overlaps the at least one desirable viewing area using the equation:

34. The transmitter device according to claim 26, wherein the transmitter device comprises a television broadcast transmitter.