Post production visual enhancement rendering

• US 20040031062A1
• Filed: 03/27/2003
• Published: 02/12/2004
• Est. Priority Date: 08/02/2001
• Status: Active Grant

• Current Assignee: OpenTV, Inc. (Kudelski SA)
• Original Assignee: Intellocity USA, Inc. (Kudelski SA)
• Inventors: Lemmons, Thomas
• Granted Patent: US 7,444,656 B2
• Time in Patent Office: Days
• US Class Current: 725/136
• CPC Class Codes: H04N 21/23412 for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/25808 Management of client data terminal profiles in network data switching protocols H04L67/303
• CPC Class Codes: H04N 21/25883 being end-user demographical data, e.g. age, family status or address arrangements for identifying locations of users in broadcast systems H04H60/52
• CPC Class Codes: H04N 21/4312 involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
• CPC Class Codes: H04N 21/4314 for fitting data in a restricted space on the screen, e.g. EPG data in a rectangular grid
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/44012 involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
• CPC Class Codes: H04N 21/4622 Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859
• CPC Class Codes: H04N 21/47205 for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally
• CPC Class Codes: H04N 21/47815 Electronic shopping payment schemes, payment architectures or payment protocols for electronic shopping systems G06Q20/12
• CPC Class Codes: H04N 21/4782 Web browsing , e.g. WebTV
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/854 Content authoring
• CPC Class Codes: H04N 21/8543 using a description language, e.g. Multimedia and Hypermedia information coding Expert Group [MHEG], eXtensible Markup Language [XML] information retrieval of semistructured data, the underlying structure being taken into account, e.g. mark-up language structure data G06F16/80
• CPC Class Codes: H04N 21/858 Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot
• CPC Class Codes: H04N 21/8586 by using a URL processing chained hypermedia data for information retrieval G06F16/94; information retrieval from the Internet by using URLs G06F16/955; URL in broadcast information H04H20/93; Web-based protocols H04L67/02
• CPC Class Codes: H04N 7/17318 Direct or substantially direct transmission and handling of requests

Abstract

Disclosed is a method and system that renders enhancement images onto receptor sites of a scene of a pre-existing program. A receptor site defines an area of a scene onto which an enhancement image may be rendered and provides information that may include size, scale, rotation, perspective, contour and coloration. Rendering processes may be direct, planar, conformal, blended, or migrated and may produce photo-realistic images that appear to be a natural part of a scene. Selection of enhancements may be targeted or opportunistic and may employ secondary selection criteria including demographics, viewer preferences, pricing and other information. Receptor site identification may employ human or software resources, or a combination thereof. Receptor site information may be determined through human or software resources or a combination thereof and may include the acquisition of characteristics of a displayed object image that then may be applied to an enhancement image.

Claims

1. A method of rendering an enhancement image onto a scene of pre-existing program content comprising:

detecting a receptor site for said enhancement;

determining receptor site characteristics including type, size, and position of said receptor site;

selecting said enhancement image from a plurality of enhancement images using said receptor site characteristics;

processing said enhancement image employing a process that corresponds to said receptor site type and that produces a processed enhancement image; and

rendering said processed enhancement image at said position of said receptor site.

2. The method of claim 1 wherein said step of detecting a receptor site further comprises:

receiving an input from an operator viewing said scene.

3. The method of claim 1 wherein said step of detecting a receptor site further comprises:

applying an image recognition software program to said scene.

4. The method of claim 1 wherein said step of processing said enhancement image further comprises:

scaling said enhancement image.

5. The method of claim 1 wherein said step of processing said enhancement image further comprises:

rotating said enhancement image.

6. The method of claim 1 wherein said step of processing said enhancement image further comprises:

warping said enhancement image.

7. The method of claim 1 wherein said step of processing said enhancement image further comprises:

warping said enhancement image to conform to the contour of a recognized object.

8. The method of claim 1 wherein said step of determining receptor site characteristics comprises:

determining coloration characteristics of said receptor site.

9. The method of claim 8 wherein said step of processing said enhancement image further comprises:

applying said coloration characteristics to said enhancement image.

10. The method of claim 1 wherein said step of selecting said enhancement image further comprises:

applying a secondary selection process.

11. A method of rendering an enhancement image onto pre-existing content comprising:

accessing a first database containing a plurality of enhancement images and information associated with said enhancement images including types of said enhancement images;

selecting said enhancement image;

accessing a second database containing a plurality of receptor sites and information associated with said receptor sites;

selecting one receptor site of said plurality of receptor sites using said type of said enhancement image;

processing said enhancement image responsive to said receptor site information to produce a processed enhancement image; and

rendering said processed enhancement image at said receptor site.

12. The method of claim 11 wherein said step of processing said enhancement image further comprises:

scaling said enhancement image.

13. The method of claim 11 wherein said step of processing said enhancement image further comprises:

rotating said enhancement image.

14. The method of claim 11 wherein said step of processing said enhancement image further comprises:

warping said enhancement image.

15. The method of claim 11 wherein said step of processing said enhancement image further comprises:

warping said enhancement image to conform to the contour of a recognized object.

16. The method of claim 11 wherein said step of processing said enhancement image further comprises:

applying coloration characteristics of said receptor site to said enhancement image.

17. The method of claim 11 wherein said step of selecting said enhancement image further comprises:

applying a secondary selection process.

18. A method of rendering an enhancement image onto pre-existing content comprising:

accessing a first database containing a plurality of receptor sites and information associated with said receptor sites including receptor site type;

selecting one receptor site of said plurality of receptor sites;

accessing a second database containing a plurality of enhancement images and information associated with said enhancement images including types of said enhancement images;

selecting said enhancement image using said receptor site type of said one receptor site;

processing said enhancement image responsive to said receptor site information of said one receptor site to produce a processed enhancement image; and

rendering said processed enhancement image at said one receptor site.

19. The method of claim 18 wherein said step of processing said enhancement image further comprises:

scaling said enhancement image.

20. The method of claim 18 wherein said step of processing said enhancement image further comprises:

rotating said enhancement image.

21. The method of claim 18 wherein said step of processing said enhancement image further comprises:

warping said enhancement image.

22. The method of claim 18 wherein said step of processing said enhancement image further comprises:

warping said enhancement image to conform to the contour of a recognized object.

23. The method of claim 18 wherein said step of processing said enhancement image further comprises:

applying coloration characteristics of said one receptor site to said enhancement image.

24. The method of claim 18 wherein said step of selecting said enhancement image further comprises:

applying a secondary selection process.

25. A method of identifying a receptor site in a scene of a program at which an enhancement image may be rendered comprising:

accessing said scene of said program;

recognizing an object in said scene;

.

defining a receptor site in said scene based upon recognition of said object; and

assigning a type to said receptor site.

26. The method of claim 25 wherein said step of recognizing an object further comprises:

receiving an operator input.

27. The method of claim 25 wherein said step of recognizing an object further comprises:

applying a pattern recognition process to said scene.

28. The method of claim 25 further comprising:

determining receptor site dimensions.

29. The method of claim 25 further comprising:

determining receptor site perspective.

30. The method of claim 25 further comprising:

determining receptor site contour.

31. The method of claim 25 further comprising:

determining receptor site illumination.

32. The method of claim 25 further comprising:

comparing the image of said recognized object with a model of said object and determining scale and illumination characteristics of said object.

33. A system for rendering an enhancement image onto a scene of pre-existing program content comprising:

first program code that detects a receptor site for said enhancement;

second program code that determines receptor site characteristics including type, size, and position of said receptor site;

third program code that selects said enhancement image from a plurality of enhancement images using said receptor site characteristics;

fourth program code that processes said enhancement image using said receptor site characteristics to produce a processed enhancement image; and

rendering program code that renders said processed enhancement image at said position of said receptor site.

34. The system of claim 33 wherein said first program code further comprises:

a routine that receives an input from an operator viewing said scene.

35. The system of claim 33 wherein said first program code further comprises:

a pattern recognition routine.

36. The system of claim 33 wherein said fourth program code further comprises:

a routine that scales said enhancement image.

37. The system of claim 33 wherein said fourth program code further comprises:

a routine that rotates said enhancement image.

38. The system of claim 33 wherein said fourth program code further comprises:

a routine that warps said enhancement image.

39. The system of claim 33 wherein said fourth program code further comprises:

a routine that warps said enhancement image to conform to the contour of a recognized object.

40. The system of claim 33 wherein said second software program further comprises:

a routine that determines coloration characteristics of said receptor site.

41. The system of claim 40 wherein said fourth program code further comprises:

a routine that applies said coloration characteristics to said enhancement image.

42. The system of claim 33 wherein said third software program that selects said enhancement image further comprises:

a secondary selection process.

43. The system of claim 42 wherein said secondary selection process further comprises:

a routine that employs viewer demographic information in selecting said enhancement image.

44. The system of claim 42 wherein said secondary selection process further comprises:

a routine that employs viewer preference information in selecting said enhancement image.

45. The system of claim 42 wherein said secondary selection process further comprises:

a routine that employs price information in selecting said enhancement image.

46. A system for rendering an enhancement image onto a scene of pre-existing program content comprising:

first access means that access a program scene;

second access means that access receptor site information for said program scene;

selection means that select an enhancement image;

processing means that process said enhancement image using said receptor site information to produce a processed enhancement image; and

rendering means that render said processed enhancement image at said receptor site.

47. An interactive television system programmed to perform the following steps:

receive a program scene;

access receptor site information for said program scene;

select an enhancement image based upon said receptor site information;

process said enhancement image if indicated by said receptor site information; and

render said enhancement image.

48. The system of claim 47 wherein said step to select an enhancement based upon said receptor site information further comprises the following step:

select said enhancement image using a secondary selection process.