VIDEO SUPERPOSITION SYSTEM AND METHOD

• US 20020018070A1
• Filed: 09/18/1996
• Published: 02/14/2002
• Est. Priority Date: 09/18/1996
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Neven Vision, Inc. (Alphabet Inc.)
• Inventors: LANIER, JARON
• Granted Patent: US 6,400,374 B2
• Time in Patent Office: Days
• US Class Current: 345/629
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06T 15/10 Geometric effects
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G09G 2340/125 wherein one of the images is motion video
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q

Abstract

A graphic image system comprising a video camera producing a first video signal defining a first image including a foreground object and a background, the foreground object preferably including an image of a human subject having a head with a face; an image position estimating system for identifying a position with respect to said foreground object, e.g., the head, the foreground object having features in constant physical relation to the position; and a computer, responsive to the position estimating system, for defining a mask region separating the foreground object from said background. The computer generates a second video signal including a portion corresponding to the mask region, responsive to said position estimating system, which preferably includes a character having a mask outline. In one embodiment, the mask region of the second video signal is keyed so that the foreground object of the first video signal shows through, with the second video signal having portions which interact with the foreground object In another embodiment, means, responsive to the position estimating system, for dynamically defining an estimated boundary of the face and for merging the face, as limited by the estimated boundary, within the mask outline of the character. Video and still imaging devices may be flexibly placed in uncontrolled environments, such as in a kiosk in a retail store, with an actual facial image within the uncontrolled environment placed within a computer generated virtual world replacing the existing background and any non-participants.

Claims

1. A graphic image system comprising:

a source of a first signal representing a first image including a moving human subject having a head with a face;

an image position estimating system for estimating the instantaneous position of said head of said human subject;

a source of a second signal representing a second image including a character having a head with a mask outline; and

means, responsive to said position estimating system and to said first and second signals, for dynamically defining an estimated boundary of said face of said human subject in said first image and for merging the face in said first image, as limited by said estimated boundary, with the second image within the mask outline.

2. The graphic image system according to claim 1, said means comprising a video superposition control for superposing said face within said estimated boundary.

3. The graphic image system according to claim 2, wherein said background video image is responsive to said image position estimating system.

4. The graphic image system according to claim 2, wherein said video superposition control comprises a chroma key video superposition unit.

5. The graphic image system according to claim 3, wherein said second image comprises a computer generated animated image stream.

6. The graphic image system according to claim 1, wherein said image position estimating system comprises a radio frequency field sensor.

7. The graphic image system according to claim 1, wherein said image position estimating system comprises an electro-acoustic transducer.

8. The graphic image system according to claim 1, wherein said image position estimating system comprises an optical position sensing system.

9. The graphic image system according to claim 1, wherein said estimated boundary comprises a geometric shape.

10. The graphic image system according to claim 1, wherein said image position estimating system comprises a physical transducer mounted proximate to said head, said first signal being received from a video camera viewing said human subject.

11. The graphic image system according to claim 10, wherein said video signal is received from a video camera, said video camera having a field of view, said estimated boundary being located at a position relative to a projection of said physical transducer in said field of view of said video camera.

12. The graphic image system according to claim 1, wherein said image position sensing system senses position along at least two axes.

13. The graphic image system according to claim 1, wherein said image position sensing system senses position along at least three axes.

14. The graphic image system according to claim 1, wherein said image position sensing system senses position along at least six axes.

15. The graphic image system according to claim 11, wherein said estimated boundary varies based on an orientation of said head.

16. The graphic image system according to claim 5, wherein said computer generated animated image stream includes a color keyed portion in a position corresponding to said estimated boundary.

17. The graphic image system according to claim 1, wherein said image position estimating system dynamically tracks a position of the head of the human subject.

18. The graphic image system according to claim 1, wherein said first image comprises an uncontrolled background.

19. The graphic image system according to claim 2, further comprising an input for receiving control signals for controlling said source of said second signal.

20. The graphic image system according to claim 1, wherein said mask outline in said second image is surrounded by an image of a head.

21. The graphic image system according to claim 1, wherein said first image includes said face and a remainder, further comprising a telecommunication system for transmitting an image of said face substantially without said remainder.

22. The graphic image system according to claim 1, wherein said first image includes said face and a remainder, further comprising a display for displaying said face substantially without said remainder.

23. The graphic image system according to claim 1, further comprising a plurality of first signals, each representing a first image including a moving human subject having a head with a face and a remainder, and an output for outputting said faces of said plurality of first images substantially without said remainders.

25. The graphic image system according to claim 24, wherein said computer generates a second video signal including a portion corresponding to said mask region.

26. The graphic image system according to claim 25, wherein said second video signal defines an image, said image being responsive to said position estimating system.

27. The graphic image system according to claim 24, further comprising a system for selectively processing a portion of said first image within said mask region.

28. The graphic image system according to claim 27, wherein said selectively processing comprises producing an output video signal defining an image which substantially includes said foreground object and substantially excludes said background.

30. The graphic image system according to claim 29, further comprising means for generating a second video signal which replaces said substantially excluded background.

31. The graphic image system according to claim 30, wherein said second video signal defines a second image having an object which interacts with said foreground object.

32. The graphic image system according to claim 31, wherein said object of said second image tracks a movement of said foreground object.

33. The graphic image system according to claim 31, wherein said object of said second image is superimposed on said foreground object.

34. The graphic image system according to claim 24, wherein said mask region comprises a chroma key.

35. The graphic image system according to claim 24, wherein said mask region is encoded as an in-band video signal.

36. The graphic image system according to claim 24, wherein said mask region is encoded as an out-of-band video signal.

37. The graphic image system according to claim 25, wherein said second video signal comprises a computer generated animated image stream.

38. The graphic image system according to claim 24, wherein said position estimating system comprises a radio frequency field sensor.

39. The graphic image system according to claim 24, wherein said position estimating system comprises an electro-acoustic transducer.

40. The graphic image system according to claim 24, wherein said position estimating system comprises an optical position sensing system.

41. The graphic image system according to claim 24, wherein said mask region comprises a geometric shape.

42. The graphic image system according to claim 24, wherein said video camera has a field of view, said mask region being defined by a projection of said position in said field of view of said video camera.

43. The graphic image system according to claim 24, wherein said position estimating system further comprises an orientation estimating system for determining an orientation of at least a portion of said foreground object, and said video camera has a field of view, said mask region being defined by a projection of said position and said orientation in said field of view of said video camera.

44. The graphic image system according to claim 24, wherein said mask region has a variable size.

45. The graphic image system according to claim 24, wherein said mask region has a variable shape.

46. The graphic image system according to claim 24, wherein said computer forms a composite image from said foreground object and a separate background image.

47. The graphic image system according to claim 24, further comprising a telecommunication system for transmitting an image of said foreground object substantially without said background.

48. The graphic image system according to claim 24, further comprising a display for displaying said foreground object substantially without said background.

49. The graphic image system according to claim 24, further comprising a first and second image, said first and second images each having a moveable foreground object and a background, and an out put for outputting said foreground objects of said first and second images substantially without said backgrounds.

24. A graphic image system comprising, in combination:

a video camera producing a first video signal defining a first image including a foreground object and a background;

position estimating system for identifying a position with respect to said foreground object, said foreground object having features in constant physical relation to said position; and

a computer, responsive to said position estimating system, for defining a mask region separating said foreground object from said background.

50. A method for separating a foreground object from a background of an image, comprising the steps of:

(a) receiving an image of a foreground object;

(b) estimating a position of said foreground object within said image;

(c) masking said image in relation to said estimated position of said foreground object, to define a boundary of said foreground object; and

(d) outputting said masked foreground object.