Video superposition system and method

• US 6,400,374 B2
• Filed: 09/18/1996
• Issued: 06/04/2002
• Est. Priority Date: 09/18/1996
• Status: Expired due to Term

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Neven Vision, Inc. (Alphabet Inc.)
• Inventors: Lanier, Jaron
• Primary Examiner(s): Saras, Steven
• Assistant Examiner(s): AWAD, AMR A
• Application Number:
  US08/716,597
• Publication Number:
  US 20020018070A1
• Time in Patent Office: 2,085 Days
• Field of Search: 345/7-9, 345/118, 345/113, 345/114, 345/330, 345/435, 345/327, 345/231, 345/584, 345/232, 345/233, 345/598, 345/333, 345/156-158, 345/630, 345/473-474, 345/590
• US Class Current: 345/630
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06T 15/10 Geometric effects
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G09G 2340/125 wherein one of the images is motion video
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q

Abstract

A graphic image system comprising a video camera producing a first video signal defining a first image including a foreground object and a background, the foreground object preferably including an image of a human subject having a head with a face; an image position estimating system for identifying a position with respect to said foreground object, e.g., the head, the foreground object having features in constant physical relation to the position; and a computer, responsive to the position estimating system, for defining a mask region separating the foreground object from said background. The computer generates a second video signal including a portion corresponding to the mask region, responsive to said position estimating system, which preferably includes a character having a mask outline. In one embodiment, the mask region of the second video signal is keyed so that the foreground object of the first video signal shows through, with the second video signal having portions which interact with the foreground object. In another embodiment, means, responsive to the position estimating system, for dynamically defining an estimated boundary of the face and for merging the face, as limited by the estimated boundary, within the mask outline of the character. Video and still imaging devices may be flexibly placed in uncontrolled environments, such as in a kiosk in a retail store, with an actual facial image within the uncontrolled environment placed within a computer generated virtual world replacing the existing background and any non-participants.

Claims

1. A graphic image system comprising:

a source of a first video signal representing a first image including a moving human subject having a head with a face;

an image position estimating system for estimating the instantaneous position of said head of said human subject;

a source of a second signal representing a second image including a character having a head with a mask outline, wherein said mask outline moves in response to the instantaneous position of said head in said first image; and

means, responsive to said position estimating system and to said first and second signals, for dynamically defining an estimated boundary of said face of said human subject in said first image and for merging the face in said first image, as limited by said estimated boundary, with the second image within the mask outline to generate a third signal representing an animated avatar having said face presented in said mask outline.

2. The graphic image system according to claim 1, said means comprising a video superposition control for superposing said face within said estimated boundary.

3. The graphic image system according to claim 2, wherein said mask outline in said second image is responsive to said image position estimating system.

4. The graphic image system according to claim 2, wherein said video superposition control comprises a chroma key video superposition unit.

5. The graphic image system according to claim 3, wherein said second image comprises a computer generated animated image stream.

6. The graphic image system according to claim 5, wherein said computer generated animated image stream includes a color keyed portion in a position corresponding to said estimated boundary.

7. The graphic image system according to claim 1, wherein said image position estimating system comprises a radio frequency field sensor.

8. The graphic image system according to claim 1, wherein said image position estimating system comprises an electro-acoustic transducer.

9. The graphic image system according to claim 1, wherein said image position estimating system comprises an optical position sensing system.

10. The graphic image system according to claim 1, wherein said estimated boundary comprises a geometric shape.

11. The graphic image system according to claim 1, wherein said image position estimating system comprises a physical transducer mounted proximate to said head, said first signal being received from a video camera viewing said human subject.

12. The graphic image system according to claim 11, wherein said video signal is received from a video camera, said video camera having a field of view, said estimated boundary being located at a position relative to a projection of said physical transducer in said field of view of said video camera.

13. The graphic image system according to claim 12, wherein said estimated boundary varies based on an orientation of said head.

14. The graphic image system according to claim 1, wherein said image position estimating system senses position along at least two axes.

15. The graphic image system according to claim 1, wherein said image position estimating system senses position along at least three axes.

16. The graphic image system according to claim 1, wherein said image position estimating g system senses position along at least six axes.

17. The graphic image system according to claim 1, wherein said image position estimating system dynamically tracks a position of the head of the human subject.

18. The graphic image system according to claim 1, wherein said mask outline in said second image is surrounded by an image of a head.

19. The graphic image system according to claim 1, wherein said first image includes said face and a remainder portion of said human subject, further comprising a telecommunication system for transmitting an image of said face substantially without said remainder portion.

20. The graphic image system according to claim 1, wherein said first image includes said face and a remainder portion of said human subject, further comprising a display for displaying said face substantially without said remainder portion.

21. The graphic image system according to claim 1, further comprising a plurality of first signals, each representing a first image including a moving human subject having a head with a face and a remainder portion of said human subject;

and an output for outputting said faces of said plurality of first images substantially without said remainder portion.

22. The graphic image system according to claim 1, wherein said first video signal has a frame rate sufficient to show animation of said moving human subject.

23. The graphic image system according to claim 1, wherein

the position estimating system further estimates the instantaneous orientation of said head of said human subject; and

said mask outline in said second image is oriented based on the instantaneous orientation of said head in said first image.

24. A method for personalizing an avatar, comprising steps for:

generating a video image of a moving person having a head with a face;

sensing and tracking a position of the moving person'"'"'s face in the video image;

generating a mask portion image including a face;

generating a background image including an animated avatar having a face position portion that moves within the background image in response to the tracked position of the moving person'"'"'s face in the video image; and

generating a superimposed image having the mask portion image including the face overlaid on the background image in the responsively moving face position portion of the animated avatar.

25. A method for personalizing an avatar as defined in claim 24, wherein the video image of the person is a live video image.

26. A method for personalizing an avatar as defined in claim 24, wherein:

the step of sensing the position of the person'"'"'s face in the video image includes sensing the orientation of the person'"'"'s face in the video image; and

the step of generating a background image including an animated avatar having a face position portion includes orienting the face position portion within the background image based on the person'"'"'s face orientation in the video image.

27. A method for personalizing an avatar as defined in claim 24, wherein the face in the mask portion image is based on the moving person'"'"'s face.

28. A method for personalizing an avatar as defined in claim 24, wherein the face in the mask portion image is generated from the video image of the moving person.

29. A method for representing multiple moving individuals, each located at a respective remote location, as personalized avatars within displayed video images, comprising steps for:

generating video images, each video image associated with one of the multiple individuals at the respective remote locations, and each individual having a head with a face;

sensing and tracking a position of each individual'"'"'s face in the respective video image;

generating respective a mask portion image that includes the respective individual'"'"'s face;

generating a background image at each remote location, each background image including animated avatars, each animated avatar corresponding to a respective individual and having a face position portion that moves within the background image in response to the moving individual'"'"'s tracked face position in the respective video image;

providing the respective mask portion images and tracked face positions to the respective remote locations over a network; and

generating a superimposed image at each of the respective remote locations, each superimposed image having the mask portion images overlaid in the respective responsively moving face position portions of the animated avatars.

30. A method for representing multiple individuals as personalized avatars within displayed video images as defined in claim 29, wherein each video image associated with one of the multiple individuals at the respective remote locations is a live video image.

31. A method for representing multiple individuals as personalized avatars within video images as defined in claim 29, wherein:

the step of sensing the position of the individual'"'"'s face in the respective video image includes sensing the orientation of the individual'"'"'s face in the video image; and

the step of generating a background image including animated avatars each having a face position portion includes orienting the respective face position portion in the background image based on the corresponding individual'"'"'s face orientation in the video image.