Gaze Intent Estimation for Retargeting of Characters

• US 20110018862A1
• Filed: 07/22/2009
• Published: 01/27/2011
• Est. Priority Date: 07/22/2009
• Status: Active Grant

• Current Assignee: Two PIC MC LLC (The Walt Disney Company)
• Original Assignee: ImageMovers Digital LLC (The Walt Disney Company)
• Inventors: Epps, Doug
• Granted Patent: US 8,860,729 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings

Abstract

Any image capturing system, and a method to use same, that includes one or more image capturing devices that have known and fixed positions relative to one another and the eye of a subject, wherein the image capturing devices capture a plurality of images of eye of a subject during a performance is disclosed. The images of the eye, along with the known and fixed relative positions of the image capturing devices are used define eye-lines for the eye of the subject. The eye-lines can then be used to manually or automatically retarget the eye of a rendered or animated figure based on the performance of the subject to retarget the gaze intent of the rendered or animated figure within a scene.

Claims

1. In a computer aided animation system, a method for retargeting the gaze of a rendered image comprising:

determining an eye-line of an eye model using a computer system;

determining a gaze-object at which the eye model is directed in response to the eye-line using the computer system;

determining a relative change in position of the gaze-object relative to the eye model using the computer system;

adjusting the eye-line of the eye model to remain on the gaze-object using the computer system;

rendering an image of the eye model in response to the adjusted eye-line of the eye model using the computer system;

displaying the rendered image with adjusted image of the eye on an output device using the computer system.

2. The method of claim 1 wherein the eye model comprises two or more model eyes, wherein the eye-line comprises two or more eye-lines and wherein determining the gaze-object at which the two or more eye models are aimed is in response to the two or more eye-lines

3. The method of claim 1 further comprising:

saving the rendered image to a memory store using the computer system; and

retrieving the rendered image from the memory store using the computer.

4. The method of claim 1 further comprising

providing a plurality of imaging devices with known positions relative to an eye of a subject actor;

and

capturing a plurality of images of the eye of the subject actor with the plurality cameras;

wherein determining an eye-line of the eye model comprises determining an eye-line of the eye of the subject actor in response to the images of the eye of the subject actor, an eye model and the known positions of the plurality of cameras;

wherein determining a gaze-object at which the eye model is directed comprises determining a gaze-object at which the eye of the subject actor is aimed; and

wherein determining a relative change in position of the gaze-object relative to the eye model comprises determining a relative change in position of the gaze-object relative to the subject actor.

5. The method of claim 4 wherein the eye of the subject actor comprises two eyes, wherein the eye model comprises two or more model eyes, wherein the eye-line comprises two eye-lines and wherein determining the gaze-object at which the two eyes of the subject actor are aimed is in response to the two eye-lines.

6. The method of claim 4 wherein the plurality of cameras have known positions relative to the eye of the subject actor and comprises two or more cameras for the eye of the subject actor.

7. The method of claim 1 wherein the model eye comprises a digital model of an ideal eye.

8. The method of claim 7 wherein determining the gaze-object comprises determining where the two eye-lines intersect.

9. An apparatus for retargeting the gaze of rendered images comprising:

a plurality of imaging devices configured to maintain a constant position and orientation relative to the eyes of a subject actor;

a computer system in communication with the plurality of imaging devices, wherein the computer system comprises one or more digital eyes models comprising a plurality of eye-lines usable to retarget the one or more digital eye models.

10. The apparatus of claim 9 wherein the plurality of imaging devices are attached to the ends of a plurality of arms, wherein the arms are rigid and securely attached to a head gear adapted to be securely attached to the head of a subject actor.

11. The apparatus of claim 10 wherein the plurality of imaging devices comprises two cameras with known locations relative to one another for each eye of the subject actor.

12. The apparatus of claim 11 wherein the plurality of imaging devices are configured to image the two eyes of the subject actor from two from two different perspectives.

13. The apparatus of claim 9 wherein the computer system is in communication with the plurality of imaging devices via wireless connection.

14. The apparatus of claim 9 wherein the at least one of the plurality of imaging devices is configured to detect illumination outside of the human visible range.

15. The apparatus of claim 10 wherein the subject actor is human.

16. The apparatus of claim 10 wherein the subject actor is an animal other than human.

17. In a computer aided animation system, a method for retargeting the gaze of a rendered figure in real-time comprising:

retrieving an eye-line of an eye model to a designated gaze-object,

determining a relative change in position of the gaze-object relative to the subject actor;

adjusting the eye-line of the eye model to remain on the designated gaze-object after the change in position;

rendering the eye model for the rendered figure in response to the adjusted eye-line of the eye model;

displaying the rendered figure and the adjusted rendered eye model on an output device.

18. The method of claim 17 wherein the eye model is constrained to adjust only in a forward direction relative to the rendered figure.

19. The method of claim 17 wherein the defined eye-line is as a ray emanating from the center of a pupil of the eye model perpendicular to a plane defined by three points on the circumference of the pupil.

20. The method of claim 17 wherein the change in position of the gaze-object relative to the rendered figure comprises a change in the designated gaze-object from a first designated gaze-object to a second designated gaze-object.