Camera system with eye monitoring

• US 20040100567A1
• Filed: 11/25/2002
• Published: 05/27/2004
• Est. Priority Date: 11/25/2002
• Status: Active Grant

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Eastman Kodak Company
• Inventors: Fedorovskaya, Elena A., Miller, Michael E., Cerosaletti, Cathleen D., Covannon, Edward
• Granted Patent: US 7,206,022 B2
• Time in Patent Office: Days
• US Class Current: 348/239
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: H04N 1/00127 Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture
• CPC Class Codes: H04N 1/2112 using still video cameras
• CPC Class Codes: H04N 1/32128 attached to the image data, e.g. file header, transmitted message header, information on the same page or in the same computer file as the image for information embedded in the image H04N1/32144
• CPC Class Codes: H04N 2101/00 Still video cameras
• CPC Class Codes: H04N 2201/3204 of data relating to a user, sender, addressee, machine or electronic recording medium H04N2201/3256 and subgroups take precedence
• CPC Class Codes: H04N 2201/3225 of data relating to an image, a page or a document
• CPC Class Codes: H04N 2201/3274 Storage or retrieval of prestored additional information
• CPC Class Codes: H04N 23/50 Constructional details
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/633 for displaying additional information relating to control or operation of the camera

Abstract

In one aspect of the invention a camera system is provided having an image capture system adapted to capture an image of a scene during an image capture sequence and an eye monitoring system adapted to determine eye information including a direction of the gaze of an eye of a user of the camera system. A controller is adapted to store the determined eye information including information characterizing eye gaze direction during the image capture sequence and to associate the stored eye information with the scene image.

Claims

1. A camera system comprising:

an image capture system adapted to capture an image of a scene during an image capture sequence;

an eye monitoring system adapted to determine eye information including a direction of the gaze of an eye of a user of the camera system; and

,

a controller adapted to store the determined eye information during the image capture sequence and to associate the stored eye information with the scene image.

2. The camera system of claim 1, wherein the eye information determined by the eye monitoring system further includes information characterizing the diameter of the pupil during the image capture sequence.

3. The camera system of claim 1, wherein the eye information determined by the eye monitoring system further includes information characterizing eye temperature during the image capture sequence.

4. The camera system of claim 1, wherein the eye information determined by the eye monitoring system further includes information characterizing eye temperature during the image capture sequence.

5. The camera system of claim 1, wherein the eye information determined by the eye monitoring system further includes eye information from both eyes of the user.

6. The camera system of claim 5 wherein the controller is further adapted to determine focus distance information during the image capture sequence based upon parallax differences between the eye gaze direction of both eyes of observer.

7. The camera system of claim 5, wherein the controller is further adapted to compare eye gaze direction information determined from monitoring of both eyes of the observer for validation.

8. The camera system of claim 1, further comprising an input transducer adapted to generate signals in response to manual input from a user of the camera system and wherein said controller is adapted to store input transducer information during the image capture sequence and to associate the input transducer information with the scene image.

9. The camera system of claim 8, wherein the input transducer further comprises a split composition mode selector and wherein controller is further adapted to permit composition of a single scene image during more than one separated composition phases and to store eye information during each of the more than one separate periods within the composition phase.

10. The camera system of claim 1, wherein the image capture system is a digital image capture system operable to form evaluation images during image composition and further comprising a display to present the evaluation images.

11. The camera system of claim 1, wherein the image capture system comprises a digital image capture system for capturing a scene image in the form of digital data and the controller converts the eye information into digital data and stores the eye information in association with the digital data from the scene image.

12. The camera system of claim 1, wherein the image capture system comprises a digital image capture system for capturing a scene image in the form of digital data and the controller converts the eye information into digital data and stores the eye information in a common data file with the digital data forming the scene image.

13. The camera system of claim 1, further comprising an additional image capture system adapted to store scene images on a photosensitive medium and having a write head to optically record the stored eye information on the photosensitive medium.

14. The camera system of claim 1, wherein the controller stores the eye information apart from the scene image and the controller associates the scene image with the eye information by recording data with the scene image which can be used to locate the eye information.

15. The camera system of claim 1, wherein the eye monitoring system comprises an eye imager adapted to capture images of the eye and an image processor adapted to determine eye information based upon analysis of the captured images of the eye.

16. The camera system of claim 1, wherein said eye monitoring system comprises an eye imager adapted to capture images of the eye and a processor adapted to determine the identity of the user of the camera system based upon the eye images and to associate the user identity with the scene image.

17. A camera system comprising:

an image capture system for capturing an archival image of a scene during an archival image capture sequence;

an eye monitoring system adapted to determine eye information from an eye of a user;

a source of context information; and

a controller adapted to store the eye information and context information during the archival image capture sequence and to associate the stored information with the archival image captured during the archival image capture sequence.

18. The camera system of claim 17 wherein the source of context information comprises a location detector adapted to determine where the archival image was captured.

19. The camera system of claim 17 wherein the context information comprises information characterizing changes in the composition of the scene during the archival image capture process.

20. The camera system of claim 19, wherein the controller causes the image capture system to capture a set of evaluation images during a composition phase of the archival image capture sequence and the source of context information comprises an image processor adapted to determine image information characterizing changes in the composition of the scene during the composition phase based on image analysis of the set of evaluation images.

21. The camera system of claim 19 wherein the controller causes the image capture system to capture a set of evaluation images during a composition phase of the archival image capture sequence and the source of context information comprises an image processor adapted to determine image information characterizing changes in the composition of the scene during the composition phase based on image analysis of the set of evaluation images to locate fiducials in the image.

22. The camera system of claim 19 wherein the controller causes the image capture system to capture a set of evaluation images during the archival image capture sequence and wherein the source of context data comprises an image processor adapted to determine information characterizing changes in the field of view of the archival image capture system by assembling a composite image mapping all portions of the scene imaged by any of the set of captured evaluation images and said processor records eye gaze information in relation to the assembled composite.

23. The camera system of claim 17, wherein said controller comprises said source of context information and said controller is adapted to cause the image capture system to capture evaluation images during the image capture sequence and to store the captured evaluation images in association with the captured archival image.

24. The camera system of claim 17, wherein the source of context information comprises a manual input and the controller monitors the manual input during the image capture sequence and associates information characterizing changes in the status of the manual input during the image capture process with the captured archival image.

25. The camera system of claim 24 wherein the image capture system comprises a taking lens unit having an adjustable zoom setting and the manual input comprises a zoom setting input.

26. The camera system of claim 24 wherein the image capture system comprises a taking lens unit having an adjustable zoom setting and the manual input comprises a zoom setting input with the controller adjusting the eye gaze information based upon the zoom setting input.

27. The camera system of claim 17, wherein the source of context data comprises a scene brightness detector.

28. The camera system of claim 17, wherein the image capture system captures the archival image in the form of a digital archival image signal and the controller stores the electronic archival image signal in association with the eye movement data signal and the context information in digital form.

29. The camera system of claim 17, further comprising an archival image capture system that captures the archival image on a photosensitive element and further comprising a recording head for optically recording the eye information and context information on the photosensitive element.

30. The camera system of claim 17, further comprising an archival capture system that captures images on a photosensitive element and the archival image capture system comprises a film data recorder for recording the eye information and context information onto the film.

31. The camera system of claim 17, wherein the eye monitoring system comprises an optical system that presents light from the scene to form an image of the scene at an eye of an observer and an eye camera that captures images of the eye during the image capture sequence.

32. The camera system of claim 31, wherein a beam splitter is disposed in the optical path between the scene and the eye of the observer, with the beam splitter adapted to deflect an image of the eye to the eye camera.

33. The camera system of claim 31 wherein the eye monitoring system further comprises an image processor to examine the eye images and to determine eye gaze direction.

34. The camera of claim 31 wherein the eye monitoring system comprises an eye camera that further detects pupil dilation and wherein the controller is further adapted to incorporate pupil dilation information in the eye information.

35. The camera system of claim 17, further comprising an evaluation image capture sequence and an evaluation image display adapted to present captured evaluation images to a display space for observation by a user, an eye imager capturing images of the display space and wherein said controller is further adapted to receive images from the eye imager and to determine information characterizing eye gaze direction of an eye in the display space.

36. The camera system of claim 35, further comprising a beam splitter interposed between the display and the eye with the beam splitter reflecting light from the eye onto the eye imager.

37. The camera system of claim 17 wherein the camera system is incorporated into a set of glasses.

38. A method for operating a camera system, the method comprising the steps of:

monitoring eye information including the eye gaze direction of an eye of a user of the camera system;

capturing an image during an image capture sequence;

storing the eye information during the image capture sequence; and

associating the stored eye information with the captured image.

39. The method of claim 38 further comprising the step of storing the eye gaze information with the archival image.

40. The method of claim 38, further comprising the steps of detecting scene composition changes during composition, storing information characterizing the changes in composition during the image capture sequence and associating the stored scene composition information with the captured image.

41. The method of claim 38 wherein the step of monitoring eye information during the image capture sequence includes monitoring a diameter of a pupil of the eye of the user during composition.

42. The method of claim 38, further comprising the steps of forming an evaluation image corresponding to the captured image and presenting the evaluation image to the user of the camera system and monitoring eye information including eye gaze information during presentation of the evaluation image.

43. A method for operating a camera system comprising the steps of:

determining eye information including eye gaze direction of an eye of a user of the camera system during an image capture sequence;

determining context information during the image capture sequence;

capturing an archival image during the image capture sequence;

storing information characterizing the eye information and the context information during the image capture sequence; and

associating the eye information and the context information with the archival image.

44. The method of claim 43, wherein the eye information further includes pupil dilation information.

45. The method of claim 43, wherein the step of determining eye information comprises capturing a sequence of images of the eye.

46. The method of claim 45, wherein the step of determining eye information further comprises the step of analyzing the sequence of eye images to derive eye information.

47. The method of claim 43 wherein the context information includes information characterizing changes in the field of view of the image capture system.

48. The method of claim 43 wherein the camera system has a user interface adapted to receive user inputs and the step of determining context information comprises monitoring changes in the user inputs.