Wearable inward-facing camera utilizing the Scheimpflug principle

• US 20180027158A1
• Filed: 10/02/2017
• Published: 01/25/2018
• Est. Priority Date: 06/14/2015
• Status: Active Grant

• Current Assignee: Facense Ltd.
• Original Assignee: Facense Ltd.
• Inventors: Tzvieli, Arie, Thieberger, Gil, Frank, Ari M.
• Granted Patent: US 10,523,852 B2
• Time in Patent Office: Days
• CPC Class Codes: A61B 2562/0271 Thermal or temperature sensors
• CPC Class Codes: A61B 5/0075 by spectroscopy, i.e. measuring spectra, e.g. Raman spectroscopy, infrared absorption spectroscopy A61B5/0071 takes precedence
• CPC Class Codes: A61B 5/0077 Devices for viewing the surface of the body, e.g. camera, magnifying lens
• CPC Class Codes: A61B 5/015 By temperature mapping of body part
• CPC Class Codes: A61B 5/163 by tracking eye movement, gaze, or pupil change
• CPC Class Codes: A61B 5/165 Evaluating the state of mind, e.g. depression, anxiety
• CPC Class Codes: A61B 5/6803 Head-worn items, e.g. helmets, masks, headphones or goggles
• CPC Class Codes: A61B 5/6814 Head
• CPC Class Codes: G01J 5/0025 Living bodies ear thermometers G01J5/0011; detecting, measuring or recording for diagnostic purposes A61B5/00
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/0176 characterised by mechanical features
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: H04N 23/55 Optical parts specially adapted for electronic image sensors; Mounting thereof
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

One aspect of this disclosure involves a wearable device that includes a frame that is worn on a user'"'"'s head, and an inward-facing camera (camera) physically coupled to the frame. The optical axis of the camera is either above the Frankfort horizontal plane and pointed upward to capture an image of a region of interest (ROI) above the user'"'"'s eyes, or the optical axis is below the Frankfort horizontal plane and pointed downward to capture an image of an ROI below the user'"'"'s eyes. The camera includes a sensor and a lens. The sensor plane is tilted by more than 2° relative to the lens plane according to the Scheimpflug principle in order to capture a sharper image. The Scheimpflug principle is a geometric rule that describes the orientation of the plane of focus of a camera when the lens plane is tilted relative to the sensor plane.

Claims

1. A wearable device, comprising:

a frame configured to be worn on a user'"'"'s head; and

an inward-facing camera (camera) physically coupled to the frame;

wherein the optical axis of the camera is either above the Frankfort horizontal plane and pointed upward to capture an image of a region of interest (ROI) above the user'"'"'s eyes, or the optical axis is below the Frankfort horizontal plane and pointed downward to capture an image of an ROI below the user'"'"'s eyes;

the camera comprises a sensor and a lens;

wherein the sensor plane is tilted by more than 2°

relative to the lens plane according to the Scheimpflug principle in order to capture a sharper image.

2. The wearable device of claim 1, wherein the ROI covers first and second areas, and the tilt between the lens plane and the sensor plane is adjusted such that the image of the first area is shaper than the image of the second area;

whereby the first area is more important for detecting a physiological response of the user than the second area.

3. The wearable device of claim 2, wherein the first area is on the user'"'"'s upper lip and the second area is on the user'"'"'s cheek, and the tilt is adjusted such that the image of the first area is shaper than the image of the second area.

4. The wearable device of claim 2, wherein the first area is on the user'"'"'s upper lip and the second area is on the user'"'"'s nose, and the tilt is adjusted such that the image of the first area is shaper than the image of the second area.

5. The wearable device of claim 2, wherein the first area is on the user'"'"'s lips and the second area is on the user'"'"'s chin, and the tilt is adjusted such that the image of the first area is shaper than the image of the second area.

6. The wearable device of claim 2, wherein the camera is a visible-light camera, the first area is on the user'"'"'s lower forehead (including an eyebrow) and the second area is on the user'"'"'s upper forehead, and the tilt is adjusted such that the image of the first area is shaper than the image of the second area.

7. The wearable device of claim 2, wherein the camera is a thermal camera, the first area is on the middle and upper part of the user'"'"'s forehead (below the hair line), the second area is on the lower part of the user'"'"'s forehead, and the tilt is adjusted such that the image of the first area is shaper than the image of the second area.

8. The wearable device of claim 1, wherein the ROI covers a first area on the cheek straight above the upper lip, a second area on the cheek from the edge of the upper lip towards the ear, and a third area on the nose:

wherein the tilt between the lens plane and the sensor plane is adjusted such that the image of the first area is shaper than both the images of the second and third areas.

9. The wearable device of claim 1, wherein the ROI covers both the region on the user'"'"'s upper lip and a second region on the body below the face when the user is standing;

and wherein the sharpness of the image of both the region and the second region, which is obtained from the camera when the user wears the frame, is better than the sharpness of an image of both the region and the second region that would be obtained from a similar camera having the same sensor and lens, but with the lens plane parallel to the sensor plane.

10. The wearable device of claim 1, wherein the camera is a thermal camera configured to take thermal measurements of the ROI;

and further comprising a computer configured to detect a physiological response based on the thermal measurements.

11. The wearable device of claim 10, wherein the sensor comprises multiple sensing elements, and the computer is configured to process time series measurements of each sensing element individually in order to detect the physiological response.

12. The wearable device of claim 10, wherein the thermal camera weighs below 10 g, is located less than 10 cm from the user'"'"'s face, and the tilt of the lens plane relative to the sensor plane is fixed;

and wherein the fixed tilt is selected according to an expected orientation between the camera and the ROI when the user wears the frame.

13. The wearable device of claim 1, wherein the camera is a visible-light camera configured to take visible-light images of the ROI;

and further comprising a computer configured to generate an avatar of the user based on the visible-light images.

14. The wearable device of claim 1, wherein the camera is a visible-light camera configured to take visible-light images of the ROI;

and further comprising a computer configured to detect the user'"'"'s emotional response based on identifying facial expressions in the visible-light images.

15. The wearable device of claim 1, wherein the camera is a light field camera configured to:

(i) implement a predetermined blurring at a certain Scheimpflug angle, and (ii) decode the predetermined blurring as function of the certain Scheimpflug angle.

16. The wearable device of claim 1, further comprising an adjustable electromechanical tilting mechanism configured to change the tilt of the lens plane relative to the sensor plane according to the Scheimpflug principle, based on the orientation between the camera and the ROI when the camera is mounted on the user'"'"'s head.

17. The wearable device of claim 16, wherein the camera, including the adjustable electromechanical tilting mechanism, weighs less than 10 g, located less than 15 cm from the user'"'"'s face, and the adjustable electromechanical tilting mechanism is able to change the tilt between the lens and sensor planes in a limited range below 30°

between the two utmost orientations between the lens and sensor planes.

18. The wearable device of claim 16, wherein the camera, including the adjustable electromechanical tilting mechanism, weighs less than 10 g, located less than 15 cm from the user'"'"'s face, and the adjustable electromechanical tilting mechanism is able to change the tilt between the lens and sensor planes in a limited range below 20°

between the two utmost orientations between the lens and sensor planes.

19. A device comprising:

an inward-facing head-mounted camera (camera) configured to capture, when worn on a user'"'"'s head, an image of a region of interest (ROI) on at least one of the following regions on the user'"'"'s face;

the forehead, the nose, the upper lip, a cheek, and the lips; and

the camera comprises a sensor and a lens;

wherein the sensor plane is tilted by more than 2°

relative to the lens plane according to the Scheimpflug principle in order to capture a sharper image.

20. The device of claim 19, wherein the ROI covers first and second areas, and the tilt between the lens plane and the sensor plane is adjusted such that the image of the first area is shaper than the image of the second area, whereby the first area is more important for detecting a physiological response of the user than the second area.