Methods and systems for ergonomic feedback using an image analysis module

• US 8,913,005 B2
• Filed: 04/08/2011
• Issued: 12/16/2014
• Est. Priority Date: 04/08/2011
• Status: Expired due to Fees

• Current Assignee: Nan Chang O-Film Optoelectronics Technology, Ltd.
• Original Assignee: FotoNation Limited (Adeia Inc.)
• Inventors: Uppuluri, Avinash, Morales, Celia, Chakravarthula, Hari, Mehra, Sumat, Paoletti, Tomaso
• Primary Examiner(s): Eisen, Alexander
• Assistant Examiner(s): Patel, Sanjiv D
• Application Number:
  US13/082,568
• Publication Number:
  US 20120256820A1
• Time in Patent Office: 1,348 Days
• Field of Search: 345/156, 34833301-33304, 382/190, 382/206
• US Class Current: 345/156
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions

Abstract

A display device can be used with an ergonomic sensor comprising an imaging device interfaced to processing hardware to obtain and analyze image data depicting a user of the display device. The ergonomic sensor can be preconfigured with data indicating ergonomic uses of the display device so that the image of the user can be analyzed with minimal or no user calibration or setup. Instead, the ergonomic sensor can analyze the image data to provide real-time feedback, such as warnings or suggestions when the user'"'"'s behavior falls outside an ergonomic use range for the display device. In some implementations, the ergonomic sensor is integrated with the display device, though in other implementations a separate element or preexisting imaging device can be used.

Claims

1. An ergonomic sensor module configured for use with a display device, the sensor module comprising:

an image sensing device;

a processing element; and

an input/output interface,

wherein the processing element is configured to obtain image data from the image sensing device and analyze the image data to determine, based on predefined data defining an ergonomic use range of the display device, whether the image data indicates that a user of the display device is using the display device within the ergonomic use range, and

wherein analyzing the image data to determine whether the image data indicates that the user of the display device is using the display device within the ergonomic use range comprises;

determining a first image portion from the image data based at least on identifying an area proximate to a face of the user, the first image portion comprising at least the area proximate to the face of the user;

analyzing the first image portion to measure intensity values across the first image portion; and

determining whether a lighting condition is outside the ergonomic use range by comparing a function of the intensity values of the first image portion, but not of a second image portion, with the ergonomic use range.

2. The sensor module of claim 1, wherein the input/output interface is connected to the display device and the processing element is further configured to output a feedback message using the display device in response to determining that the image data indicates that the user of the display device is not using the display device within the ergonomic use range.

3. The sensor module of claim 1, integrated into a structural element of the display device.

4. The sensor module of claim 1, configured as a separate element attached to the display device.

5. The sensor module of claim 1, interfaced to a processing device that is also interfaced to the display device.

6. The sensor module of claim 1, wherein the processing element is configured to analyze the image data to determine whether the image data indicates that the user of the display device is using the display device within the ergonomic use range independently of a change in ambient lighting conditions.

7. The sensor module of claim 1, wherein analyzing the image data to determine whether the image data indicates that the user of the display device is using the display device within the ergonomic use range comprises:

analyzing the image data to determine at least one of a distance or viewing angle of the user relative to the display device and determining whether the distance or viewing angle is outside the ergonomic use range.

9. The sensor module of claim 1, wherein analyzing the image data to determine whether the image data indicates that the user of the display device is using the display device within the ergonomic use range comprises:

analyzing the image data to determine at least one of a roll angle of the face of the user, a yaw angle of the face of the user, or a pitch angle of the face of the user, respectively relative to the display device to determine if the face is within an ergonomic use range based on the determined angle or angles.

10. The sensor module of claim 1, wherein analyzing the image data to determine whether the image data indicates that the user of the display device is using the display device within the ergonomic use range comprises:

analyzing the image data to identify multiple users in view and select a particular user from the multiple users, the image data of the particular user analyzed to determine if the particular user is using the display device within the ergonomic use range.

11. The sensor module of claim 1, wherein analyzing the image data to determine whether the image data indicates that the user of the display device is using the display device within the ergonomic use range comprises:

analyzing the image data to determine whether the user has been proximate the display device beyond a threshold length of time without a break.

12. The sensor module of claim 1, wherein the processing element comprises hardware logic interfaced to the sensor, the hardware logic configured to analyze the image data to determine whether the image data indicates that the user is using the display device within the ergonomic use range.

13. The sensor module of claim 1, wherein the processing element is configured to analyze the image data to determine whether the image data indicates that the user of the display device is using the display device within the ergonomic use range independently of a shape of the face of the user.

15. The sensor module of claim 1, wherein the processing element is configured to analyze the image data to determine whether the image data indicates that the user of the display device is using the display device within the ergonomic use range independently of precaptured posture data for the user.

16. The sensor module of claim 1, wherein the processing element is configured to determine an orientation of the display device.

28. The sensor module of claim 1, wherein the ergonomic use range for the lighting condition is from 100 to 900 lux units of luminous emittance.

34. The sensor module of claim 1, wherein determining the first image portion from the image data comprises:

analyzing the image data to measure intensity values across the image data; and

identifying the first image portion as an area where the intensity values of the first image portion are higher than the intensity values of the second image portion.

8. An ergonomic sensor module configured for use with a display device, the sensor module comprising:

an image sensing device;

a processing element; and

an input/output interface,

wherein the processing element is configured to obtain image data from the image sensing device and analyze the image data to determine, based on predefined data defining an ergonomic use range of the display device, whether the image data indicates that a user of the display device is using the display device within the ergonomic use range, wherein analyzing the image data to determine whether the image data indicates that the user of the display device is using the display device within the ergonomic use range comprises;

analyzing the image data to determine whether the user has stared without blinking beyond a threshold period of time defined in the ergonomic use range.

31. The sensor module of claim 8, wherein the input/output interface is connected to the display device and the processing element is further configured to output a feedback message using the display device in response to determining that the image data indicates that the user has stared without blinking beyond the threshold period of time defined in the ergonomic use range.

14. An ergonomic sensor module configured for use with a display device, the sensor module comprising:

an image sensing device;

a processing element; and

an input/output interface,

wherein the processing element is configured to obtain image data from the image sensing device and analyze the image data to determine, based on predefined data defining an ergonomic use range of the display device, whether the image data indicates that a user of the display device is using the display device within the ergonomic use range, and

wherein the processing element is configured to analyze the image data to determine whether the image data indicates that the user of the display device is using the display device within the ergonomic use range, wherein analyzing the image data to determine whether the image data indicates that the user of the display device is using the display device within the ergonomic use range comprises determining whether a glare condition, outside of the ergonomic use range, exists.

17. A computerized method, comprising:

accessing image data from an image sensing device, the image data depicting the user of a display device;

analyzing the image data to determine, based on predefined data defining an ergonomic use range of the display device, whether the image data indicates that the user of the display device is using the display device within the ergonomic use range, and

wherein analyzing the image data to determine whether the image data indicates that the user of the display device is using the display device within the ergonomic use range comprises;

determining a first image portion from the image data based at least on identifying an area proximate to a face of the user, the first image portion comprising at least the area proximate to the face of the user;

analyzing the first image portion to measure intensity values across the first image portion; and

determining whether a lighting condition is outside the ergonomic use range by comparing a function of the intensity values of the first image portion, but not of a second image portion, with the ergonomic use range.

18. The computerized method of claim 17, further comprising:

providing output data representing a feedback message to be displayed in real-time using the display device in response to analyzing.

19. The computerized method of claim 18, wherein the feedback message indicates that the user of the display device is not using the display device within the ergonomic use range.

20. The computerized method of claim 17, further comprising:

determining an orientation of the display device.

29. The computerized method of claim 17, wherein the ergonomic use range for the lighting condition is from 100 to 900 lux units of luminous emittance.

35. The computerized method of claim 17, wherein determining the first image portion from the image data comprises:

analyzing the image data to measure intensity values across the image data; and

identifying the first image portion as an area where the intensity values of the first image portion are higher than the intensity values of the second image portion.

21. A non-transitory computer-readable medium, storing one or more instructions which, when executed by one or more processors, cause performance of a method comprising:

accessing image data from an image sensing device, the image data depicting a user of a display device; and

analyzing the image data to determine, based on predefined data defining an ergonomic use range of the display device, whether the image data indicates that the user of the display device is using the display device within the ergonomic use range, and

wherein analyzing the image data to determine whether the image data indicates that the user of the display device is using the display device within the ergonomic use range comprises;

determining a first image portion from the image data based at least on identifying an area proximate to a face of the user, the first image portion comprising at least the area proximate to the face of the user;

analyzing the first image portion to measure intensity values across the first image portion; and

determining whether a lighting condition is outside the ergonomic use range by comparing a function of the intensity values of the first image portion, but not of a second image portion, with the ergonomic use range.

22. The non-transitory computer-readable medium of claim 21, further comprising instructions which when executed cause:

code for providing output data representing a feedback message to be displayed in real-time using the display device in response to analyzing.

23. The non-transitory computer-readable medium of claim 22, wherein the feedback message indicates that the user of the display device is not using the display device within the ergonomic use range.

30. The non-transitory computer-readable medium of claim 21, wherein the ergonomic use range for the lighting condition is from 100 to 900 lux units of luminous emittance.

36. The non-transitory computer-readable medium of claim 21, wherein determining the first image portion from the image data comprises:

analyzing the image data to measure intensity values across the image data; and

identifying the first image portion as an area where the intensity values of the first image portion are higher than the intensity values of the second image portion.

24. A non-transitory computer-readable medium, storing one or more instructions which, when executed by one or more processors, cause performance of a method comprising:

accessing image data from an image sensing device, the image data depicting a user of a display device; and

analyzing the image data to determine, based on predefined data defining an ergonomic use range of the display device, whether the image data indicates that the user of the display device is using the display device within the ergonomic use range, and

wherein analyzing the image data to determine whether the image data indicates that the user of the display device is using the display device within the ergonomic use range comprises;

analyzing the image data to determine whether the user has stared without blinking beyond a threshold period of time defined in the ergonomic use range.

32. The non-transitory computer-readable medium of claim 24, further comprising instructions which when executed cause:

code for providing output a feedback message using the display device in response to determining that the image data indicates that the user has stared without blinking beyond the threshold period of time defined in the ergonomic use range.

25. A non-transitory computer-readable medium, storing one or more sequences of instructions which, when executed by one or more processors, cause performance of a method comprising:

accessing image data from an image sensing device, the image data depicting a user of a display device;

analyzing the image data to determine, based on predefined data defining an ergonomic use range of the display device, whether the image data indicates that the user of the display is using the display within the ergonomic use range, wherein analyzing the image data to determine whether the image data indicates that the user of the display device is using the display device within the ergonomic use range comprises determining whether a glare condition, outside of the ergonomic use range, exists.

26. A computerized method, comprising:

accessing image data from an image sensing device, the image data depicting a user of a display device;

analyzing the image data to determine, based on predefined data defining an ergonomic use range of the display device, whether the image data indicates that the user of the display device is using the display device within the ergonomic use range, and

wherein analyzing the image data to determine whether the image data indicates that the user of the display device is using the display device within the ergonomic use range comprises;

analyzing the image data to determine whether the user has stared without blinking beyond a threshold period of time defined in the ergonomic use range.

33. The computerized method of claim 26, further comprising:

providing output a feedback message using the display device in response to determining that the image data indicates that the user has stared without blinking beyond the threshold period of time defined in the ergonomic use range.

27. A computerized method, comprising:

accessing image data from an image sensing device, the image data depicting a user of a display device;

analyzing the image data to determine, based on predefined data defining an ergonomic use range of the display device, whether the image data indicates that the user of the display is using the display within the ergonomic use range, wherein analyzing the image data to determine whether the image data indicates that the user of the display device is using the display device within the ergonomic use range comprises determining whether a glare condition, outside of the ergonomic use range, exists.