Digital device and driver monitoring method thereof

• US 10,015,411 B2
• Filed: 05/05/2015
• Issued: 07/03/2018
• Est. Priority Date: 02/25/2015
• Status: Active Grant

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Kim, Kwanhyung, Kwon, Youngman
• Primary Examiner(s): Kelley, Christopher S
• Assistant Examiner(s): Kwan, Matthew
• Application Number:
  US14/704,553
• Publication Number:
  US 20160248957A1
• Time in Patent Office: 1,155 Days
• Field of Search: 348 77
• CPC Class Codes: B60K 2360/00 Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
• CPC Class Codes: B60K 28/02 responsive to conditions relating to the driver
• CPC Class Codes: G06V 10/141 Control of illumination
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G08B 21/06 indicating a condition of sleep, e.g. anti-dozing alarms
• CPC Class Codes: G08B 29/18 Prevention or correction of operating errors G08B29/02, G08B29/12 take precedence
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 23/70 Circuitry for compensating brightness variation in the scene
• CPC Class Codes: H04N 23/71 Circuitry for evaluating the brightness variation
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

A method of monitoring a driver including obtaining images from the first and second camera unit, checking whether a brightness of the image obtained from the first camera unit is equal to or greater than a reference value or whether the driver is detected from the image obtained from the second camera unit, if the brightness of the image obtained from the first camera unit is not equal to or greater than the reference value or the driver is not detected from the image obtained from the second camera unit, obtaining a first comparative image from the first camera unit by turning off the second illumination unit and a second comparative image from the first camera unit by turning on the second illumination unit, checking whether a difference value between a brightness of the obtained first comparative image and a brightness of the obtained second comparative image is equal to or greater than a reference difference value, and if the difference value between the brightness of the obtained f comparative image and the brightness of the obtained second comparative image is not equal to or greater than the reference difference value, turning off the second illumination unit.

Claims

1. A digital device comprising:

a first illumination unit configured to generate a first light;

a second illumination unit configured to generate a second light;

a first camera unit;

a second camera unit; and

a control unit configured to;

check whether a brightness of an image obtained from the first camera unit is equal to or greater than a reference value or whether a driver is detected from an image obtained from the second camera unit, and check whether a brightness of an image obtained from the second camera unit is equal to or greater than a reference value or whether the driver is detected from an image obtained from the first camera unit,

if the brightness of the image obtained from the first camera unit is not equal to or greater than the reference value or if the driver is not detected from the image obtained from the second camera unit, obtaining a first comparative image from the first camera unit by turning off the second illumination unit and obtaining a second comparative image from the first camera unit by turning on the second illumination unit, and if the brightness of the image obtained from the second camera unit is less than the reference value or if the driver is not detected from the image obtained from the first camera unit, obtaining a third comparative image from the second camera unit by turning off the first illumination unit and obtaining a fourth comparative image from the second camera unit by turning on the first illumination unit,

check whether a difference value between the brightness of the first comparative image and the brightness of the second comparative image is equal to or greater than a reference difference value, and check whether a difference value between the brightness of the third comparative image and the brightness of the fourth comparative image is equal to or greater than a reference difference value, and

if the difference value between the brightness of the obtained first comparative image and the brightness of the obtained second comparative image is not equal to or greater than the reference difference value, turning off the second illumination unit, and if the difference value between the brightness of the obtained third comparative image and the brightness of the obtained fourth comparative image is less than the reference difference value, turning off the first illumination unit.

2. The digital device of claim 1, wherein the first camera unit is disposed adjacent to the first illumination unit, and

wherein the second camera unit is disposed adjacent to the second illumination unit.

3. The digital device of claim 1, wherein the first illumination unit and the first camera are configured as a first module, and

wherein the second illumination unit and the second camera are configured as a second module.

4. The digital device of claim 1, wherein each of the first light generated from the first illumination unit and the second light generated from the second illumination unit comprises a light of an infrared wavelength range.

5. The digital device of claim 1, wherein an intensity of the first light generated from the first illumination unit is equal to that of the second light generated from the second illumination unit.

6. The digital device of claim 1, further comprising:

a storage unit configured to store the reference value for the image brightness and the reference difference value,

wherein the reference value and the reference difference value are preset.

7. A method of controlling a digital device for monitoring a driver, the digital device including a first illumination unit, a second illumination unit, a first camera unit and a second camera unit, the method comprising:

obtaining images from the first camera unit and the second camera unit;

checking, via a control unit of the digital device, whether a brightness of the image obtained from the first camera unit is equal to or greater than a reference value or whether the driver is detected from the image obtained from the second camera unit, and checking, via the control unit of the digital device, whether a brightness of the image obtained from the second camera unit is equal to or greater than a reference value or whether the driver is detected from the image obtained from the first camera unit;

if the brightness of the image obtained from the first camera unit is not equal to or greater than the reference value or the driver is not detected from the image obtained from the second camera unit, obtaining a first comparative image from the first camera unit by turning off the second illumination unit and a second comparative image from the first camera unit by turning on the second illumination unit, and if the brightness of the image obtained from the second camera unit is less than the reference value or the driver is not detected from the image obtained from the first camera unit, obtaining a third comparative image from the second camera unit by turning off the first illumination unit and a fourth comparative image from the second camera unit by turning on the first illumination unit;

checking whether a difference value between a brightness of the obtained first comparative image and a brightness of the obtained second comparative image is equal to or greater than a reference difference value, and checking whether a difference value between a brightness of the obtained third comparative image and a brightness of the obtained fourth comparative image is equal to or greater than a reference difference value; and

if the difference value between the brightness of the obtained first comparative image and the brightness of the obtained second comparative image is not equal to or greater than the reference difference value, turning off the second illumination unit, and if the difference value between the brightness of the obtained third comparative image and the brightness of the obtained fourth comparative image is less than the reference difference value, turning off the first illumination unit.

8. The method of claim 7, further comprising:

receiving a driver monitoring request signal; and

turning on the first illumination unit, the second illumination unit, the first camera unit and the second camera unit.

9. The method of claim 7, wherein the reference value for the image brightness is preset and saved.

10. The method of claim 9, wherein the reference value for the image brightness is determined depending on an internal vehicle environment of the driver.

11. The method of claim 7, wherein a timing point of turning off the second illumination unit and a timing point of turning on the second illumination unit have a predetermined time interval.

12. The method of claim 11, wherein the predetermined time interval comprises a time for obtaining the first comparative image from the first camera unit.

13. The method of claim 7, wherein the reference difference value is preset and saved.

14. The method of claim 7, further comprising:

checking whether both of the first illumination unit and the second illumination unit are turned off; and

if both of the first illumination unit and the second illumination unit are turned off, displaying a warning to the driver and turning on both of the first illumination unit and the second illumination unit.

15. The method of claim 14, wherein the warning is displayed by displaying a created warning message on a display unit or turning on/off a warning notification illumination continuously for a predetermined time.

16. The method of claim 14, further comprising:

turning on both of the first illumination unit and the second illumination unit after an elapse of a predetermined time from a timing point of displaying the warning to the driver.

17. The method of claim 14, further comprising:

if both of the first illumination unit and the second illumination unit are not turned off, obtaining the images from the first camera unit and the second camera unit;

checking whether the driver is detected from each of the images obtained from the first camera unit and the second camera unit; and

if the driver is detected from each of the images obtained from the first camera unit and the second camera unit, turning on the second illumination unit.

18. The method of claim 17, wherein if the driver is not detected from each of the images obtained from the first camera unit and the second camera unit, turning off the second illumination and repeatedly performing the steps following the step of turning off the second illumination.

19. The method of claim 7, wherein the checking whether the difference value is equal to or greater than the reference difference value, comprises:

if the difference value between the brightness of the obtained first comparative image and the brightness of the obtained second comparative image is equal to or greater than the reference difference value, checking whether both of the first illumination unit and the second illumination unit are turned off; and

if both of the first illumination unit and the second illumination unit are turned off, displaying a warning to the driver and turning on both of the first illumination unit and the second illumination unit.

20. The method of claim 7, wherein the checking whether the difference value is equal to or greater than the reference difference value comprises:

if the difference value between the brightness of the obtained first comparative image and the brightness of the obtained second comparative image is equal to or greater than the reference difference value, obtaining the images from the first camera unit and the second camera unit;

checking whether the driver is detected from each of the images obtained from the first camera unit and the second camera unit; and

if the driver is detected from each of the images obtained from the first camera unit and the second camera unit, turning on the second illumination unit.