Occupant monitoring systems and methods

• US 9,533,687 B2
• Filed: 12/30/2014
• Issued: 01/03/2017
• Est. Priority Date: 12/30/2014
• Status: Active Grant

• Current Assignee: Joyson Safety Systems Acquisition, LLC (Ningbo Junsheng Investment Group Co., Ltd.)
• Original Assignee: TK Holdings Incorporated (Ningbo Junsheng Investment Group Co., Ltd.)
• Inventors: Lisseman, Jason Carl, Cech, Len, Gardner, Valerie
• Primary Examiner(s): ARTHUR JEANGLAUD, GERTRUDE
• Application Number:
  US14/586,124
• Publication Number:
  US 20160185354A1
• Time in Patent Office: 735 Days
• Field of Search: 701/36, 701/45, 180/268, 180/272, 180/273, 280/735
• US Class Current: 1/1
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60R 2011/001 Vehicle control means, e.g. steering-wheel or column
• CPC Class Codes: B60W 10/20 including control of steering systems
• CPC Class Codes: B60W 2040/0818 Inactivity or incapacity of driver
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2422/50 on a steering column
• CPC Class Codes: B60W 2510/20 Steering systems
• CPC Class Codes: B60W 2710/20 Steering systems
• CPC Class Codes: B60W 40/09 Driving style or behaviour
• CPC Class Codes: B62D 1/046 Adaptations on rotatable parts of the steering wheel for accommodation of switches
• CPC Class Codes: F21S 45/47 Passive cooling, e.g. using fins, thermal conductive elements or openings
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G06V 20/593 Recognising seat occupancy
• CPC Class Codes: H04N 23/20 for generating image signals from infrared radiation only
• CPC Class Codes: H04N 23/51 Housings
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

Various implementations relate to an operator monitoring system (OMS). Certain implementations include an OMS coupled to a rotatable portion of a steering wheel assembly of a vehicle. For example, the OMS may include an imaging unit, such as a camera, that is coupled to a central hub portion of the steering wheel assembly. The imaging unit has a field of view directed toward one or more occupants in the vehicle and is configured to capture an image signal corresponding to an imaging area in the field of view. The imaging area can be configured to encapsulate an expected position of one or more vehicle occupants. The OMS also includes one or more processing units in electrical communication with the imaging unit that receives and processes the image signal from the imaging unit to determine an occupant state and, in some implementations, provide feedback based on the determined occupant state.

Claims

1. An occupant monitoring system (OMS) for monitoring at least one occupant in a vehicle, the vehicle having a steering wheel assembly configured to rotate relative to a steering column, the OMS comprising:

an imaging unit configured to be coupled to a steering wheel assembly, the imaging unit having a field of view inside of the vehicle and being configured to capture an image signal corresponding to an imaging area in the vehicle, the imaging area being within the field of view,

a steering angle sensor, and

a processing unit, the processing unit including a processor and a memory operably coupled to the processor, the memory having computer-executable instructions stored thereon that, when executed by the processor, cause the processor to;

receive an electrical signal from the steering angle sensor and the image signal from the imaging unit;

process the electrical signal to determine the angle of rotation of the steering wheel assembly and/or the rate of angular rotation of the steering wheel assembly; and

adjust the image signal captured of the imaging area to correct for the angle of rotation of the steering wheel assembly and/or the rate of angular rotation of the steering wheel assembly,

wherein the imaging area encapsulates an expected position of the occupant in the vehicle, and

wherein the imaging unit is configured to rotate with a central portion of the steering wheel assembly of the vehicle.

2. The OMS of claim 1, wherein the imaging unit is coupled to the central portion of the steering wheel assembly.

3. The OMS of claim 2, wherein the steering wheel assembly includes a rim and a hub, the central portion including the hub.

4. The OMS of claim 3, wherein the hub includes a housing, and the imaging unit is disposed within the housing.

5. The OMS of claim 4, wherein the housing is coupled adjacent to a backcover of the hub.

6. The OMS of claim 5, wherein the housing is coupled adjacent to the backcover via a mounting bracket and the system further comprises at least one light source disposed adjacent the imaging unit in the housing, and wherein at least a portion of the heat generated by one or more of the light source or the imaging unit is transferred from the housing to the hub.

7. The OMS of claim 4, wherein the housing is integrally formed with a backcover of the hub.

8. The OMS of claim 1, wherein the computer-executable instructions further cause the processor to:

adjust the imaging unit to correct for the angle of rotation of the steering wheel assembly and/or the rate of angular rotation of the steering wheel assembly.

9. The OMS of claim 1, wherein the OMS comprises a single modular unit configured to be coupled to the steering wheel assembly.

10. The OMS of claim 1, wherein the imaging unit includes a camera.

11. The OMS of claim 1, further including a light source configured to illuminate the imaging area.

12. The OMS of claim 11, wherein the light source comprises an infrared light source.

13. The OMS of claim 11, wherein the light source comprises at least one light emitting diode.

14. The OMS of claim 11, wherein the light source comprises a light bar, the light bar comprising one or more light emitting diodes.

15. The OMS of claim 14, wherein the light bar comprises at least a first section of one or more light emitting diodes and a second section of one or more light emitting diodes, wherein the first section is configured to illuminate visible light in response to the OMS being in one of an operational mode or a non-operational mode, and the second section is configured to illuminate infrared light in the direction of the field of view of the imaging unit.

16. The OMS of claim 1, wherein the computer-executable instructions further cause the processor to:

process the adjusted image signal to identify an analysis region;

analyze the analysis region to determine an occupant information parameter;

analyze the occupant information parameter to determine an occupant state; and

output a signal based on the determined occupant state.

17. The OMS of claim 16, wherein identifying the analysis region comprises determining the position and/or orientation of the occupant'"'"'s head and/or eyes.

18. The OMS of claim 16, wherein analyzing the analysis region comprises determining deviation in the position and/or orientation of the occupant'"'"'s head and/or eyes from a predetermined position.

19. The OMS of claim 16, wherein the analyzing the occupant information parameter to determine the occupant state comprises determining that the position and/or orientation of the occupant'"'"'s head and/or eyes are deviated from a predetermined position for at least a predetermined amount of time.

20. The occupant monitoring system of claim 1, further comprising a disabling unit configured to temporarily disable the occupant monitoring system when a driver signal is received and the vehicle is stationary.

21. The occupant monitoring system of claim 20, wherein the driver signal is output upon rotation of the steering wheel assembly of the vehicle by at least a predetermined amount and/or for a predetermined amount of time.

22. The occupant monitoring system of claim 21, wherein the driver signal ceases to be output when the steering wheel assembly is returned to a substantially central position for at least a predetermined amount of time.

23. A occupant monitoring system (OMS) for monitoring an occupant in a vehicle comprising:

a steering wheel assembly configured to be rotatably coupled to a steering column of the vehicle, the steering wheel assembly including a rim surrounding a centrally located hub;

an imaging unit coupled to the steering wheel assembly, the imaging unit having a field of view and being configured to capture an image of an imaging area within the field of view;

wherein the imaging area encapsulates an expected position of the occupant,

wherein the imaging unit is configured to be coupled to and rotate with a central portion of the steering wheel assembly of the vehicle, and

wherein the hub includes a housing directly coupled to a backcover of the hub, and the imaging unit is disposed within the housing.

24. The OMS of claim 23, further comprising a steering angle sensor.

25. The OMS of claim 24, further comprising a processing unit, the processing unit including a processor and a memory operably coupled to the processor, the memory having computer-executable instructions stored thereon that, when executed by the processor, cause the processor to:

receive an electrical signal from the steering angle sensor;

process the electrical signal to determine the angle of rotation of the steering wheel assembly and/or the rate of angular rotation of the steering wheel assembly; and

adjust an image signal corresponding to the image captured of the imaging area by the imaging unit to correct for the angle of rotation of the steering wheel assembly and/or the rate of angular rotation of the steering wheel assembly.

26. A method of monitoring an occupant in a vehicle using an occupant monitoring system, the method comprising:

coupling an occupant monitoring system (OMS) to a rotatable portion of a steering wheel assembly of the vehicle such that the OMS rotates with the rotatable portion, the OMS including an imaging unit configured to capture an image signal corresponding to an imaging area in the vehicle, the imaging area being within the field of view of the imaging unit;

processing an electrical signal from a steering angle sensor to determine the angle of rotation of the steering wheel assembly and/or the rate of angular rotation of the steering wheel assembly;

adjusting the image signal captured of the imaging area to correct for the angle of rotation of the steering wheel assembly and/or the rate of angular rotation of the steering wheel assembly;

processing the adjusted image signal to identify an analysis region;

analyzing the analysis region to determine an occupant information parameter;

analyzing the occupant information parameter to determine an occupant state; and

outputting a signal based on the determined occupant state.

27. The method of claim 26, wherein the rotatable portion of the steering wheel assembly of the vehicle is configured to rotate relative to a steering column.

28. The method of claim 26, wherein the imaging area encapsulates an expected position of the driver.

29. The method of claim 26, wherein the OMS further includes at least one processing unit, and one or more of the processing, analyzing, and outputting steps are performed by the processing unit.

30. The method of claim 26, wherein the vehicle includes at least one processing unit, and one or more of the processing, analyzing, and outputting steps are performed by the processing unit.