SYSTEM AND METHODS OF MONITORING DRIVER BEHAVIOR FOR VEHICULAR FLEET MANAGEMENT IN A FLEET OF VEHICLES USING DRIVER-FACING IMAGING DEVICE

• US 20190147262A1
• Filed: 11/11/2017
• Published: 05/16/2019
• Est. Priority Date: 11/11/2017
• Status: Active Grant

• Current Assignee: Bendix Commercial Vehicle Systems LLC (Estate of Heinz Hermann Thiele)
• Original Assignee: Bendix Commercial Vehicle Systems LLC (Estate of Heinz Hermann Thiele)
• Inventors: Kuehnle, Andreas U., Li, Zheng
• Granted Patent: US 10,339,401 B2
• Time in Patent Office: Days
• CPC Class Codes: B60W 40/09 Driving style or behaviour
• CPC Class Codes: G06Q 10/00 Administration; Management
• CPC Class Codes: G06Q 10/06398 Performance of employee with respect to a job function
• CPC Class Codes: G06Q 50/40 Business processes related to the transportation industry shipping G06Q10/083
• CPC Class Codes: G06V 20/593 Recognising seat occupancy
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G07C 5/008 communicating information to a remotely located station transmission systems for measured values G08C

Abstract

Systems and methods monitor driver behavior for vehicular fleet management in a fleet of vehicles using driver-facing imaging device. The systems and methods herein relate generally to vehicular fleet management for enhancing safety of the fleet and improving the performance of the fleet drivers, and further relate to monitoring the operation of fleet vehicles using one or more driver-facing imaging devices disposed in the fleet vehicles for recording activities of the fleet drivers and their passengers, storing information relating to the monitored activities, selectively generating warnings related to the monitored activities, and reporting the monitored activities to a central fleet management system for use in enhancing the safety of the vehicles of the fleet and for helping to improve the performance of the fleet drivers.

Claims

1. A safety system monitoring a seatbelt worn condition of an associated vehicle during operation of the associated vehicle by an associated driver, the system comprising:

an imaging device disposed in the associated vehicle, the imaging device capturing an image of the associated driver in the associated vehicle and generating image data representative of the captured image of the associated driver; and

a control device comprising;

a processor;

an image data input operatively coupled with the processor, the image data input receiving the image data from the imaging device;

a non-transient memory device operatively coupled with the processor, the non-transient memory device storing safe model data comprising a recommended value range of a seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle; and

control logic stored in the non-transient memory device, the control logic being executable by the processor to;

process the image data to determine an operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle by;

inspecting the image data to locate a head of a human; and

inspecting the image data to determine one or more pairs of discontinuous collinear seatbelt lines below the located head of the human;

perform a comparison between the recommended value range of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle and the operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle; and

determine a state of vehicle operation compliance as a one of;

a seatbelt non-compliance state in accordance with a first result of the comparison between the recommended value range of the seatbelt use parameter and the operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle, wherein the control logic is executable by the processor to generate seatbelt use non-compliance data in accordance with the first result, or

a seatbelt use compliance state in accordance with a second result of the comparison between the recommended value range of the seatbelt use parameter and the operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle; and

an output operatively coupled with the processor, the output selectively receiving the seatbelt use non-compliance data from the processor and generating a seatbelt use non-compliance signal representative of the operational value of the seatbelt use parameter of the monitored seatbelt worn condition being outside of the recommended value range of the safe model data.

2. The safety system according to claim 1, wherein:

the control logic is executable by the processor to generate seatbelt use compliance data in a accordance with the second result; and

the output is operable to selectively receives the seatbelt use compliance data from the processor and generate a seatbelt use compliance signal representative of the operational value of the seatbelt use parameter of the monitored seatbelt worn condition being within the recommended value range of the safe model data.

4. The safety system according to claim 1, wherein:

the non-transient memory device stores a plurality of calibration images of a plurality of drivers wearing a seatbelt, each of the plurality of calibration images having an origin point relative to an image of an interior of the associated vehicle as the safe model data comprising the recommended value range of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle; and

the control logic stored in the non-transient memory device is executable by the processor to determine a particular first driver of the associated vehicle and to process the image data to determine, based on a calibration image corresponding to the determined particular first driver having the origin point, a disposition of a seatbelt in the image data as the operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle.

5. The safety system according to claim 1, wherein:

the non-transient memory device stores a generic model of a physical appearance of a buckled seatbelt as the safe model data comprising the recommended value range of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle; and

the control logic stored in the non-transient memory device is executable by the processor to process the image data to determine, based on the generic model of the physical appearance of a buckled seatbelt, a disposition of a seatbelt in the image data as the operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle.

6. The safety system according to claim 1, wherein:

the non-transient memory device stores one or more of a calibration image and/or a generic model of a position relative to the imaging device of a buckle of an unbuckled seatbelt as the safe model data comprising the recommended value range of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle; and

the control logic stored in the non-transient memory device is executable by the processor to process the image data to determine, based on the position relative to the imaging device of the buckle of the unbuckled seatbelt of the one or more of the calibration image and/or the generic model, a disposition of a seatbelt buckle in the image data as the operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle.

8. The safety system according to claim 1, wherein:

the control logic stored in the non-transient memory device is executable by the processor to process the image data to determine the operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle by;

inspecting the image data to locate a person;

retrieving from the memory a seatbelt position model; and

comparing the retrieved seatbelt model with a position of the person located in the image data relative to a preferred line orientation of an image of the seatbelt.

3. (canceled)

7. (canceled)

9-19. -19. (canceled)

20. A safety system monitoring a seatbelt worn condition of an associated vehicle during operation of the associated vehicle by an associated driver, the system comprising:

an imaging device disposed in the associated vehicle, the imaging device capturing an image of the associated driver in the associated vehicle and generating image data representative of the captured image of the associated driver; and

a control device comprising;

a processor;

an image data input operatively coupled with the processor, the image data input receiving the image data from the imaging device;

a non-transient memory device operatively coupled with the processor, the non-transient memory device storing a generic model of a physical appearance of a seatbelt buckled without the associated driver as safe model data comprising an outside of recommended value range of a seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle; and

control logic stored in the non-transient memory device, the control logic being executable by the processor to;

process the image data to determine, based on the generic model of the physical appearance of the seatbelt buckled without the associated driver, a disposition of a seatbelt in the image data as an operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle;

perform a comparison between the recommended value range of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle and the operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle; and

determine a state of vehicle operation compliance as a one of;

a seatbelt non-compliance state in accordance with a first result of the comparison between the recommended value range of the seatbelt use parameter and the operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle, wherein the control logic is executable by the processor to generate seatbelt use non-compliance data in accordance with the first result, or

a seatbelt use compliance state in accordance with a second result of the comparison between the recommended value range of the seatbelt use parameter and the operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle; and

an output operatively coupled with the processor, the output selectively receiving the seatbelt use non-compliance data from the processor and generating a seatbelt use non-compliance signal representative of the operational value of the seatbelt use parameter of the monitored seatbelt worn condition being outside of the recommended value range of the safe model data.

21. A safety system monitoring a seatbelt worn condition of an associated vehicle during operation of the associated vehicle by an associated driver, the system comprising:

an imaging device disposed in the associated vehicle, the imaging device capturing an image of the associated driver in the associated vehicle and generating image data representative of the captured image of the associated driver; and

a control device comprising;

a processor;

an image data input operatively coupled with the processor, the image data input receiving the image data from the imaging device;

a non-transient memory device operatively coupled with the processor, the non-transient memory device storing a generic model of a position relative to the imaging device of a buckle of an unbuckled seatbelt as safe model data comprising a recommended value range of a seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle;

control logic stored in the non-transient memory device, the control logic being executable by the processor to;

process the image data to determine a visibility or invisibility in the image data of a seatbelt on the associated driver in contrast to clothing on the associated driver;

process the image data to determine, based on;

i) the visibility or invisibility in the image data of the seatbelt on the associated driver; and

ii) the position relative to the imaging device of the buckle of the unbuckled seatbelt of the generic model, a disposition of a seatbelt buckle in the image data as an operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle;

perform a comparison between the recommended value range of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle and the operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle; and

determine a state of vehicle operation compliance as a one of;

a seatbelt non-compliance state in accordance with a first result of the comparison between the recommended value range of the seatbelt use parameter and the operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle, wherein the control logic is executable by the processor to generate seatbelt use non-compliance data in accordance with the first result, or

a seatbelt use compliance state in accordance with a second result of the comparison between the recommended value range of the seatbelt use parameter and the operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle; and

an output operatively coupled with the processor, the output selectively receiving the seatbelt use non-compliance data from the processor and generating a seatbelt use non-compliance signal representative of the operational value of the seatbelt use parameter of the monitored seatbelt worn condition being outside of the recommended value range of the safe model data.

22. A safety system monitoring a seatbelt worn condition of an associated vehicle during operation of the associated vehicle by an associated driver, the system comprising:

an imaging device disposed in the associated vehicle, the imaging device capturing an image of the associated driver in the associated vehicle and generating image data representative of the captured image of the associated driver; and

a control device comprising;

a processor;

an image data input operatively coupled with the processor, the image data input receiving the image data from the imaging device;

a non-transient memory device operatively coupled with the processor, the non-transient memory device storing safe model data comprising a recommended value range of a seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle; and

control logic stored in the non-transient memory device, the control logic being executable by the processor to;

process the image data to determine an operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle by;

inspecting the image data to locate a head of a human;

inspecting the image data to determine a first pair of parallel discontinuous collinear seatbelt lines below the located head of the human; and

inspecting the image data to determine a first pair of parallel continuous seatbelt lines located in a discontinuity defined by the first pair of parallel discontinuous collinear seatbelt lines;

perform a comparison between the recommended value range of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle and the operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle; and

determine a state of vehicle operation compliance as a one of;

a seatbelt non-compliance state in accordance with a first result of the comparison between the recommended value range of the seatbelt use parameter and the operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle, wherein the control logic is executable by the processor to generate seatbelt use non-compliance data in accordance with the first result, or

a seatbelt use compliance state in accordance with a second result of the comparison between the recommended value range of the seatbelt use parameter and the operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle; and

an output operatively coupled with the processor, the output selectively receiving the seatbelt use non-compliance data from the processor and generating a seatbelt use non-compliance signal representative of the operational value of the seatbelt use parameter of the monitored seatbelt worn condition being outside of the recommended value range of the safe model data.

23. A method of monitoring a seatbelt worn condition of an associated vehicle during operation of the associated vehicle by an associated driver, the method comprising:

capturing an image of the associated driver in the associated vehicle by an imaging device disposed in the associated vehicle, and generating by the imaging device image data representative of the captured image of the associated driver;

receiving the generated image data at an image data input of a control device including a processor, a non-transient memory device operatively coupled with the processor and storing safe model data in the non-transient memory device, the safe model data comprising a recommended value range of a seatbelt use parameter of the monitored seatbelt worn condition, and control logic stored in the non-transient memory device;

processing the image data by the processor executing the control logic to determine an operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle by;

inspecting the image data to locate a head of a human; and

inspecting the image data to determine one or more pairs of discontinuous collinear seatbelt lines below the located head of the human;

performing by the processor executing the control logic a comparison between the recommended value range of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle and the operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle;

determining by the processor executing the control logic a state of vehicle operation compliance as a one of;

a seatbelt non-compliance state in accordance with a first result of the comparison between the recommended value range and the operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle, wherein the processor generates seatbelt use non-compliance data in accordance with the first result, or

a seatbelt compliance state in accordance with a second result of the comparison between the recommended value range and the operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle; and

selectively generating, by the processor executing the control logic, at an output operatively coupled with the processor a seatbelt use non-compliance signal based on the seatbelt use non-compliance data, the compliance seatbelt use non-compliance signal being representative of the operational value of the seatbelt use parameter of the monitored seatbelt worn condition being outside of the recommended range of the safe model data.

24. The method according to claim 23, further comprising:

selectively generating, by the processor executing the control logic, seatbelt use compliance data in accordance with the second result of the comparison between the recommended value range and the operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle; and

selectively generating, by the processor executing the control logic, at the output a seatbelt use compliance signal representative of the operational value of the seatbelt use parameter of the monitored seatbelt worn condition being outside of the recommended range of the safe model data.

26. The method according to claim 23, further comprising:

storing in the non-transient memory device a plurality of calibration images of a plurality of drivers wearing a seatbelt, each of the plurality of calibration images having an origin point relative to an image of an interior of the associated vehicle as the safe model data comprising the recommended value range of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle; and

executing the control logic by the processor to determine a particular first driver of the associated vehicle and to process the image data to determine, based on a calibration image corresponding to the determined particular first driver having the origin point, a disposition of a seatbelt in the image data as the operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle.

27. The method according to claim 23, further comprising:

storing in the non-transient memory device a generic model of a physical appearance of a buckled seatbelt as the safe model data comprising the recommended value range of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle; and

executing the control logic by the processor to process the image data to determine, based on the generic model of the physical appearance of a buckled seatbelt, a disposition of a seatbelt in the image data as the operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle.

28. The method according to claim 23, further comprising:

storing in the non-transient memory device one or more of a calibration image and/or a generic model of a position relative to the imaging device of a buckle of an unbuckled seatbelt as the safe model data comprising the recommended value range of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle; and

executing the control logic by the processor to process the image data to determine, based on the position relative to the imaging device of the buckle of the unbuckled seatbelt of the one or more of the calibration image and/or the generic model, a disposition of a seatbelt buckle in the image data as the operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle.

30. The method according to claim 23, further comprising:

executing the control logic by the processor to process the image data to determine the operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle by;

inspecting the image data to locate a person;

retrieving from the memory a seatbelt position model; and

comparing the retrieved seatbelt model with a position of the person located in the image data relative to a preferred line orientation of an image of the seatbelt.

25. (canceled)

29. (canceled)

31. A method of monitoring a seatbelt worn condition of an associated vehicle during operation of the associated vehicle by an associated driver, the method comprising:

capturing by an imaging device in the associated vehicle an image of the associated driver in the associated vehicle, and generating image data representative of the captured image of the associated driver;

receiving the generated image data at an image data input of a control device including a processor, a non-transient memory device operatively coupled with the processor and storing in the non-transient memory device a generic model of a physical appearance of a seatbelt buckled without driver as safe model data comprising an outside of recommended value range of a seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle;

executing the control logic by the processor to process the image data to determine, based on the generic model of the physical appearance of the seatbelt buckled without the driver, a disposition of a seatbelt in the image data as an operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle;

performing by the processor executing the control logic a comparison between the recommended value range of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle and the operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle; and

determining by the processor executing the control logic a state of vehicle operation compliance as a one of;

a seatbelt non-compliance state in accordance with a first result of the comparison between the recommended value range and the operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle, wherein the processor generates seatbelt use non-compliance data in accordance with the first result, or

a seatbelt compliance state in accordance with a second result of the comparison between the recommended value range and the operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle; and

receiving the seatbelt use non-compliance data from the processor at an output operatively coupled with the processor, and selectively generating at the output a seatbelt use non-compliance signal representative of the operational value of the seatbelt use parameter of the monitored seatbelt worn condition being outside of the recommended value range of the safe model data.

32. A method of monitoring a seatbelt worn condition of an associated vehicle during operation of the associated vehicle by an associated driver, the method comprising:

capturing by an imaging device in the associated vehicle an image of the associated driver in the associated vehicle, and generating image data representative of the captured image of the associated driver;

receiving the generated image data at an image data input of a control device including a processor, a non-transient memory device operatively coupled with the processor and storing in the non-transient memory device a generic model of a position relative to the imaging device of a buckle of an unbuckled seatbelt as safe model data comprising a recommended value range of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle;

executing the control logic by the processor to process the image data to determine a visibility or invisibility in the image data of a seatbelt on the associated driver in contrast to clothing on the associated driver;

executing the control logic by the processor to process the image data to determine, based on;

i) the visibility or invisibility in the image data of the seatbelt on the associated driver; and

ii) the position relative to the imaging device of the buckle of the unbuckled seatbelt of the generic model, a disposition of a seatbelt buckle in the image data as an operational value of a seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle;

performing by the processor executing the control logic a comparison between the recommended value range of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle and the operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle; and

determining by the processor executing the control logic a state of vehicle operation compliance as a one of;

a seatbelt non-compliance state in accordance with a first result of the comparison between the recommended value range and the operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle, wherein the processor generates seatbelt use non-compliance data in accordance with the first result, or

a seatbelt compliance state in accordance with a second result of the comparison between the recommended value range and the operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle; and

receiving the seatbelt use non-compliance data from the processor at an output operatively coupled with the processor, and selectively generating at the output a seatbelt use non-compliance signal representative of the operational value of the seatbelt use parameter of the monitored seatbelt worn condition being outside of the recommended value range of the safe model data.

33. A method of monitoring a seatbelt worn condition of an associated vehicle during operation of the associated vehicle by an associated driver, the method comprising:

capturing by an imaging device in the associated vehicle an image of the associated driver in the associated vehicle, and generating image data representative of the captured image of the associated driver;

receiving the generated image data at an image data input of a control device including a processor, a non-transient memory device operatively coupled with the processor and storing in the non-transient memory device a generic model of a physical appearance of a seatbelt buckled without driver as safe model data comprising an outside of recommended value range of a seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle;

executing the control logic stored in the non-transient memory device by the processor to process the image data to determine the operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle by;

inspecting the image data to locate a head of a human;

inspecting the image data to determine a first pair of parallel discontinuous collinear seatbelt lines below the located head of the human; and

inspecting the image data to determine a first pair of parallel continuous seatbelt lines located in a discontinuity defined by the first pair of parallel discontinuous collinear seatbelt lines;

performing by the processor executing the control logic a comparison between the recommended value range of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle and the operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle; and

determining by the processor executing the control logic a state of vehicle operation compliance as a one of;

a seatbelt non-compliance state in accordance with a first result of the comparison between the recommended value range and the operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle, wherein the processor generates seatbelt use non-compliance data in accordance with the first result, or

a seatbelt compliance state in accordance with a second result of the comparison between the recommended value range and the operational value of the seatbelt use parameter of the monitored seatbelt worn condition of the associated vehicle; and

receiving the seatbelt use non-compliance data from the processor at an output operatively coupled with the processor, and selectively generating at the output a seatbelt use non-compliance signal representative of the operational value of the seatbelt use parameter of the monitored seatbelt worn condition being outside of the recommended value range of the safe model data.