WARNING A VEHICLE OPERATOR OF UNSAFE OPERATION BEHAVIOR BASED ON A 3D CAPTURED IMAGE STREAM

• US 20080169914A1
• Filed: 01/12/2007
• Published: 07/17/2008
• Est. Priority Date: 01/12/2007
• Status: Active Grant

• Current Assignee: LinkedIn Corporation (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Goldman, Steven D., Sessa, Anthony J., Arnold, Kenneth C., Paolini, Michael A., ALBERTSON, JACOB C
• Granted Patent: US 7,792,328 B2
• Time in Patent Office: Days
• US Class Current: 340/438
• CPC Class Codes: B60W 40/09 Driving style or behaviour
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: G08B 21/06 indicating a condition of sleep, e.g. anti-dozing alarms

Abstract

A computer-implemented method, system, and program product includes a behavior processing system for capturing a three-dimensional movement of an operator of a vehicle, wherein the three-dimensional movement is determined by using at least one image capture device aimed at the operator, and identifying at least one behavior of the operator from the captured three-dimensional movement. An operator monitoring system for a vehicle determines whether the predicted behavior and the current operational state of the vehicle indicate unsafe operation of the vehicle and the operator monitoring system notifies the operator when unsafe operator of the vehicle is indicated by the operator'"'"'s behavior for the current operational state of the vehicle.

Claims

1. A computer-implemented method for automated detection of unsafe operation of a vehicle and automated response to detected unsafe operator of a vehicle, comprising:

capturing a three-dimensional movement of an operator of a vehicle, wherein the three-dimensional movement is determined using at least one image capture device aimed at the operator;

identifying a three-dimensional object properties stream using the captured movement;

identifying a particular defined behavior of the operator represented by the three-dimensional object properties stream by comparing the identified three-dimensional object properties stream with a plurality of behavior definitions;

identifying a current operational state of the vehicle;

identifying whether the particular defined behavior and the current operational state of the vehicle match at least one unsafe operation definition from among a plurality of unsafe operation definitions; and

in response to matching the at least one unsafe operation definition, notifying the operator of the type of unsafe operation currently performed based on the at least one unsafe operation definition.

2. The method according to claim 1, wherein capturing a three-dimensional movement of the operator of a vehicle further comprises capturing the three-dimensional movement using a stereoscopic image device to identify and track a particular three-dimensional movement of the operator.

3. The method according to claim 1, wherein identifying a particular defined behavior of the operator represented by the three-dimensional object properties stream further comprises:

identifying whether the three-dimensional object properties stream maps to a behavior of the operator, wherein the three-dimensional object properties stream is enabled to comprise images of the operator of the vehicle, images of items carried within the vehicle, images of at least one passenger within the vehicle, images from outside the vehicle, and images within the interior of the vehicle; and

responsive to identifying the three-dimensional object properties stream maps to the behavior of the operator, identifying the particular defined behavior of the operator represented by the three-dimensional object properties stream.

4. The method according to claim 1, wherein identifying a particular defined behavior of the operator representing the three-dimensional object properties stream further comprises calculating a percentage probability that the captured three-dimensional movement represents a particular behavior defined in the particular behavior definition.

5. The method according to claim 1, wherein identifying a particular defined behavior of the operator representing the three-dimensional object properties stream further comprises identifying at least one of body movement of the operator comprising a fine granularity movement of at least one of a facial expression, an eye movement, a muscle contraction, and change in skin surface characteristics of the operator within the three-dimensional object properties stream matching at least one of the plurality of behavior definitions.

6. The method according to claim 1, wherein identifying whether the particular defined behavior and the current operational state of the vehicle match at least one unsafe operation definition from among a plurality of unsafe operation definitions, further comprises identifying whether a 3D position of the head and the eyes of the operator during the current operational state of the vehicle matches the unsafe 3D position of the head and eyes for the current operational state defined in the at least one unsafe operation definition.

7. The method according to claim 1, wherein identifying whether the particular defined behavior and the current operational state of the vehicle match at least one unsafe operation definition from among a plurality of unsafe operation definitions, further comprises identifying whether the detected behavior of the operator talking on a portable telephone device during the current operational state of the vehicle matches the unsafe portable telephone use behavior for the current operational state defined in the at least one unsafe operation definition.

8. The method according to claim 1, wherein notifying the operator of the type of unsafe operation currently performed based on the at least one unsafe operation definition further comprises:

generating at least one control signal from among a tactile output control signal, an audio output control signal, and a graphical output control signal, wherein the at least one control signal controls output that notifies the operator of the type of unsafe operation; and

controlling output of the at least one control signal to a controller for controlling the output that notifies the operator the type of unsafe operation.

9. The method according to claim 1, further comprising:

recording the particular defined behavior in a storage system responsive to the current operational state specifying at least one from among an airbag deployment, an antilock brake system actuation, and detecting damage to at least one part of the vehicle.

10. A system for automated detection of unsafe operation of a vehicle and automated response to detected unsafe operator of a vehicle, comprising:

a behavior processing system communicatively connected to a network, further comprising;

means for capturing a three-dimensional movement of an operator of a vehicle, wherein the three-dimensional movement is determined using at least one image capture device aimed at the operator;

means for identifying a three-dimensional object properties stream using the captured movement;

means for identifying a particular defined behavior of the operator represented by the three-dimensional object properties stream by comparing the identified three-dimensional object properties stream with a plurality of behavior definitions;

an operator monitoring system communicatively connected to the behavior processing system via the network, further comprising;

means for identifying a current operational state of the vehicle;

means for identifying whether the particular defined behavior and the current operational state of the vehicle match at least one unsafe operation definition from among a plurality of unsafe operation definitions; and

means, in response to matching the at least one unsafe operation definition, for notifying the operator of the type of unsafe operation currently performed based on the at least one unsafe operation definition.

11. The system according to claim 10, wherein the means for capturing a three-dimensional movement of the operator of a vehicle further comprises means for capturing the three-dimensional movement using a stereoscopic image device to identify and track a particular three-dimensional movement of the operator.

12. The system according to claim 10, wherein the means for identifying a particular defined behavior of the operator represented by the three-dimensional object properties stream further comprises:

means for identifying whether the three-dimensional object properties stream maps to a behavior of the operator, wherein the three-dimensional object properties stream is enabled to comprise images of the operator of the vehicle, images of items carried within the vehicle, images of at least one passenger within the vehicle, images from outside the vehicle, and images within the interior of the vehicle; and

means, responsive to identifying the three-dimensional object properties stream maps to the behavior of the operator, for identifying the particular defined behavior of the operator represented by the three-dimensional object properties stream.

13. The system according to claim 10, wherein the means for identifying a particular defined behavior of the operator representing the three-dimensional object properties stream further comprises means for identifying at least one of body movement of the operator comprising a fine granularity movement of at least one of a facial expression, an eye movement, a muscle contraction, and change in skin surface characteristics of the operator within the three-dimensional object properties stream matching at least one of the plurality of behavior definitions.

14. The system according to claim 10, wherein the means for identifying whether the particular defined behavior and the current operational state of the vehicle match at least one unsafe operation definition from among a plurality of unsafe operation definitions, further comprises means for identifying whether a 3D position of the head and the eyes of the operator during the current operational state of the vehicle matches the unsafe 3D position of the head and eyes for the current operational state defined in the at least one unsafe operation definition.

15. The system according to claim 10, wherein the means for identifying whether the particular defined behavior and the current operational state of the vehicle match at least one unsafe operation definition from among a plurality of unsafe operation definitions, further comprises means for identifying whether the detected behavior of the operator talking on a portable telephone device during the current operational state of the vehicle matches the unsafe portable telephone use behavior for the current operational state defined in the at least one unsafe operation definition.

16. The system according to claim 10, wherein the means for notifying the operator of the type of unsafe operation currently performed based on the at least one unsafe operation definition further comprises:

means for generating at least one control signal from among a tactile output control signal, an audio output control signal, and a graphical output control signal, wherein the at least one control signal controls output that notifies the operator of the type of unsafe operation; and

means for controlling output of the at least one control signal to a controller for controlling the output that notifies the operator the type of unsafe operation.

17. The system according to claim 10, the operator monitoring system further comprising means for recording the particular defined behavior in a storage system responsive to the current operational state specifying at least one from among an airbag deployment, an antilock brake system actuation, and detecting damage to at least one part of the vehicle.

18. A program product comprising a storage-type computer-usable medium including a computer-readable program for automated detection of unsafe operation of a vehicle and automated response to detected unsafe operator of a vehicle, wherein the computer-readable program when executed on a computer causes the computer to:

capture a three-dimensional movement of an operator of a vehicle, wherein the three-dimensional movement is determined using at least one image capture device aimed at the operator;

identify a three-dimensional object properties stream using the captured movement;

identify a particular defined behavior of the operator represented by the three-dimensional object properties stream by comparing the identified three-dimensional object properties stream with a plurality of behavior definitions;

identify a current operational state of the vehicle;

identify whether the particular defined behavior and the current operational state of the vehicle match at least one unsafe operation definition from among a plurality of unsafe operation definitions; and

in response to matching the at least one unsafe operation definition, notify the operator of the type of unsafe operation currently performed based on the at least one unsafe operation definition.

19. The program product of claim 18, wherein said computer-readable program is transmitted over a network.

20. The program product of claim 18, wherein the computer-readable program when executed on a computer further causes the computer to capture the three-dimensional movement using a stereoscopic image device to identify and track a particular three-dimensional movement of the operator.