PREDICTING VEHICLE MOVEMENTS BASED ON DRIVER BODY LANGUAGE

• US 20170124407A1
• Filed: 11/04/2015
• Published: 05/04/2017
• Est. Priority Date: 11/04/2015
• Status: Active Grant

• Current Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Original Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Inventors: Micks, Ashley Elizabeth, Banvait, Harpreetsingh, Jain, Jinesh J., Reiff, Brielle
• Granted Patent: US 9,864,918 B2
• Time in Patent Office: Days
• CPC Class Codes: B60W 40/08 related to drivers or passengers
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30268 Vehicle interior
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/19 Sensors therefor
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

Systems, methods, and devices for predicting driver intent and future movements of a human driven vehicles are disclosed herein. A system for predicting future movements of a vehicle includes a camera system, a boundary component, a body language component, and a prediction component. The camera system is configured to capture an image of a vehicle. The boundary component is configured to identify a sub-portion of the image corresponding to an area where a driver of a vehicle is located. The body language component configured to detect a driver'"'"'s body language. The prediction component configured to predict future motion of the vehicle based on the driver'"'"'s body language detected by the body language component.

Claims

1. A system comprising:

a camera system configured to capture an image of a vehicle;

a boundary component configured to identify a sub-portion of the image corresponding to an area where a driver of a vehicle is located;

a body language component configured to detect a driver'"'"'s body language; and

a prediction component configured to predict future motion of the vehicle based on the driver'"'"'s body language detected by the body language component.

2. The system of claim 1, wherein the body language component is configured to detect a driver'"'"'s body language by identifying one or more of a driver'"'"'s head orientation, a gaze direction, and a gesture.

3. The system of claim 1, wherein the body language component is configured to process image data in the sub-portion of the image to detect the driver'"'"'s body language.

4. The system of claim 1, wherein the boundary component is configured to locate the vehicle within the image.

5. The system of claim 1, wherein the boundary component is configured to identify the sub-portion of the image based one or more of:

identification of one or more windows; and

identification of a region of the vehicle where a driver would likely be located.

6. The system of claim 1, wherein the prediction component is configured to access a database or model that correlates the driver'"'"'s body language detected by the body language component with one or more future vehicle movements.

7. The system of claim 6, wherein the database or model correlates one or more of the following with the one or more future vehicle movements:

a waving motion of a hand;

a hand gesture comprising a palm facing toward the vehicle with fingers upward;

a gaze direction of the driver for a threshold period of time;

a series of head movements; and

a series of quick changes in gaze direction.

8. The system of claim 6, wherein the database or model correlates a gaze direction with a future driving direction.

9. The system of claim 6, wherein the database or model correlates a gesture with a current driving context, wherein the driving context comprises a stop at an intersection, an approach to an intersection, driving down a road with one or more nearby vehicles, merging onto a roadway, exiting a roadway, entering a parking lot or parking spot, or exiting a parking lot or parking spot.

10. A computer implemented method comprising:

receiving an image of a proximal vehicle in a region near a vehicle;

determining a region of the image that contains a driver of the proximal vehicle;

processing image data in the region of the image that contains the driver of the proximal vehicle to detect one or more of a head orientation, a gaze direction, and a gesture of the driver;

accessing a database or model that correlates one or more of the head orientation, the gaze direction, and the gesture with one or more future vehicle movements; and

determining a vehicle maneuver based on the one or more future vehicle movements of the proximal vehicle.

11. The method of claim 10, further comprising locating the vehicle within the image.

12. The method of claim 10, wherein determining the region of the image that contains the driver comprises determining the region based on one or more of:

a location of one or more windows of the vehicle; and

a predicted location of a driver'"'"'s seat in the vehicle.

13. The method of claim 10, wherein the database or model correlates one or more of the following with the one or more future vehicle movements:

a waving motion of a hand;

a hand gesture comprising a palm facing toward the vehicle with fingers upward;

a gaze direction of the driver for a threshold period of time;

a series of head movements; and

a series of changes in gaze direction.

14. Computer readable storage media storing instructions that, when executed by one or more processors, cause the processors to:

receive an image of a vehicle;

identify a boundary around a region of the image where a driver of a vehicle is located; and

predict future motion of the vehicle based on the driver'"'"'s body language, wherein the driver'"'"'s body language comprise one or more of a driver'"'"'s head orientation, a gaze direction, and a gesture.

15. The computer readable storage media of claim 14, wherein the instructions further cause the processor to detect one or more of the driver'"'"'s head orientation, the gaze direction, and the gesture.

16. The computer readable storage media of claim 15, wherein detecting one or more of the driver'"'"'s head orientation, the gaze direction, and the gesture comprises processing image data in the boundary to detect the head orientation, the gaze direction, and the gesture.

17. The computer readable storage media of claim 14, wherein the instructions further cause the processor to locate the vehicle within the image.

18. The computer readable storage media of claim 14, wherein identifying the boundary around the region comprises one or more of:

identifying of one or more windows of the vehicle; and

identifying of a region of the vehicle where a driver would likely be located.

19. The computer readable storage media of claim 14, wherein the instructions further cause the processor to access a database or model that correlates one or more of the driver'"'"'s head orientation, the gaze direction, and the gesture with one or more future vehicle movements

20. The computer readable storage media of claim 14, wherein the instructions further cause the processor determine a driving maneuver to be performed by a parent vehicle based on the predicted future motion.