System and method for learning driving information in vehicle

• US 9,485,474 B2
• Filed: 03/13/2014
• Issued: 11/01/2016
• Est. Priority Date: 12/27/2013
• Status: Active Grant

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: Kim, Jinwoo, Han, Woo Yong
• Primary Examiner(s): Natnael, Paulos M
• Application Number:
  US14/208,276
• Publication Number:
  US 20150189241A1
• Time in Patent Office: 964 Days
• US Class Current: 1/1
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2540/225 Direction of gaze
• CPC Class Codes: B60W 30/00 Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
• CPC Class Codes: B60W 40/09 Driving style or behaviour
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30268 Vehicle interior
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G07C 5/02 Registering or indicating driving, working, idle, or waiting time only apparatus forming part of taximeters G07B13/00
• CPC Class Codes: G07C 5/0866 the electronic data carrier being a digital video recorder in combination with video camera
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A system for learning driving information in a vehicle, includes a first camera mounted on an inner ceiling of the vehicle and configured to detect a projected position of a nose of a driver; and a second camera mounted on an inner front of the vehicle and configured to detect a front position of the nose of the driver. Further, the system includes a driver'"'"'s status information processing unit configured to detect the line of sight of the driver by calibrating an image corresponding to the projected position and an image corresponding to the front position in real time, to analyze information about the line of sight that is detected, and to store the analyzed information in a learning data database.

Claims

1. A system for learning driving information in a vehicle, the system comprising:

a first camera mounted on an inner ceiling of the vehicle and configured to detect a projected position of a nose of the driver;

a second camera mounted on an inner front of the vehicle and configured to detect a front position of the nose of the driver;

a driver'"'"'s status information processing unit configured to

detect the line of sight of the driver by calibrating an image corresponding to the projected position and an image corresponding to the front position in real time,

analyze information about the line of sight that is detected, and

store the analyzed information in a driving learning database as learning data,

wherein the driver'"'"'s status information processing unit is further configured to

recognize driver information of the driver after temporally storing images captured by the first and second cameras,

determine whether the driver who gets in the vehicle is a pre-registered driver based on the driver information,

register the driver information of the driver who gets in the vehicle when the driver is the pre-registered driver, and

register the driver as a new driver when the driver is not the pre-registered driver.

2. The system of claim 1, wherein the first camera has a capability of rotation and movement.

3. The system of claim 1, wherein the first camera comprises a depth sensor.

4. The system of claim 1, wherein the driver'"'"'s status information processing unit updates the driving learning database.

5. The system of claim 1, further comprising:

a voice recognition unit configured to recognize a voice of the driver who gets in the vehicle and provide information about the recognized voice to the driver'"'"'s status information processing unit.

6. The system of claim 1, further comprising:

a camera controlling unit configured to calibrate the operations of the first camera and second camera in accordance with a processed result from the driver'"'"'s status information processing unit.

7. The system of claim 1, further comprising:

a user interface unit configured to input user interface information to the driver'"'"'s status information processing unit or receive a feedback on a processed result of a user interface from the driver'"'"'s status information processing unit.

8. The system of claim 7, wherein the user interface is based on a multi-modal interface.

9. The system of claim 1, wherein the learning data in the driving learning database is exchanged with another system to learn driving information in another vehicle.

10. A method for learning driving information in a vehicle through the use of images captured by first and second cameras, the method comprising:

recognizing a voice of a driver when the driver is detected in the vehicle;

recognizing a face image of the driver through the first and second cameras;

calculating a mutual location between the first and second cameras;

detecting an uttered position of the voice and a location of a face of the driver by moving and rotating each of the first and second cameras; and

building learning data by tracking a line of sight of the driver,

wherein said recognizing a face image comprises;

recognizing driver information of the driver after temporally storing the images captured by the first and second cameras;

determining whether the driver who gets in the vehicle is a pre-registered driver based on the driver information;

registering the driver information of the driver who gets in the vehicle when the driver is a pre-registered driver; and

registering the driver as a new driver when the driver is not a pre-registered driver.

11. The method of claim 10, further comprising:

detecting a projected position of a nose of the driver using the first camera, which is mounted on an inner ceiling of the vehicle.

12. The method of claim 10, further comprising:

detecting a front position of the nose of the driver using the second camera, which is mounted on an inner front of the vehicle.

13. The method of claim 10, wherein said building learning data comprises:

recognizing the line of sight of the driver in the vehicle to detect the driver information of the driver; and

storing learning data from the detected driver information.

14. The method of claim 13, wherein said building learning data comprises:

combining one or more of a voice instruction, a gesture instruction of the driver, or the line of sight of the driver in accordance with the detected driver information.

15. The method of claim 10, wherein said building learning data comprises:

updating the learning data in real time;

providing a feedback based on the learning data that is updated in real time from the learning data that is built previously.

16. A system for learning driving information in a vehicle, the system comprising:

a first camera mounted on an inner ceiling of the vehicle and including a depth sensor, the depth sensor being configured to capture first image information of a driver of the vehicle, the first image information comprising a depth map image of the driver'"'"'s head;

a second camera mounted on an inner front of the vehicle and configured to capture second image information of the driver;

a driver'"'"'s status information processing unit configured to

detect a projected position of a nose of the driver from the depth map image, the projected position of the nose of the driver being the most prominent feature in a forward-looking direction of the driver in the depth map image,

detect a front position of the nose of the driver from the second image information,

detect the line of sight of the driver by calibrating the first image information corresponding to the projected position of the nose of the driver and the second image information corresponding to the front position of the nose of the driver,

analyze information about the line of sight that is detected, and

store the analyzed information in a driving learning database as learning data.