SYSTEM AND METHOD FOR LEARNING DRIVING INFORMATION IN VEHICLE

• US 20150189241A1
• Filed: 03/13/2014
• Published: 07/02/2015
• Est. Priority Date: 12/27/2013
• Status: Active Grant

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: KIM, Jinwoo, HAN, Woo Yong
• Granted Patent: US 9,485,474 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/42 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2540/225 Direction of gaze
• CPC Class Codes: B60W 30/00 Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
• CPC Class Codes: B60W 40/09 Driving style or behaviour
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30268 Vehicle interior
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G07C 5/02 Registering or indicating driving, working, idle, or waiting time only apparatus forming part of taximeters G07B13/00
• CPC Class Codes: G07C 5/0866 the electronic data carrier being a digital video recorder in combination with video camera
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A system for learning driving information in a vehicle, includes a first camera mounted on an inner ceiling of the vehicle and configured to detect a projected position of a nose of a driver; and a second camera mounted on an inner front of the vehicle and configured to detect a front position of the nose of the driver. Further, the system includes a driver'"'"'s status information processing unit configured to detect the line of sight of the driver by calibrating an image corresponding to the projected position and an image corresponding to the front position in real time, to analyze information about the line of sight that is detected, and to store the analyzed information in a learning data database.

Claims

1. A system for learning driving information in a vehicle, the system comprising:

a first camera mounted on an inner ceiling of the vehicle and configured to detect a projected position of a nose of a driver;

a second camera mounted on an inner front of the vehicle and configured to detect a front position of the nose of the driver;

a driver'"'"'s status information processing unit configured to detect the line of sight of the driver by calibrating an image corresponding to the projected position and an image corresponding to the front position in real time, to analyze information about the line of sight that is detected, and to store the analyzed information in a driving learning database as a learning data.

2. The system of claim 1, wherein the first camera has a capability of rotation and movement.

3. The system of claim 1, wherein the first camera comprises a depth sensor.

4. The system of claim 1, wherein the driver'"'"'s status information processing unit updates the learning database.

5. The system of claim 1, further comprising:

a voice recognition unit configured to recognize a voice of the driver who gets in the vehicle and provide information about the recognized voice to the driver'"'"'s status information processing unit.

6. The system of claim 1, further comprising:

a camera controlling unit configured to calibrate the operations of the first camera and second camera in accordance with the processed result from the driver'"'"'s status information processing unit.

7. The system of claim 1, further comprising:

a user interface unit configured to input user interface information to the driver'"'"'s status information processing unit or receive a feedback on the processed result of a user interface from the driver'"'"'s status information processing unit.

8. The system of claim 7, wherein the user interface is based on a multi-modal interface.

9. The system of claim 1, wherein the learning data in the driving learning database is exchanged with another system for learning driving information in the other vehicle.

10. A method for learning driving information in a vehicle through the use of images captured by first and second cameras, the method comprising:

recognizing a voice of a driver when the driver is detected in the vehicle;

recognizing a face image of the driver through the first and second cameras;

calculating a mutual location between the first and second cameras;

detecting a uttered position of the voice and a location of a face of the driver by moving and rotating each of the first and second cameras; and

building learning data by tracking a line of sight of the driver.

11. The method of claim 10, wherein said recognizing a face image comprises:

recognizing information of the driver after temporally storing the images captured by the first and second cameras;

determining whether the driver who gets in the vehicle is a pre-registered driver based on the driver information;

registering the driver information of the driver who gets in the vehicle when the driver is a pre-registered driver; and

registering the driver as a new driver when the driver is not a pre-registered driver.

12. The method of claim 10, wherein the first camera is mounted on an inner ceiling of the vehicle and is configured to detect a projected position of a noise of the driver.

13. The method of claim 10, wherein the second camera is mounted on an inner front of the vehicle and is configured to detect a front position of the nose.

14. The method of claim 10, wherein said building learning data comprises:

recognizing the line of sight of the driver in the vehicle to detect driver information of the driver; and

storing learning data from the detected driver information.

15. The method of claim 14, wherein said building learning data comprises:

combining one or more a voice instruction, gesture instruction of the driver, or a line of sight of the driver in accordance with the detected driver information.

16. The method of claim 10, wherein said building learning data comprises:

updating the learning data in real time;

providing a feedback on learning data that is updated in real time to the learning data that is built previously.