Method and apparatus for controlling vehicle using motion recognition with face recognition

• US 9,576,189 B2
• Filed: 08/26/2015
• Issued: 02/21/2017
• Est. Priority Date: 08/27/2014
• Status: Active Grant

• Current Assignee: CHEMTRONICS Co., Ltd.
• Original Assignee: CHEMTRONICS Co., Ltd.
• Inventors: Lim, Yong-Je, Kim, See-Wook
• Primary Examiner(s): Shafi, Muhammad
• Application Number:
  US14/836,912
• Publication Number:
  US 20160063315A1
• Time in Patent Office: 545 Days
• Field of Search: 701/23, 701/27, 701/36, 701 40- 42, 701/45, 701/57, 701/58, 701/83, 701/84, 701/301, 701/302, 340/995.1, 340/995.13, 340/995.19, 340/995.21, 340/995.23
• US Class Current: 1/1
• CPC Class Codes: B60Q 1/04 the devices being headlights
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: H04N 5/44 Receiver circuitry for the reception of television signals according to analogue transmission standards H04N5/14 takes precedence

Abstract

Provided is a vehicle control apparatus including an image obtaining part that obtains an image of outside of a vehicle; a face recognizing part that finds and tracks a face of a person from an image obtained by the image obtaining part based on feature elements human faces and extracts feature information regarding a face of a person; a face authenticating part that compares the feature information extracted by the face recognizing part to feature information regarding faces of persons registered in advance and, if a difference therebetween is less than or equal to a critical value, authenticates that the recognized face as a registered face; the motion recognizing part that, when authentication is completed by the face authenticating part, recognizes a motion; and a control part that recognizes a motion of a face-recognized and face-authenticated person and performs a set function.

Claims

1. A vehicle control apparatus comprising:

an image obtaining part configured to obtain an image of an area outside a vehicle;

a face recognizing part configured to;

find and track a face of a person outside the vehicle, the face of the person outside the vehicle being in the image obtained by the image obtaining part, the face recognizing part being configured to find the face of the person outside the vehicle and track the face of the person outside the vehicle based on feature elements of human faces; and

extract feature information regarding the face of the person outside the vehicle;

a face authenticating part configured to;

compare the feature information extracted by the face recognizing part to feature information regarding faces of one or more persons registered in advance, the feature information regarding the faces of the one or more persons registered in advance being stored in a non-transitory computer readable storage medium; and

authenticate the face of the person outside the vehicle as being a registered face if a first difference between the feature information extracted by the face recognizing part and the feature information regarding the faces of the one or more persons registered in advance is less than or equal to a first critical value;

a motion recognizing part configured to recognize a motion of the person outside the vehicle if the face of the person outside the vehicle is authenticated as being a registered face; and

a control part configured to;

compare the motion recognized by the motion recognizing part to a motion registered in advance, the motion registered in advance being stored in the memory;

cause a specific function of the vehicle to be performed if a second difference between the motion recognized by the motion recognizing part and the motion stored in advance is greater than a second critical value, the specific function of the vehicle comprising one or more of opening a vehicle door, closing the vehicle door, opening a vehicle trunk, or closing the vehicle trunk; and

cause the motion recognizing part to request the person outside the vehicle having the registered face to show the motion again if the second difference is less than or equal to the second critical value.

2. The vehicle control apparatus of claim 1, wherein the image obtaining part is configured to;

remove noise from the image of the area outside the vehicle; and

convert the image of the area outside the vehicle to a binary image.

3. The vehicle control apparatus of claim 1, the face recognizing part is further configured to:

find and track faces of a plurality of people outside the vehicle if the image obtained by the image obtaining part includes a plurality of faces; and

extract feature information regarding all of the faces of the plurality of people of people outside the vehicle.

4. The vehicle control apparatus of claim 3, the face authenticating part is further configured to:

individually compare the feature information of all of the faces of the plurality of people outside the vehicle if a plurality of faces exist in the image obtained by the image obtaining part to the feature information regarding the faces of one or more persons registered in advance; and

authenticate all of the faces individually.

5. The vehicle control apparatus of claim 1, wherein the motion recognizing part is further configured to recognize a change of the face of the person outside the vehicle authenticated as having a registered face or a change of a body motion of the person outside the vehicle authenticated as having a registered face.

6. The vehicle control apparatus of claim 5, wherein the motion recognizing part is configured to notify completion of the motion recognition.

7. The vehicle control apparatus of claim 1, wherein the motion recognizing part is configured to detect motions comprising changes in facial expressions including changes of shapes of eyes and a mouth and changes of body motions including changes of shapes of palms, hands, and feet.

8. The vehicle control apparatus of claim 1, further comprising:

an inside communication part configured to transmit data to an onboard computer (ECU) of the vehicle for performing a first specific function of the vehicle based on the motion recognition; and

an outside communication part configured to transmit data to an internet network for performing a second specific function of the vehicle based on the motion recognition.

9. The vehicle control apparatus of claim 1, further comprising:

a face storage part configured to register and store the faces of the one or more persons in advance, the face storage part comprising a portion of or having connectivity to the memory;

a motion storage part configured to register and store motions of the one or more persons having faces stored in advance, based on the faces of the one or more persons, and specific functions associated with the motions, the motion storage part comprising a portion of or having connectivity to the memory; and

a memory part configured to;

store programs that, when executed by a processor, facilitate control of the vehicle control apparatus; and

store identification information for transmitting data via an outside communication part.

10. The vehicle control apparatus of claim 1, further comprising:

an input part that configured to turn the vehicle control apparatus on and off, input and store a face recognition command, input and store a motion recognition command, and to input information regarding a recipient of data to be transmitted via an internet.

11. The vehicle control apparatus of claim 1, wherein the specific function of the vehicle further comprises one or more of turning on vehicle headlights, turning off vehicle headlights, turning on lamps outside a vehicle, turning off lamps outside the vehicle, turning on lamps inside the vehicle, turning off lamps inside the vehicle, a function for capturing a currently parked location, transmitting a captured image to a mobile phone corresponding to a registered phone number, turning on a vehicle black-box, or turning off a vehicle black-box.

12. The vehicle control apparatus of claim 1, wherein the vehicle control apparatus is configured to add new services by upgrading or updating programs via an internet.

13. A vehicle control method performed by a vehicle control apparatus, the method comprising:

registering, by a storage part, one or more faces of persons associated with a vehicle, the registered one or more faces being stored in a memory;

registering, by the storage part, one or more motions of the one or more persons associated with a vehicle having a registered face, and storing the one or more motions of the one or more persons associated with the vehicle having a registered face in the memory obtaining an image of an area outside the vehicle, by an image obtaining part;

finding a face of a person outside the vehicle, by a face recognizing part, the face of the person outside the vehicle being in the image obtained by the image obtaining part, the face of the person outside the vehicle being identified based on feature elements of human faces;

extracting, by the face recognizing part, feature information regarding the face of the person outside the vehicle;

comparing, by a face authenticating part, the feature information regarding the face of the person outside the vehicle to feature information regarding the one or more faces of persons registered and stored by the storage part in advance;

authenticating, by the face authenticating part, the face of the person outside the vehicle as a registered face, if a first difference between the feature information extracted by the face recognizing part and the feature information regarding the faces of the one or more persons registered in advance is less than or equal to a first critical value;

recognizing, by a motion recognizing part, a motion of the person outside the vehicle if the face of the person outside the vehicle is authenticated as being a registered face when the authentication is completed by the face authenticating part;

comparing, by a control part, the motion recognized by the motion recognizing part to the one or more motions registered and stored by the storage part in advance;

causing, by the control part, a specific function of the vehicle to be performed based on one or more motions stored in advance if a second difference between the motion recognized by the control part and one or more motions stored in advance is greater than a second critical value associated with the second quantifiable difference between the motion recognized by the motion recognizing part and the one or more motions stored in advance, the specific function of the vehicle comprising one or more of opening a vehicle door, closing the vehicle door, opening a vehicle trunk, or closing the vehicle trunk; and

requesting, by the motion recognizing part, the person outside the vehicle to show a motion again, if the second quantifiable difference between the motion recognized by the motion recognizing part and the one or more motions stored in advance is less than or equal to the second predetermined critical value.

14. The vehicle control method of claim 13, further comprising:

removing noise from the image obtained by the image obtaining part; and

converting the image obtained by the imaging obtaining part to a binary image.

15. The vehicle control method of claim 13, further comprising:

recognizing a change of the face of the person outside the vehicle authenticated as having a registered face or a change of a body motion of the person outside the vehicle authenticated as having a registered face.

16. The vehicle control method of claim 13, further comprising:

finding and tracking faces of a plurality of people outside the vehicle if the image obtained by the image obtaining part includes a plurality of faces; and

extracting feature information regarding all of the faces of the plurality of people outside the vehicle;

individually comparing the feature information of all of the faces of the plurality of people outside the vehicle if a plurality of faces are recognized by the face recognizing part in the image obtained by the image obtaining part to the feature information regarding the faces of one or more persons registered in advance; and

authenticating all of the faces individually.

17. The vehicle control method of claim 13, wherein each of the recognizing of the face of the person outside the vehicle and the recognizing of the motion of the person outside the vehicle further comprises:

determining whether recognition is successful.

18. The vehicle control method of claim 13, further comprising:

transmitting data to an onboard computer (ECU) via an inside communication part to perform the specific function of the vehicle based on the motion recognition or to an internet via an outside communication part to perform the specific function of the vehicle based on the motion recognition.

19. The vehicle control method of claim 13, wherein the specific function further comprises one or more of turning on vehicle headlights, turning off vehicle headlights, turning on lamps inside the vehicle, turning off lamps inside the vehicle, turning on lamps outside the vehicle, turning off lamps outside the vehicle, a function for capturing a currently parked location and transmitting a captured image to a mobile phone corresponding to a registered phone number, turning on a vehicle black-box, or turning off the vehicle black-box.