Notification control apparatus and method for controlling notification

• US 10,885,788 B2
• Filed: 04/24/2017
• Issued: 01/05/2021
• Est. Priority Date: 04/24/2017
• Status: Active Grant

• Current Assignee: Mitsubishi Electric Corporation
• Original Assignee: Mitsubishi Electric Corporation
• Inventors: Miyahara, Tadashi, Shimotani, Mitsuo, Obata, Naohiko, Ueno, Yoshinori, Matsuda, Yuji
• Primary Examiner(s): Nwugo, Ojiako K
• Application Number:
  US16/487,595
• Publication Number:
  US 20200058222A1
• Time in Patent Office: 1,352 Days
• Field of Search: 340468
• CPC Class Codes: B60Q 1/525 automatically indicating risk of collision between vehicles in traffic or with pedestrians, e.g. after risk assessment using the vehicle sensor data
• CPC Class Codes: B60Q 1/547 for issuing requests to other traffic participants; for confirming to other traffic participants they can proceed, e.g. they can overtake
• CPC Class Codes: B60Q 2400/50 Projected symbol or information, e.g. onto the road or car body
• CPC Class Codes: B60Q 5/005 automatically actuated
• CPC Class Codes: B60Q 5/006 indicating risk of collision between vehicles or with pedestrians
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G08G 1/005 including pedestrian guidance indicator
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

An object of the present invention to provide a notification control apparatus and a method for controlling notification that achieve display to a notification target with improved accuracy. A notification control apparatus in the present invention includes the following: an attribute detecting unit that detects an attribute of an object located around a subject vehicle; an input recognizing unit that recognizes an input message entered by the driver of the subject vehicle; a message converting unit that converts, in accordance with the attribute detected by the attribute detecting unit, the input message recognized by the input recognizing unit into a notification message indicating the same meaning as the input message; and a road projection controller that controls projection of the notification message converted by the message converting unit onto at least part of a road located around the object.

Claims

1. A notification control apparatus comprising:

a processor to execute a program; and

a memory to store the program which, when executed by the processor, performs processes of,

detecting an attribute of an object located around a subject vehicle;

recognizing an input message entered by a driver of the subject vehicle;

converting, in accordance with the detected attribute, the recognized input message into a notification message indicating the same meaning as the input message; and

controlling projection of the converted notification message onto at least part of a road located around the object.

2. The notification control apparatus according to claim 1, wherein

the program, when executed by the processor, performs a process of generating a concept of the input message that is notified to the object, from the recognized input message, and

the converting process comprises converting the generated concept of the input message.

3. The notification control apparatus according to claim 2, wherein

the program, when executed by the processor, performs processes of,

detecting a travel state of the subject vehicle,

detecting a relative position of the object relative to the subject vehicle,

detecting a road condition in a direction of travel of the subject vehicle, and

determining presence or absence of the object to be notified, on the basis of the detected travel state, the detected relative position of the object, and the detected road condition, and

upon determining the presence of the object to be notified, the converting process comprises converting the concept of the input message into the notification message.

4. The notification control apparatus according to claim 3, wherein

the program, when executed by the processor, further performs a process of calculating a route from a current position of the subject vehicle to a destination, and

the determining process comprises determining the presence or absence of the object to be notified, on the basis of the detected travel state, the detected relative position of the object, the detected road condition, and the calculated route.

5. The notification control apparatus according to claim 3, wherein

the process of detecting an attribute comprises detecting an orientation of a face of the object as the attribute, and

upon determining the presence of the object to be notified, the determining process comprises estimating a direction in which the object will move, on the basis of a change in the detected orientation of the face of the object, a change in the detected relative position of the object, and the detected road condition.

6. The notification control apparatus according to claim 5, wherein

upon determining that the object is located within a predetermined range that is predetermined based on a travel route of the subject vehicle, the converting process comprises converting the concept of the input message into the notification message in a form of a line drawing, and

upon determining that the object is located within the predetermined range, the process of controlling projection comprises controlling projection of the line drawing between the subject vehicle and the object.

7. The notification control apparatus according to claim 6, wherein

upon estimating that the object will move from the predetermined range toward the travel route of the subject vehicle, the process of controlling projection comprises controlling projection of the notification message onto a location toward which the face of the object is oriented.

8. The notification control apparatus according to claim 2, wherein

the recognizing process comprises recognizing a sound indicating the input message, and

the generating process comprises the concept of the input message from the recognized sound.

9. The notification control apparatus according to claim 8, wherein

the recognizing process comprises recognizing a gesture indicating the input message, and

upon recognizing the gesture and the sound, the generating process comprises generating the concept of the input message.

10. The notification control apparatus according to claim 2, wherein the recognizing process comprises recognizing a gesture for selecting one of a plurality of the objects located around the subject vehicle.

11. The notification control apparatus according to claim 10, wherein

the recognizing process comprises recognizing a sound indicating the input message,

upon recognizing the gesture and the sound, the generating process comprises generating the concept of the input message, and

the process of controlling projection comprises controlling projection of the notification message onto at least part of a road located around the object identified through the gesture.

12. The notification control apparatus according to claim 1, wherein the program, when executed by the processor, performs a process of controlling audio output of the notification message to the object.

13. The notification control apparatus according to claim 1, wherein the attribute of the object includes a person'"'"'s age, a person'"'"'s physical feature, an orientation of person'"'"'s face, a language used by a person, and a kind of animal.

14. The notification control apparatus according to claim 1, wherein

the program, when executed by the processor, further performs a process of communicating with a portable communication terminal carried with a person who is the object located around the subject vehicle,

the communicating process comprises acquiring, from the portable communication terminal, language information that is set in the portable communication terminal, and

the process of detecting an attribute comprises detecting, as the attribute, the acquired language information.

15. The notification control apparatus according to claim 1, wherein the process of controlling projection comprises controlling projection of the notification message in animation form or of the notification message in blinking form onto the road.

16. The notification control apparatus according to claim 1, wherein

the recognizing process comprises a process of summarizing an utterance of the driver, and

the summarizing process comprises summarizing the utterance to form the input message.

17. The notification control apparatus according to claim 1, wherein

the recognizing process comprises recognizing a sight line of the driver and a sound of the driver, and

upon recognizing that the driver has uttered a word while looking at the object, the process of controlling projection comprises controlling projection of the notification message.

18. The notification control apparatus according to claim 1, wherein

the recognizing process comprises recognizing a gesture of the driver and a sound of the driver, and

upon recognizing that the driver has uttered a word while pointing at the object, the process of controlling projection comprises controlling projection of the notification message.

19. The notification control apparatus according to claim 1, wherein

the program, when executed by the processor, further performs a process of displaying at least one of a plurality of the objects located around the subject vehicle, and

when the driver selects the displayed object, the process of controlling projection comprises controlling projection of the notification message onto at least part of the road located around the selected object.

20. A method for controlling notification, comprising:

detecting an attribute of an object located around a subject vehicle;

recognizing an input message entered by a driver of the subject vehicle;

converting, in accordance with the detected attribute, the recognized input message into a notification message indicating the same meaning as the input message; and

controlling projection of the converted notification message onto at least part of a road located around the object.