IN-VEHICLE APPARATUS AND INFORMATION DISPLAY SYSTEM

• US 20120245945A1
• Filed: 03/20/2012
• Published: 09/27/2012
• Est. Priority Date: 03/23/2011
• Status: Active Grant

• Current Assignee: DENSO Corporation
• Original Assignee: DENSO Corporation
• Inventors: Miyauchi, Hideo, Masuda, Koichi
• Granted Patent: US 8,700,408 B2
• Time in Patent Office: Days
• US Class Current: 704/275
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G10L 15/06 Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice G10L15/14 takes precedence
• CPC Class Codes: G10L 2015/228 of application context

Abstract

An in-vehicle apparatus receives an image data representative of a screen image from a portable terminal with a touch panel. The apparatus extracts a text code data from the image data, and identifies a text-code display area in the screen image. The apparatus determines a command text based on a user-uttered voice command. The apparatus identifies a text-code display area as a subject operation area in the screen image of the portable terminal, based on the command text, the text code data extracted from image data, and information on the text-code display area corresponding to the text code data. An area of the screen image of the touch panel corresponding to the text-code display area is identified as the subject operation area, and a signal indicative of the subject operation area identified is transmitted to the portable terminal.

Claims

1. An in-vehicle apparatus receiving image data representative of a screen image from a portable terminal having a touch panel, and the apparatus displaying the screen image of the touch panel of the portable terminal based on the image data, the apparatus comprising:

a vehicle communication unit receiving the image data from the portable terminal;

an extraction unit extracting a text code data from the image data received by the vehicle communication unit, the image data having the text code data that represents at least one of a text and a code in the screen image of the touch panel of the portable terminal;

a text-code display area identification unit identifying a text-code display area in the screen image;

a voice recognition unit recognizing a user-uttered voice command and converting a recognized voice command into a command text; and

a subject operation area set unit identifying a text-code display area as a subject operation area in the screen image of the portable terminal, wherein

when the voice recognition unit acquires the command text, a determined text code data of the command text is determined based on the command text acquired from the voice recognition unit, the text code data extracted by the extraction unit, and information corresponding to the text code data on the text-code display area,

the text-code display area is acquired based on the determined text code data,

the subject operation area is identified as an area of the screen image of the touch panel corresponding to the text-code display area acquired by the subject operation area set unit, and

a signal indicative of the subject operation area identified by the subject operation area set unit is transmitted to the portable terminal by the vehicle communication unit.

2. The in-vehicle apparatus of claim 1 further comprising:

an association generation unit generating a text-code-and-area association between the text code data and the text-code display area based on the information of the text code data from the extraction unit and the text-code display area identified by the text-code display area identification unit, wherein

the subject operation area set unit identifies the text code data corresponding to the command text with reference to the text-code-and-area association and the text-code display area of the identified text code data.

3. The in-vehicle apparatus of claim 1, further comprising:

an in-vehicle touch panel displaying the screen image of the portable terminal based on the image data received by the vehicle communication unit, and the in-vehicle touch panel being operational, wherein

when the in-vehicle touch panel receives an operation, an operation area of the in-vehicle touch panel is determined in the screen image,

an area of the touch panel of the portable terminal corresponding to the operation area of the in-vehicle touch panel is identified as the subject operation area, and

a signal indicative of the subject operation area is transmitted from the vehicle communication unit to the portable terminal.

4. The in-vehicle apparatus of claim 1 further comprising:

an association storage unit storing a predetermined association between a code and a text indicating contents of the code, wherein

based on the association stored in the association storage unit, when the text code data extracted by the extraction unit is a code, the association generation unit generates the text-code-and-area association between the text code data and the text-code display area by using the text code data of the text indicating the contents of the code.

5. The in-vehicle apparatus of claim 1 wherein,

the extraction unit extracts the text code data from the image data received by the vehicle communication unit each time the vehicle communication unit receives new image data and the screen image is displayed,

the text-code display area identification unit identifies the text-code display area each time the text code data is extracted,

newly-extracted text code data and newly-identified text-code display area are associated to define a new association, and

the new association is appended to the existing text-code-and-area association.

6. The in-vehicle apparatus of claim 5 further comprising:

a presentation unit displaying the screen image of the portable terminal based on the image data received by the vehicle communication unit, wherein

when the voice recognition unit acquires a text, the subject operation area set unit acquires the text code data corresponding to the text, and determines if the text code data is included in the screen image currently being displayed,

when the text code data is included in the screen image, the subject operation area set unit identifies the text-code display area corresponding to the text code data, and identifies a corresponding area on the screen image of the touch panel of the portable terminal as the subject operation area, and

when the text code data is not determined to be included in the screen image currently being displayed, the presentation unit provides a message that a process indicated by the text code data cannot be performed in the screen image currently being displayed.

7. The in-vehicle apparatus of claim 6, wherein when the subject operation area set unit determines that the text code data corresponding to the text acquired by the voice recognition unit is not included in the screen image, the presentation unit further displays a notice requesting to switch the currently displayed screen.

8. The in-vehicle apparatus of claim 1, wherein the subject operation area set unit identifies the subject operation area in the screen image of the touch panel of the portable terminal by acquiring the text-code display area corresponding to the text code data that is an exact match with the command text recognized by the voice recognition unit and by mapping the text-code display area acquired to an area in the screen image of the touch panel of the portable terminal.

9. The in-vehicle apparatus of claim 1 further comprising:

an input unit receiving an input, wherein

the subject operation area set unit identifies the subject operation area in the screen image of the touch panel of the portable terminal by determining a partial match of at least one character in the command text with the text code data in the text-code-and-area association for establishing partial match text code data entries, and by receiving a selected text code data inputted from the input, wherein the selected text code data is one of a partial match text code data from the partial match text code data entries.

10. The in-vehicle apparatus of claim 1 further comprising:

a presentation unit displaying the screen image of the touch panel to the user, wherein the text code data included in the text-code-and-area association generated by the association generation unit is emphasized in the screen image displayed on the presentation unit.

11. The in-vehicle apparatus of claim 1 further comprising:

a list unit generating a list of the text code data included in text-code-and-area association generated by the association generation unit, wherein

when the text code data of the command text corresponding to the text-code display area is not identified by the subject operation area set unit, the text code data in the text-code-and-area association generated by the association generation unit is shown in a list generated by the list unit.

12. The in-vehicle apparatus of claim 1, wherein when the screen image includes multiple division areas and a subject text code data exists in more than one of the multiple division areas, the association generation unit generates the text-code-and-area association with a division attribute text added to the association, in which one of multiple division areas of the screen image having a subject text code data is specified

13. The in-vehicle apparatus of claim 1 further comprising:

a display unit displaying the position of text code data provided in the multiple division areas of the screen image of the touch panel of the portable device, the display unit further receiving an input in regards to the text code data in the multiple division areas, wherein

when the text code data corresponding to the command text is provided in more than one of the multiple division areas, the subject operation area set unit determines the text code data according to the input received by the display unit.

14. An information display system, comprising:

a portable terminal including

a touch panel receiving an input from a user,

a terminal communication unit communicating with other devices, and

a display unit displaying a screen image according to an image data, the image data having a text code data that represents at least one of a text and a code in the screen image of the touch panel; and

an in-vehicle apparatus including

a vehicle communication unit communicating with the terminal communication unit of the portable terminal, wherein the vehicle communication unit receives the image data from the portable terminal via the terminal communication unit,

an extraction unit extracting the text code data from the image data received by the vehicle communication unit,

a text-code display area identification unit identifying a text-code display area in the screen image,

a voice recognition unit recognizing a user-uttered voice command and converts a recognized voice command into a command text, and

a subject operation area set unit identifying a text-code display area as a subject operation area in the screen image of the portable terminal,

wherein when the voice recognition unit acquires the command text, a determined text code data of the command text is determined based on the command text acquired from the voice recognition unit, the text code data extracted by the extraction unit, and information corresponding to the text code data on the text-code display area,

the text-code display area is acquired based on the determined text code data,

an area of the screen image of the touch panel corresponding to the text-code display area acquired by the subject operation area set unit is identified as the subject operation area, and

a signal indicative of the subject operation area identified by the subject operation area set unit is transmitted to the portable terminal by the vehicle communication unit,

wherein when the portable terminal receives the signal indicative of the subject operation area from the in-vehicle apparatus, the signal is treated in a manner similar to an input operation of the identified subject operation area of the touch panel of the portable terminal.

15. An information display system comprising:

a voice recognition server recognizing a user-uttered voice command;

a portable terminal including a touch panel to receive an input and a display unit to display a screen image according to an image data; and

an in-vehicle apparatus including a display unit to display an image, wherein the in-vehicle apparatus receives an image data from a portable terminal having a touch panel and displays a screen image of the touch panel based on the image data received,

the in-vehicle apparatus further including

a vehicle communication unit receiving the image data of the screen image of the portable terminal,

an extraction unit extracting text code data from the image data received by the vehicle communication unit, the image data includes the text code data that represents at least one of a text and a code in the screen image of the touch panel of the portable terminal,

a text-code display area identification unit identifying a text-code display area in the screen image,

a voice recognition server communication unit transmitting a user-uttered voice command to the voice recognition server and receives a command text from the voice recognition server based on the user-uttered voice command, and

a subject operation area set unit identifying a text-code display area as a subject operation area in the screen image of the portable terminal, wherein, when the command text is acquired from the voice recognition server communication unit, a determined text code data of the command text is determined based on the command text acquired, the text code data extracted by the extraction unit, and information corresponding to the text code data on the text-code display area, and the determined text code data and the text-code display area is acquired based on the determined text code data, and an area of the screen image of the touch panel of the portable terminal corresponding to the text-code display area is determined as the subject operation area, and a signal indicative of the subject operation area identified by the subject operation area set unit is transmitted to the portable terminal by the vehicle communication unit.