SYSTEM AND METHOD FOR IN-VEHICLE INTERACTION

• US 20150178990A1
• Filed: 12/19/2013
• Published: 06/25/2015
• Est. Priority Date: 12/19/2013
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Ng-Thow-Hing, Victor, Bark, Karlin, Tran, Cuong
• Granted Patent: US 9,613,459 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B60K 2360/149 Instrument input by detecting viewing direction not otherwise provided for
• CPC Class Codes: B60K 2360/151 Instrument output devices for configurable output
• CPC Class Codes: B60K 2360/177 Augmented reality
• CPC Class Codes: B60K 2360/186 Displaying information according to relevancy
• CPC Class Codes: B60K 2360/55 Remote control arrangements
• CPC Class Codes: B60K 2370/149 Input by detecting viewing direction
• CPC Class Codes: B60K 2370/151 Reconfigurable output
• CPC Class Codes: B60K 2370/1529 Head-up displays
• CPC Class Codes: B60K 2370/177 Augmented reality
• CPC Class Codes: B60K 2370/186 Displaying Information according to relevancy
• CPC Class Codes: B60K 2370/52 Control of displays
• CPC Class Codes: B60K 2370/55 Remote controls
• CPC Class Codes: B60K 2370/736 the user being the driver
• CPC Class Codes: B60K 2370/739 the user being the passenger
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 35/20 Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 35/23 Head-up displays [HUD] optical aspects of head-up displays G02B27/01
• CPC Class Codes: B60K 35/28 characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B60K 35/29 Instruments characterised by the way in which information is handled, e.g. showing information on plural displays or prioritising information according to driving conditions
• CPC Class Codes: B60K 35/654 : the user being the driver
• CPC Class Codes: B60K 35/656 : the user being a passenger
• CPC Class Codes: B60K 35/80 : Arrangements for controlling instruments
• CPC Class Codes: B60K 35/81 : for controlling displays
• CPC Class Codes: G06F 3/011 : Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 : Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/147 : using display panels
• CPC Class Codes: G06T 19/006 : Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 20/20 : in augmented reality scenes

Abstract

A system and method for interaction in a vehicle includes establishing a bi-directional interaction session between a first display and a second display via a communication link, wherein each of the first display and the second display are communicatively coupled to the vehicle and the first display is operated by a first user and the second display is operated by a second user. Upon receiving a first interaction input associated generating a first augmented reality object based on the first interaction input and transmitting via the communication link the first augmented reality object to the second display. Upon receiving a second interaction input, generating a second augmented reality object based upon the second interaction input and transmitting via the communication link the second augmented reality object to the first display.

Claims

1. A method for interaction in a vehicle, comprising:

establishing a bi-directional interaction session between a first display and a second display via a communication link, wherein each of the first display and the second display are communicatively coupled to the vehicle and the first display is operated by a first user and the second display is operated by a second user;

upon receiving a first interaction input associated with the first user, generating a first augmented reality object based on the first interaction input;

transmitting via the communication link the first augmented reality object to the second display for displaying the first augmented reality object on the second display;

upon receiving a second interaction input associated with the second user, generating a second augmented reality object based upon the second interaction input; and

transmitting via the communication link the second augmented reality object to the first display for displaying the second augmented reality object on the first display.

2. The method of claim 1, wherein the first augmented reality object is based on the first interaction input and a local view displayed on the first display.

3. The method of claim 1, wherein the second augmented reality object is based on the second interaction input and a local view displayed on the second display.

4. The method of claim 1, wherein the first augmented reality object comprises a real-time point of view of the first user.

5. The method of claim 1, wherein one of the first display or the second display is a driver display operated by a driver of the vehicle and the other of the first display or the second display is a passenger display operated by a passenger of the vehicle.

6. The method of claim 5, wherein the driver display is a heads-up display disposed on a front window of the vehicle.

7. The method of claim 1, wherein establishing the bi-directional interaction session comprises establishing the bi-directional interaction session upon detecting an initialization event.

8. The method of claim 7, wherein the initialization event comprises a request for interaction and indicates one other display communicatively coupled to the vehicle for interaction.

9. The method of claim 1, wherein establishing the bi-directional interaction session enables real-time display and augmentation of data between the first display and the second display, wherein the data is received from a plurality of sensors disposed about the vehicle.

10. A non-transitory computer-readable storage medium storing instructions that, when executed by a vehicle computer, causes the computer to perform the steps of:

establishing a bi-directional interaction session between a first display and a second display via a communication link, wherein each of the first display and the second display are communicatively coupled to the vehicle and the first display is operated by a first user and the second display is operated by a second user;

upon receiving a first interaction input associated with the first user, generating a first augmented reality object based on the first interaction input;

transmitting via the communication link the first augmented reality object to the second display for displaying the first augmented reality object on the second display;

upon receiving a second interaction input associated with the second user, generating a second augmented reality object based upon the second interaction input; and

transmitting via the communication link the second augmented reality object to the first display for displaying the second augmented reality object on the first display.

11. The non-transitory computer-readable storage medium of claim 10, wherein the first augmented reality object is based on the first interaction input and a local view displayed on the first display.

12. The non-transitory computer-readable storage medium of claim 10, wherein the second augmented reality object is based on the second interaction input and a local view displayed on the second display.

13. The non-transitory computer-readable storage medium of claim 10, wherein one of the first display or the second display is a driver display operated by a driver of the vehicle and the other of the first display or the second display is a passenger display operated by a passenger of the vehicle.

14. The non-transitory computer-readable storage medium of claim 10, wherein the first augmented reality object is overlaid on a local view displayed on the second display..

15. A system for interaction in a vehicle, comprising:

a first display and a second display each communicatively coupled to the vehicle and each configured for bi-directional interaction between the first display and the second display, wherein the first display is operated by a first user and the second display is operated by a second user;

an interaction module for receiving a first interaction input associated with the first user and a second interaction input associated with the second user; and

an augmented reality module for generating a first augmented reality object based on the first interaction input and rendering the first augmented reality object on the second display and for generating a second augmented reality object based on the second interaction input and rendering the second augmented reality object on the second display.

16. The system of claim 15, wherein the augmented reality module generates the first augmented reality object based on the first interaction input and a local view displayed on the first display.

17. The system of claim 15, wherein the augmented reality module generates the second augmented reality object based on the second interaction input and a local view displayed on the second display.

18. The system of claim 15, wherein one of the first display or the second display is a driver display operated by a driver of the vehicle and the other of the first display or the second display is a passenger display operated by a passenger of the vehicle.

19. The system of claim 15, wherein the interaction module is configured to detect an initialization event thereby establishing the bi-directional interaction session.

20. The system of claim 19, wherein the initialization event comprises a request for interaction indicating a further display communicatively coupled to the vehicle for interaction.