Mirroring deeplinks

• US 10,089,053 B2
• Filed: 08/01/2014
• Issued: 10/02/2018
• Est. Priority Date: 08/01/2014
• Status: Active Grant

• Current Assignee: Audi AG (Volkswagen AG), Volkswagen AG
• Original Assignee: Audi AG (Volkswagen AG), Volkswagen AG
• Inventors: Gerlach, Simon, Scholz, Lars, Post, Henning
• Primary Examiner(s): Yeung, Matthew
• Application Number:
  US14/449,218
• Publication Number:
  US 20160034238A1
• Time in Patent Office: 1,523 Days
• Field of Search: None
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/1423 controlling a plurality of local displays, e.g. CRT and flat panel display
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G09G 2370/16 Use of wireless transmission of display information
• CPC Class Codes: G09G 5/003 Details of a display terminal, the details relating to the control arrangement of the display terminal and to the interfaces thereto specific for a CRT G09G1/165; for a flat panel G09G3/2092
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04W 4/80 Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication
• CPC Class Codes: H04W 76/14 Direct-mode setup

Abstract

An in-vehicle system for providing a driver with access to functionality of a mobile device that includes a head unit with an in-vehicle display for providing infotainment to the driver, software that configures the user interface with a graphical scheme native to the head unit, and a software interface by which the head unit can receive deep links from the mobile device. Deep links identify application contexts available within the mobile software applications, and are made available to a user from within in-vehicle software applications. When a user selects a deep link from within an in-vehicle software application, the deep link is followed and the appropriate application context is rendered by mobile device. The application context is then mirrored onto the in-vehicle display, where a driver can view and interact with it using mirroring technology.

Claims

1. An in-vehicle system for providing a user with access to functionality of a mobile device, the system comprising:

a head unit for providing information and entertainment to the driver, having;

a processor;

an in-vehicle display;

in-vehicle software applications accessible to the user via the in-vehicle software applications;

a non-transitory data storage on which is stored computer code which, when executed on the processor, causes the head unit to provide a user interface for the in-vehicle software applications; and

user interface software that configures the user interface with a graphical scheme native to the head unit;

a communication interface providing communication between the head unit and a mobile device;

means for providing deep links to the head unit, wherein each of the deep links identifies a mobile software application stored on the mobile device and an application context available within the identified mobile software application;

means for enabling a user to select at least one of the deep links from within a user interface screen provided on the in-vehicle display using one of the in-vehicle software applications;

means for establishing a communication session using a mirroring protocol connection between the one of the in-vehicle software applications on the head unit and the mobile software application identified by the selected deep link; and

software for mirroring, under control of the mobile software application on the mobile device, an application context on the in-vehicle display which is also rendered on a display of the mobile device, wherein the mirrored application context is the application context identified by the selected deep link,

wherein the selected deep link includes encoded required actions that a mobile application should automatically take upon launch to arrive at a specific application context of interest without requiring the user to navigate through intervening screens to arrive at the specific application context of interest,

wherein the application context rendered on the display of the mobile device and mirrored on the in-vehicle display under the control of the mobile software application running on the mobile device is displayed on the in-vehicle display along with a navigation icon,

wherein the mobile applications on the mobile phone advertise services that can be provided to in-vehicle software of the head unit, and the head unit requests content from the mobile applications,

wherein, in response to the request for content, the mobile applications deliver content to the head unit in accordance with a HMI protocol so as to communicate selected items of information to the head unit encapsulated in fields in accordance with a schema, whereby the head unit parses the received information according to the schema,

wherein selection of the navigation icon by the user triggers the head unit to terminate the mirroring protocol connection between the head unit and the mobile software application identified by the deep link to return control to the head unit to provide the user interface for the in-vehicle software application on the in-vehicle display, and

wherein the data format set forth by the HMI protocol enables mobile applications to transmit deep links to the head unit, the HMI protocol is implemented as an XML interface, and the schema defining the XML interface include fields encapsulating deep links so as to enable specific application contexts from within mobile applications to be accessible via mirroring.

2. The in-vehicle system of claim 1, wherein the software for mirroring the application context includes client software that adheres to a mirroring protocol.

3. The in-vehicle system of claim 1, wherein the deep links identify application contexts available within the mobile software applications.

4. The in-vehicle system of claim 1, wherein the means for enabling a user to select at least one of the deep links comprises a user control for user selection of a graphical icon for the deep link, wherein the graphical icon is rendered by one of the in-vehicle software applications in accordance with the graphical scheme native to the head unit.

5. The in-vehicle system of claim 1, wherein the in-vehicle display shows graphical icons corresponding to the mobile software applications alongside graphical icons corresponding to in-vehicle software applications on a common user interface screen and in accordance with the graphical scheme native to the head unit.

21. The in-vehicle system of claim 1, wherein following termination of the mirroring protocol connection, control of the in-vehicle display is returned to the head unit, which renders an application context displayed before the mirroring protocol connection was established.

22. The in-vehicle system of claim 1, wherein in response to user selection of a graphical icon having an associated deep link, the head unit invokes software to determine whether the user should be permitted to follow the deep link or whether access to the deep link should be blocked.

23. The in-vehicle system of claim 22, wherein the determination to block is based on at least one of safety considerations, user experience considerations, or a state of the vehicle.

24. The in-vehicle system of claim 23, wherein the state of the vehicle includes at least one of movement of the vehicle or vehicle speed exceeding a threshold.

6. An in-vehicle system for providing access to functionality of a mobile device, the system comprising:

a head unit having;

a processor;

an in-vehicle display;

in-vehicle software applications accessible to a user via the in-vehicle display; and

a non-transitory data storage on which is stored computer code which, when executed on the processor, causes the head unit to provide a user interface on the in-vehicle display;

a communication interface providing communication between the head unit and a mobile device;

software implementing a protocol by which the mobile device registers deep links with the head unit, wherein each of the deep links identifies a mobile software application stored on the mobile device and an application context within the identified mobile software application;

a user control by which a user can select at least one deep link from within a user interface screen provided on the in-vehicle display by one of the in-vehicle software applications; and

software implementing a mirroring protocol to establish a mirroring protocol connection to render, on the in-vehicle display, the application context identified by the selected deep link,

wherein the selected deep link includes encoded required actions that a mobile application should automatically take upon launch to arrive at a specific application context of interest without requiring the user to navigate through intervening screens to arrive at the specific application context of interest,

wherein the mobile applications on the mobile phone advertise services that can be provided to in-vehicle software of the head unit, and the head unit requests content from the mobile applications,

wherein, in response to the request for content, the mobile applications deliver content to the head unit in accordance with a HMI protocol so as to communicate selected items of information to the head unit encapsulated in fields in accordance with a schema, whereby the head unit parses the received information according to the schema,

wherein the application context rendered on the display of the mobile device and mirrored on the in-vehicle display under the control of the mobile software application on the mobile device is displayed on the in-vehicle display along with a navigation icon, wherein selection of the navigation icon by the user triggers in-vehicle software to terminate the mirroring protocol connection between the head unit and the mobile software application identified by the deep link to return control to the head unit to provide the user interface for the in-vehicle software application on the in-vehicle display, and

wherein the data format set forth by the HMI protocol enables mobile applications to transmit deep links to the head unit, the HMI protocol is implemented as an XML interface, and the schema defining the XML interface include fields encapsulating deep links so as to enable specific application contexts from within mobile applications to be accessible via mirroring.

7. The in-vehicle system of claim 6, wherein the software implementing the protocol that renders the application context identified by the deep link includes client software that adheres to a mirroring protocol.

8. The in-vehicle system of claim 6, wherein at least one of the deep links is associated with either or both of a graphical icon and a textual display for the mobile software application identified by the deep link.

9. The in-vehicle system of claim 8, wherein the either or both of a graphical icon and a textual display for the mobile software application is rendered by at least one of the in-vehicle software applications in accordance with a graphical scheme that is native to the head unit.

10. The in-vehicle system of claim 9, wherein the graphical icon represents a point of interest displayed within a mapping interface provided by an in-vehicle navigation application;

and

the deep link associated with the graphical icon encodes instructions for one of the mobile software applications to navigate to a user interface screen containing information about the point of interest.

11. The in-vehicle system of claim 9, wherein the graphical icon corresponds to an audio application stored on the mobile device;

and

the graphical icon corresponding to the audio application is displayed as an alternative to audio capabilities native to the head unit.

12. The in-vehicle system of claim 9, wherein the head unit determines the in-vehicle software application in which the graphical icon will be displayed based on information conveyed by the mobile software application.

13. The in-vehicle system of claim 6, further comprising software that monitors for assessing one or both of a state of a vehicle containing the in-vehicle system and a state of a driver driving the vehicle, and

software for determining whether to launch the mobile software application identified by the selected deep link based on an assessment of whether it is safe to launch the mobile software application.

14. The in-vehicle system of claim 6, wherein the selected deep link includes a placeholder in which the head unit can provide vehicle-specific information to the mobile software application corresponding to the selected deep link.

15. The in-vehicle system of claim 6, further comprising

a monitor that determines a state of a vehicle containing the in-vehicle system, and

software for automatically invoking, responsive to the determined state of the vehicle, a deep link.

25. The in-vehicle system of claim 6, wherein following termination of the mirroring protocol connection, control of the in-vehicle display is returned to the head unit, which renders an application context displayed before the mirroring protocol connection was established.

26. The in-vehicle system of claim 6, wherein in response to user selection of a graphical icon having an associated deep link, the head unit invokes software to determine whether the user should be permitted to follow the deep link or whether access to the deep link should be blocked.

27. The in-vehicle system of claim 26, wherein the determination to block is based on at least one of safety considerations, user experience considerations, or a state of the vehicle.

28. The in-vehicle system of claim 27, wherein the state of the vehicle includes at least one of movement of the vehicle or vehicle speed exceeding a threshold.

16. A method for providing functionality of a mobile device to an in-vehicle system, the method comprising:

providing a user interface on a head unit display,

configuring the user interface in accordance with a graphical scheme implemented by in-vehicle software;

receiving a communication in accordance with a software protocol from a mobile device, wherein the communication provides a deep link that identifies a mobile software application stored on the mobile device and an application context within the mobile software application;

enabling a user to select the deep link from within a user interface screen provided on the in-vehicle display by one of the in-vehicle software applications while interacting with in-vehicle software via the in-vehicle display;

establishing a communication session using a mirroring protocol connection between the head unit and the mobile software application identified by the selected deep link; and

rendering under control of the mobile software application on the mobile device, the application context identified by the deep link on the head unit display,

wherein the deep link includes encoded required actions that a mobile application should automatically take upon launch to arrive at a specific application context of interest without requiring the user to navigate through intervening screens to arrive at the specific application context of interest,

wherein the mobile applications on the mobile phone advertise services that can be provided to in-vehicle software of the head unit, and the head unit requests content from the mobile applications,

wherein, in response to the request for content, the mobile applications deliver content to the head unit in accordance with a HMI protocol so as to communicate selected items of information to the head unit encapsulated in fields in accordance with a schema, whereby the head unit parses the received information according to the schema,

wherein the application context rendered on the display of the mobile device and mirrored on the in-vehicle display under the control of the mobile software application on the mobile device is displayed on the in-vehicle display along with a navigation icon, wherein selection of the navigation icon by the user triggers in-vehicle software to terminate the mirroring protocol connection between the head unit and the mobile software application identified by the deep link to return control to the head unit to provide the user interface for the in-vehicle software application on the in-vehicle display, and

wherein the data format set forth by the HMI protocol enables mobile applications to transmit deep links to the head unit, the HMI protocol is implemented as an XML interface, and the schema defining the XML interface include fields encapsulating deep links so as to enable specific application contexts from within mobile applications to be accessible via mirroring.

17. The method of claim 16, wherein enabling a user to select the deep link comprises associating either or both of a graphical icon and a textual display with the deep link and enabling the user to select the either or both of a graphical icon and a textual display.

18. The method of claim 17, comprising displaying the either or both of a graphical icon and a textual display alongside graphical icons or textual displays for in-vehicle software applications.

19. The method of claim 16, comprising determining whether to follow the selected deep link based on an assessment of a state of a vehicle containing the in-vehicle system.

20. The method of claim 16, comprising automatically following a deep link based on an assessment of a state of a vehicle containing the in-vehicle system.

29. The method of claim 16, wherein following termination of the mirroring protocol connection, control of the in-vehicle display is returned to the head unit, which renders an application context displayed before the mirroring protocol connection was established.

30. The method of claim 16, wherein in response to user selection of a graphical icon having an associated deep link, the head unit invokes software to determine whether the user should be permitted to follow the deep link or whether access to the deep link should be blocked.

31. The method of claim 30, wherein the determination to block is based on at least one of safety considerations, user experience considerations, or a state of the vehicle.

32. The method of claim 30, wherein the state of the vehicle includes at least one of movement of the vehicle or vehicle speed exceeding a threshold.