Method and apparatus for presenting task-related objects in an augmented reality display

• US 10,338,786 B2
• Filed: 05/25/2018
• Issued: 07/02/2019
• Est. Priority Date: 02/22/2013
• Status: Active Grant

• Current Assignee: HERE Global B.V. (Nokia Corporation)
• Original Assignee: HERE Global B.V. (Nokia Corporation)
• Inventors: Tuukkanen, Marko Tapio
• Primary Examiner(s): Crawford, Jacinta M
• Assistant Examiner(s): Doan, Phuc N
• Application Number:
  US15/990,177
• Publication Number:
  US 20180275848A1
• Time in Patent Office: 403 Days
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range

Abstract

An approach is provided for causing a presentation in an augmented reality user interface for user guidance. The approach involves causing a presentation of one or more indications of one or more parts in an augmented reality user interface, wherein the one or more parts are associated with at least one task. The approach also involves causing a presentation of one or more guides for aligning the one or more indications with the one or more parts in the augmented reality user interface.

Claims

1. A method comprising:

initiating, by an apparatus, one or more indications of one or more parts in a plurality of augmented reality user interfaces of respective user devices, wherein the one or more parts are associated with at least one task associated with a vehicle to be performed via coordinated actions of users of the respective user devices, and wherein each of the augmented reality user interfaces comprises an electronic display of a camera;

initiating, by the apparatus, one or more guides for alignment of the one or more indications with the one or more parts respectively in the augmented reality user interfaces corresponding to the coordinated actions to be performed by the users, wherein the alignment is realized by movement of the respective augmented reality user interface to align the one or more indications with the one or more parts located in real space, and wherein the electronic display allows the respective user to view the augmented reality user interface during the alignment and while performing the coordinated actions;

processing, by the apparatus, sensor information, telematics information, or a combination thereof collected at the user devices, the vehicle, or a combination thereof to determine the coordinated actions;

initiating, by the apparatus in response to the sensor information, the telematics information, or a combination thereof, a presentation in the electronic display of a tutorial video via one or more networks, wherein the tutorial video is a step by step guide to perform the coordinated actions as steps; and

upon determining based on the sensor information, the telematics information, or a combination thereof that one step shown on one of the user devices is completed by one of the users that uses the user device, initiating, by the apparatus, real-time skipping the step from the presentation of the tutorial video on the remaining one or more user devices and presenting a next step of the tutorial video on the remaining one or more user devices,

wherein the different steps of the tutorial video are presented among the user devices based, at least in part, on the real-time skipping.

2. A method of claim 1, further comprising:

processing a field of view of the respective augmented reality user interface to cause, at least in part, a recognition of the one or more parts, the at least one task, or a combination thereof.

3. A method of claim 2, further comprising:

determining one or more reference objects associated with the field of view, the at least one task, or a combination thereof; and

determining the one or more guides based, at least in part, on the one or more reference objects.

4. A method of claim 3, wherein the one or more guides include, at least in part, one or more outline representations of the one or more reference objects.

5. A method of claim 1, wherein the one or more guides indicate, at least in part, a direction, an orientation, or a combination thereof to move the respective user device presenting the respective augmented reality user interface for aligning the one or more indications with the one or more parts.

6. A method of claim 1, further comprising:

determining an input for specifying the one or more parts, the at least one task, an association between the one or more parts and the at least one task, or a combination thereof.

7. A method of claim 1, wherein the apparatus is a server or a platform that communicates with the user devices via the one or more networks.

8. A method of claim 1, wherein the apparatus is included in one of the user devices communicating with the other one or more of the user devices via one or more short range communication networks included in the one or more networks.

9. A method of claim 8, wherein the one or more short range communication networks are established via tapping the one user device with the other one or more user devices.

10. An apparatus comprising:

at least one processor; and

at least one memory including computer program code for one or more programs,

the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following,

initiate one or more indications of one or more parts in a plurality of augmented reality user interfaces of respective user devices, wherein the one or more parts are associated with at least one task associated with a vehicle to be performed via coordinated actions of users of the respective user devices, and wherein each of the augmented reality user interfaces comprises an electronic display of a camera;

initiate one or more guides for alignment of the one or more indications with the one or more parts respectively in the augmented reality user interfaces corresponding to the coordinated actions to be performed by the users, wherein the alignment is realized by movement of the respective augmented reality user interface to align the one or more indications with the one or more parts located in real space, and wherein the electronic display allows the respective user to view the augmented reality user interface during the alignment and while performing the coordinated actions;

process sensor information, telematics information, or a combination thereof collected at the user devices, the vehicle, or a combination thereof to determine the coordinated actions;

initiate, in response to the sensor information, the telematics information, or a combination thereof, a presentation in the electronic display of a tutorial video via one or more networks, wherein the tutorial video is a step by step guide to perform the coordinated actions as steps; and

upon determining based on the sensor information, the telematics information, or a combination thereof that one step shown on one of the user devices is completed by one of the users that uses the user device, initiate real-time skipping the step from the presentation of the tutorial video on the remaining one or more user devices and presenting a next step of the tutorial video on the remaining one or more user devices,

wherein the different steps of the tutorial video are presented among the user devices based, at least in part, on the real-time skipping.

11. An apparatus of claim 10, wherein the apparatus is further caused to:

process a field of view of the respective augmented reality user interface to cause, at least in part, a recognition of the one or more parts, the at least one task, or a combination thereof.

12. An apparatus of claim 11, wherein the apparatus is further caused to:

determine one or more reference objects associated with the field of view, the at least one task, or a combination thereof; and

determine the one or more guides based, at least in part, on the one or more reference objects.

13. An apparatus of claim 12, wherein the one or more guides include, at least in part, one or more outline representations of the one or more reference objects.

14. An apparatus of claim 10, wherein the one or more guides indicate, at least in part, a direction, an orientation, or a combination thereof to move the respective user device presenting the respective augmented reality user interface for aligning the one or more indications with the one or more parts.

15. An apparatus of claim 10, wherein the apparatus is further caused to:

determine an input for specifying the one or more parts, the at least one task, an association between the one or more parts and the at least one task, or a combination thereof.

16. An apparatus of claim 10, wherein the apparatus is a server or a platform that communicates with the user devices via the one or more networks.

17. An apparatus of claim 10, wherein the apparatus is included in one of the user devices communicating with the other one or more of the user devices via one or more short range communication networks included in the one or more networks.

18. An apparatus of claim 17, wherein the one or more short range communication networks are established via tapping the one user device with the other one or more user devices.