METHOD AND APPARATUS FOR PRESENTING TASK-RELATED OBJECTS IN AN AUGMENTED REALITY DISPLAY

• US 20180275848A1
• Filed: 05/25/2018
• Published: 09/27/2018
• Est. Priority Date: 02/22/2013
• Status: Active Grant

• Current Assignee: HERE Global B.V. (HERE Holding Corporation)
• Original Assignee: HERE Global B.V. (HERE Holding Corporation)
• Inventors: TUUKKANEN, Marko Tapio
• Granted Patent: US 10,338,786 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range

Abstract

An approach is provided for causing a presentation in an augmented reality user interface for user guidance. The approach involves causing a presentation of one or more indications of one or more parts in an augmented reality user interface, wherein the one or more parts are associated with at least one task. The approach also involves causing a presentation of one or more guides for aligning the one or more indications with the one or more parts in the augmented reality user interface.

Claims

1. A method comprising:

initiating, by an apparatus, indications of one or more parts in a plurality of augmented reality user interfaces of respective user devices, wherein the one or more parts are associated with at least one task associated with at least one vehicle to be performed via coordinated actions of users of the respective user devices, and wherein each of the augmented reality user interfaces comprises an electronic display of a camera;

initiating, by the apparatus, one or more guides for alignment of the one or more indications with the one or more parts respectively in the augmented reality user interfaces corresponding to one or more of the coordinated actions to be performed by the respective user device, wherein the alignment is realized by movement of the respective augmented reality user interface to align the one or more indications with the one or more parts located in real space, and wherein the electronic display allows the respective user to view the augmented reality user interface during the alignment and while performing the one or more of the coordinated actions;

processing, by the apparatus, sensor information, telematics information, or a combination thereof collected at the user devices, the at least one vehicle, or a combination thereof to determine the one or more of the coordinated actions; and

initiating, by the apparatus in response to the sensor information, the telematics information, or a combination thereof, real-time presentation in the electronic display of the respective user device of a tutorial video as a step by step guide to perform the one or more of the coordinated actions, wherein the step by step guide, the one or more of the coordinated actions, or a combination thereof are different among the user devices.

2. A method of claim 1, further comprising:

processing a field of view of the respective augmented reality user interface to cause, at least in part, a recognition of the one or more parts, the at least one task, or a combination thereof.

3. A method of claim 2, further comprising:

determining one or more reference objects associated with the field of view, the at least one task, or a combination thereof; and

determining the one or more guides based, at least in part, on the one or more reference objects.

4. A method of claim 3, wherein the one or more guides include, at least in part, one or more outline representations of the one or more reference objects.

5. A method of claim 1, wherein the one or more guides indicate, at least in part, a direction, an orientation, or a combination thereof to move the respective user device presenting the respective augmented reality user interface for aligning the one or more indications with the one or more parts.

6. A method of claim 1, further comprising:

determining an input for specifying the one or more parts, the at least one task, an association between the one or more parts and the at least one task, or a combination thereof.

7. A method of claim 1, further comprising:

presenting information related to the one or more parts, the at least one task, or a combination thereof in association with the one or more indications, wherein the information includes, at least in part, one or more instructions for performing at least a portion of the at least one task.

8. A method of claim 1, further comprising:

initiating, in response to determining based on the sensor information, the telematics information, or a combination thereof that one or more steps to perform the one or more of the coordinated actions are completed by the respective user, real-time skipping the one or more steps from the presentation in the electronic display of the respective user device of the tutorial video.

9. A method of claim 1, wherein the apparatus is a server or a platform communicates with the user devices via one or more networks.

10. A method of claim 1, wherein the apparatus includes one or more of the user devices communicate via one or more short range communication networks.

11. An apparatus comprising:

at least one processor; and

at least one memory including computer program code for one or more programs,

the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following,

initiate indications of one or more parts in a plurality of augmented reality user interfaces of respective user devices, wherein the one or more parts are associated with at least one task associated with at least one vehicle to be performed via coordinated actions of users of the respective user devices, and wherein each of the augmented reality user interfaces comprises an electronic display of a camera;

initiate one or more guides for alignment of the one or more indications with the one or more parts respectively in the augmented reality user interfaces corresponding to one or more of the coordinated actions to be performed by the respective user device, wherein the alignment is realized by movement of the respective augmented reality user interface to align the one or more indications with the one or more parts located in real space, and wherein the electronic display allows the respective user to view the augmented reality user interface during the alignment and while performing the one or more of the coordinated actions;

process sensor information, telematics information, or a combination thereof collected at the user devices, the at least one vehicle, or a combination thereof to determine the one or more of the coordinated actions; and

initiate, in response to the sensor information, the telematics information, or a combination thereof, real-time presentation in the electronic display of the respective user device of a tutorial video as a step by step guide to perform the one or more of the coordinated actions, wherein the step by step guide, the one or more of the coordinated actions, or a combination thereof are different among the user devices.

12. An apparatus of claim 11, wherein the apparatus is further caused to:

process a field of view of the respective augmented reality user interface to cause, at least in part, a recognition of the one or more parts, the at least one task, or a combination thereof.

13. An apparatus of claim 12, wherein the apparatus is further caused to:

determine one or more reference objects associated with the field of view, the at least one task, or a combination thereof; and

determine the one or more guides based, at least in part, on the one or more reference objects.

14. An apparatus of claim 13, wherein the one or more guides include, at least in part, one or more outline representations of the one or more reference objects.

15. An apparatus of claim 11, wherein the one or more guides indicate, at least in part, a direction, an orientation, or a combination thereof to move the respective user device presenting the respective augmented reality user interface for aligning the one or more indications with the one or more parts.

16. An apparatus of claim 11, wherein the apparatus is further caused to:

determine an input for specifying the one or more parts, the at least one task, an association between the one or more parts and the at least one task, or a combination thereof.

17. An apparatus of claim 11, wherein the apparatus is further caused to:

present information related to the one or more parts, the at least one task, or a combination thereof in association with the one or more indications, wherein the information includes, at least in part, one or more instructions for performing at least a portion of the at least one task.

18. An apparatus of claim 11, wherein the apparatus is further caused to:

initiate, in response to determining based on the sensor information, the telematics information, or a combination thereof that one or more steps to perform the one or more of the coordinated actions are completed by the respective user, real-time skipping the one or more steps from the presentation in the electronic display of the respective user device of the tutorial video.

19. An apparatus of claim 11, wherein the apparatus is a server or a platform communicates with the user devices via one or more networks.

20. An apparatus of claim 11, wherein the apparatus includes one or more of the user devices communicate via one or more short range communication networks.