Path visualization for augmented reality display device based on received data and probabilistic analysis

• US 9,852,547 B2
• Filed: 03/23/2015
• Issued: 12/26/2017
• Est. Priority Date: 03/23/2015
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bostick, James E., Ganci, Jr., John M., Rakshit, Sarbajit K., Trim, Craig M.
• Primary Examiner(s): Brier, Jeffery A
• Application Number:
  US14/665,423
• Publication Number:
  US 20160284125A1
• Time in Patent Office: 1,009 Days
• CPC Class Codes: G01C 21/3647 Guidance involving output of stored or live camera images or video streams
• CPC Class Codes: G01C 21/365 Guidance using head up displays or projectors, e.g. virtual vehicles or arrows projected on the windscreen or on the road itself
• CPC Class Codes: G01C 21/3885 Transmission of map data to client devices; Reception of map data by client devices
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0183 Adaptation to parameters characterising the motion of the vehicle
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06Q 50/01 Social networking
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A computing device receives data regarding a destination, a current GPS location, map data of the current location and destination, and a direction and angle of view of an augmented reality display device. The computing device accesses social media content related to an area that includes the location and the destination, and analyzes the content to determine recent conditions and attributes associated with the location and destination area. The computing device determines a path from the location to the destination based on the map data, the current location and destination, and the social media content. The computing device sends information to display a virtual image of the path overlaid on a real-world view of the augmented reality display device, based on the direction of view and the incline level of view, of the augmented reality display device.

Claims

1. A method comprising:

a computing device operatively coupled to an augmented reality display device receiving a location of a user selected destination, a current location as determined by a global positioning service (GPS), map data corresponding to the current location and the user selected destination, a direction of view of the augmented reality display device, and a viewing angle of the augmented reality display device;

the computing device determining a path which connects the current location to the user selected destination, based on the map data that corresponds to the current location and the user selected destination, wherein the map data includes terrain and waterway information of an area including the current location and the user selected destination, and the path;

the computing device receiving social media content, wherein the social media content that is received relates to the area that includes the current location, the user selected destination, and the path connecting the current location and the user selected destination;

the computing device determining conditions and attributes associated with the area that includes the current location, the user selected destination, and the path, wherein the conditions and attributes are based on an analysis of the social media content that is received;

responsive to determining a predicted change to the conditions and attributes of the area that includes the path, the computing device determining whether the predicted change to the conditions and attributes of the area that includes the current location, the user selected destination, and the path, affects a difficulty level of the path, based on a semantic analysis and a sentiment analysis included within the analysis of the social media content that is received;

the computing device receiving a selection to display information indicating the conditions and attributes of the area that includes the current location, the user selected destination, and the path wherein the selection is based on a user gesture detected by a camera operatively coupled to and aligned with the direction of view of the augmented reality display device; and

responsive to the computing device receiving the user gesture, the computing device displaying the information indicating the conditions and attributes, and the predicted change to the conditions and attributes, of the area that includes the current location, the user selected destination, and the path, based on the analysis of the social media content, wherein the augmented reality display device displays the information indicating the conditions and attributes, the predicted change to the conditions and attributes affecting the difficulty level of the path, and a virtual image of the path, as a virtual overlay over a real-world view as seen through the augmented reality display device.

2. The method of claim 1, wherein the computing device receiving at least a portion of the information indicating the conditions of the path over a telecommunications network.

3. The method of claim 1, further comprising:

the computing device receiving input from a camera aligned with the direction of view and the incline angle of view of the augmented reality display device, wherein the input includes a recording by the camera of a new path traversed by a first person; and

the computing device initiating a display of a virtual image of the new path overlaid on a real-world view of the augmented reality display device of a second person, based on the input of the recording of the new path traversed by the first person.

4. The method of claim 1, wherein analyzing the social media content, further comprises:

the computing device performing a syntactical analysis on the social media content in which a word type of each word within the social media content, a word position within the social media content, and phrases of the social media content are determined;

the computing device performing a semantic analysis in which a meaning associated with one or a combination of words of the social media content is determined;

the computing device identifying keywords and key phrases of the social media content relevant to the area of the current location and the user selected destination, based on the syntactical analysis and the semantic analysis;

the computing device performing the sentiment analysis of the social media content to determine an intent of the social media content relevant to the difficulty level of the path based on the conditions and attributes of the current location, the user selected destination, and the path; and

the computing device determining the information indicating the conditions and attributes of the area that includes the current location, the user selected destination, and the path, based on results of the syntactical analysis, semantic analysis, and sentiment analysis applied to the social media content that is received.

5. The method of claim 1, wherein the computing device and the augmented reality display device are configured for option selection during operation by a user without requiring direct contact by the hands of the user of either or both of the augmented reality display device and the computing device, wherein the option selection is made by one or a combination of a predefined user movement from a group of:

a hand gesture, an arm gesture, and a head-movement gesture, as detected by the camera included in the augmented reality display device.

6. The method of claim 1, wherein displaying the information indicating the conditions and attributes of the path to the augmented reality display device includes providing virtual overlays to the augmented reality display device of an indicator of dangerous conditions for the area of the path, and a separate indicator of dangerous conditions of the path, wherein an indicator can be one or a combination of a color coding, a symbol, and a text-based message.

7. The method of claim 1, wherein displaying the information indicating the conditions and the attributes of the path to the augmented reality display device includes providing virtual overlays to the augmented reality display device of an indicator of warning for areas of known accidents.

8. The method of claim 1, further comprising:

the computing device displaying the virtual image of the path, and the area of the path from the current location to the user selected destination, removing the display of real-world objects obstructing a view of the path.

9. A computer program product comprising:

a computer-readable hardware storage device having program instructions embodied therewith, wherein the program instructions are executable by a computer processor, the program instructions comprising;

program instructions for a computing device operatively coupled to an augmented reality display device to receive a location of a user selected destination, a current location as determined by a global positioning service (GPS), map data corresponding to the current location and the user selected destination, a direction of view of the augmented reality display device, and a viewing angle of the augmented reality display device;

program instructions to determine a path which connects the current location to the user selected destination, based on the map data that corresponds to the current location and the user selected destination, wherein the map data includes topography and feature information of an area including the current location and the user selected destination, and the path;

program instructions to receive social media content, wherein the social media content that is received relates to the area that includes the current location, the user selected destination, and the path connecting the current location and the user selected destination;

program instructions to determine conditions and attributes associated with the area that includes the current location, the user selected destination, and the path, wherein the conditions and the attributes are based on an analysis of the social media content that is received;

responsive to determining a predicted change to the conditions and attributes of the area that includes the path, program instructions to determine whether the predicted change to the conditions and attributes of the area that includes the current location, the user selected destination, and the path, affects a difficulty level of the path, based on a semantic analysis and a sentiment analysis included within the analysis of the social media content that is received;

program instructions to receive a selection to display information indicating the conditions and attributes of the area that includes the current location, the user selected destination, and the path wherein the selection is based on a user gesture detected by a camera operatively coupled to and aligned with the direction of view of the augmented reality display device; and

responsive to the computing device receiving the user gesture, program instructions to display the information indicating the conditions and attributes, and the predicted change to the conditions and attributes, of the area that includes the current location, the user selected destination, and the path, based on the analysis of the social media content, wherein the augmented reality display device displays the information indicating the conditions and attributes, the predicted change to the conditions and attributes affecting the difficulty level of the path, and a virtual image of the path, as a virtual overlay over a real-world view as seen through the augmented reality display device.

10. The computer program product of claim 9, wherein analyzing the social media content, further comprises:

program instructions to perform a syntactical analysis on the social media content in which a word type of each word within the social media content, a word position within the social media content, and phrases of the social media content are determined;

program instructions to perform a semantic analysis in which a meaning associated with one or a combination of words of the social media content is determined;

program instructions to identify keywords and key phrases of the social media content relevant to the area of the current location and the user selected destination, based on the syntactical analysis and the semantic analysis; and

program instructions to perform the sentiment analysis of the social media content to determine an intent of the social media content relevant to the difficulty level of the path based on the conditions and attributes of the current location, the user selected destination, and the path;

program instructions to determine the information indicating the conditions and attributes of the area that includes the current location, the user selected destination, and the path, based on results of the syntactical analysis, the semantic analysis and the sentiment analysis applied to the social media content that is received.

11. The computer program product of claim 9, wherein program instructions to display of the information indicating the conditions and attributes of the path to the augmented reality display device includes providing virtual overlays to the augmented reality display device of an indicator of dangerous conditions for the area of the path, and a separate indicator of dangerous conditions of the path, wherein an indicator can be one or a combination of a color coding, a symbol, and a text-based message.

12. The computer program product of claim 9, wherein program instructions to display of the information indicating the conditions and the attributes of the path to the augmented reality display device includes providing virtual overlays to the augmented reality display device of an indicator of warning for areas of known accidents.

13. A computer system for displaying a virtual path through an augmented reality display device, the computer system comprising:

one or more computer processors;

one or more computer readable, non-transitory storage media; and

program instructions stored on the computer readable, media for execution by at least one of the one or more processors, the program instructions comprising;

program instructions for a computing device operatively coupled to an augmented reality display device to receive a location of a user selected destination, a current location as determined by a global positioning service (GPS), map data corresponding to the current location and the user selected destination, a direction of view of the augmented reality display device, and a viewing angle of the augmented reality display device;

program instructions to determine a path which connects the current location to the user selected destination, based on the map data that corresponds to the current location and the user selected destination, wherein the map data includes topography and feature information of an area including the current location and the user selected destination, and the path;

program instructions to receive social media content, wherein the social media content that is received relates to the area that includes the current location, the user selected destination, and the path connecting the current location and the user selected destination;

program instructions to determine conditions and attributes associated with the area that includes the current location, the user selected destination, and the path, wherein the conditions and the attributes are based on an analysis of the social media content that is received;

responsive to determining a predicted change to the conditions and attributes of the area that includes the path, program instructions to determine whether the predicted change to the conditions and attributes of the area that includes the current location, the user selected destination, and the path, affects a difficulty level of the path, based on a semantic analysis and a sentiment analysis included within the analysis of the social media content that is received;

program instructions to receive a selection to display information indicating the conditions and attributes of the area that includes the current location, the user selected destination, and the path wherein the selection is based on a user gesture detected by a camera operatively coupled to and aligned with the direction of view of the augmented reality display device; and

responsive to the computing device receiving the user gesture, program instructions to display the information indicating the conditions and attributes, and the predicted change to the conditions and attributes, of the area that includes the current location, the user selected destination, and the path, based on the analysis of the social media content, wherein the augmented reality display device displays the information indicating the conditions and attributes, the predicted change to the conditions and attributes affecting the difficulty level of the path, and a virtual image of the path, as a virtual overlay over a real-world view as seen through the augmented reality display device.

14. The computer system of claim 13, wherein program instructions to display of the information indicating the conditions and attributes of the path to the augmented reality display device includes providing virtual overlays to the augmented reality display device of an indicator of dangerous conditions for the area of the path, and a separate indicator of dangerous conditions of the path, wherein an indicator can be one or both of a color coding and a symbol.

15. The computer system of claim 13, wherein program instructions to display of the information indicating the conditions and the attributes of the path to the augmented reality display device includes providing virtual overlays to the augmented reality display device of an indicator of warning for areas of known accidents.