Providing augmented reality links to stored files

• US 10,620,817 B2
• Filed: 01/13/2017
• Issued: 04/14/2020
• Est. Priority Date: 01/13/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Green, Sophie D., Jansen, Grace, Protonotarios, Antony, Russell, James, Vlissidis, Alexandros Elio
• Primary Examiner(s): Baderman, Scott T
• Assistant Examiner(s): Velez-Lopez, Mario M
• Application Number:
  US15/406,219
• Publication Number:
  US 20180203579A1
• Time in Patent Office: 1,187 Days
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/168 Details of user interfaces specifically adapted to file systems, e.g. browsing and visualisation, 2d or 3d GUIs query results presentation G06F16/156
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06V 10/225 based on a marking or identifier characterising the area
• CPC Class Codes: G06V 20/20 in augmented reality scenes

Abstract

Augmented reality links to stored files. A marker of a real object or a real location as viewed by an augmented reality (AR) device of a computer system is stored, in which the marker is stored in a database with a link to a location of a file or a folder in a file system of the connectable computer system. Saving and retrieval of a file or a folder to and from the connectable computer system are enabled by user selection of the real object or the real location via user interaction with the AR device thereby detecting the marker and activating the link to the computer system to save a file or a folder to the location or to retrieve a file or a folder from the location.

Claims

1. A method of providing augmented reality links to stored files, said method comprising:

facilitating, via an augmented reality (AR) device, navigation in a computer system, the facilitating comprising;

creating a link to a file system object, the creating being based, in part, on a user viewing through the AR device a computer screen of a computer system while a file system object is being displayed by the computer screen, the AR device being coupled to the computer system using a two-way pairing, and the creating comprising;

displaying, on the AR device, an option to be selected by the user, the option enabling the user to create a marker of a real-world reference, the creating the marker causing an image, as seen through the AR device, to be superimposed onto the real-world reference, the marker being stored in a database of the computer system, the marker comprising a characteristic of the real-world reference;

based on the user selecting the option, displaying, on the AR device, a virtual icon representing the file system object being displayed by the computer screen, the virtual icon being overlaid, as seen through the AR device, on top of the computer screen, the virtual icon being linked to the file system object of the computer system; and

recognizing, by the AR device, a command made of user, the command causing the virtual icon to be moved from where it is overlaid over the computer screen to beyond the computer screen and dropped onto the image superimposed onto the real-world reference, the dropping causing the link to the file system object to be created.

2. The method as claimed in claim 1, wherein the link between the file system object and the marker, is stored in the database in a form of a file path to the location of the file system object in the computer system.

3. The method as claimed in claim 1, wherein the creating further comprises recognizing, based on the virtual icon being dropped onto the image superimposed on the real-world reference, the marker of the real-world reference and looking up the database to determine whether the link is provided for the marker.

4. The method as claimed in claim 1, wherein the facilitating further comprises:

retrieving, via the AR device, content of the file system object based on the link created, the retrieving comprising;

detecting, by the AR device, that the marker exists in the database, the detecting being based on recognizing a user movement, the user movement causing the AR device to compare characteristics of the real-world reference to the marker and, based on the characteristics of the marker matching the real-world reference, enabling the user to open the file system object via the AR device; and

opening, via the AR device, the file system object on at least one of the AR device or the computer screen, the opening being based on the virtual icon appearing, based on the detecting, and the user selecting the virtual icon, the selecting activating the link to the file system object.

5. The method as claimed in claim 4, wherein the opening the file system object includes connecting to the computer system via the two-way pairing based on a reference to the computer system in the link.

6. The method as claimed in claim 1, wherein the characteristic of the real-world reference includes a shape of the real-world reference.

7. The method as claimed in claim 1, wherein the characteristic of the real-world reference includes a real location based on coordinates referenced in the marker.

8. The method as claimed in claim 1, wherein the command includes a user action, the user action being selected from the group consisting of a gesture of the user, and a voice command of the user.

9. The method as claimed in claim 1, wherein the file system object comprises an object selected from the group consisting of an individual computer file in a format capable of being opened by a computer application, and a folder in the form of a group of files.

10. A system for providing augmented reality links to stored files, said system comprising:

a memory; and

a processor in communication with the memory, wherein the system is configured to perform a method, said method comprising;

facilitating, via an augmented reality (AR) device, navigation in a computer system, the facilitating comprising;

creating a link to a file system object, the creating being based, in part, on a user viewing through the AR device a computer screen of a computer system while a file system object is being displayed by the computer screen, the AR device being coupled to the computer system using a two-way pairing, and the creating comprising;

displaying, on the AR device, an option to be selected by the user, the option enabling the user to create a marker of a real-world reference, the creating the marker causing an image, as seen through the AR device, to be superimposed onto the real-world reference, the marker being stored storing, in a database of the computer system, the marker comprising a characteristic of the real-world reference;

based on the user selecting the option, displaying, on the AR device, a virtual icon representing the file system object being displayed by the computer screen, the virtual icon being overlaid, as seen through the AR device, on top of the computer screen, the virtual icon being linked to the file system object of the computer system; and

recognizing, by the AR device, a command of the user, the command causing the virtual icon to be moved from where it is overlaid over the computer screen to beyond the computer screen and dropped onto the image superimposed onto the real-world reference, the dropping causing the link to the file system object to be created.

11. The system as claimed in claim 10, wherein the link between the file system object and the marker, is stored in the database, and the link is saved in the database in a form of a file path to the location of the file system object in the computer system.

12. The system as claimed in claim 10, wherein the creating further comprises recognizing, based on the virtual icon being dropped onto the image superimposed on the real-world reference, the marker of the real-world reference and looking up the database to determine whether the link is provided for the marker.

13. The system as claimed in claim 10, wherein the characteristic of the real-world reference includes at least one of:

a shape of the real-world reference, and a real location based on coordinates referenced in the marker.

14. The system as claimed in claim 10, wherein the facilitating further comprises:

retrieving, via the AR device, content of the file system object based on the link created, the retrieving comprising;

detecting, by the AR device, that the marker exists in the database, the detecting being based on recognizing a user movement, the user movement causing the AR device to compare characteristics of the real-world reference to the marker and, based on the characteristics of the marker matching the real-world reference, enabling the user to open the file system object via the AR device; and

opening, via the AR device, the file system object on at least one of the AR device or the computer screen, the opening being based on the virtual icon appearing, based on the detecting, and the user selecting the virtual icon, the selecting activating the link to the file system object.

15. A computer program product for providing augmented reality links to stored files, said computer program product comprising:

a computer readable storage medium readable by a processing circuit and storing instructions for execution by the processing circuit for performing a method comprising;

facilitating, via an augmented reality (AR) device, navigation in a computer system, the facilitating comprising;

creating a link to a file system object, the creating being based, in part, on a user viewing through the AR device a computer screen of a computer system while a file system object is being displayed by the computer screen, the AR device being coupled to the computer system using a two-way pairing, and the creating comprising;

displaying, on the AR device, an option to be selected by the user, the option enabling the user to create a marker of a real-world reference, the creating the marker causing an image, as seen through the AR device, to be superimposed onto the real-world reference, the marker being stored in a database of the computer system, the marker comprising a characteristic of the real-world reference;

based on the user selecting the option, displaying, on the AR device, a virtual icon representing the file system object being displayed by the computer screen, the virtual icon being overlaid, as seen through the AR device, on top of the computer screen, the virtual icon being linked to the file system object of the computer system; and

recognizing, by the AR device, a command of the user, the command causing the virtual icon to be moved from where it is overlaid over the computer screen to beyond the computer screen and dropped onto the image superimposed onto the real-world reference, the dropping causing the link to the file system object to be created.

16. The computer program product as claimed in claim 15, wherein the link between the file system object and the marker is stored in the database, and the link is saved in the database in a form of a file path to the location of the file system object in the computer system.

17. The computer program product as claimed in claim 15, wherein the creating further comprises recognizing, based on the virtual icon being dropped onto the image superimposed on the real-world reference, the marker of the real-world reference and looking up the database to identify the link is provided for the marker.

18. The computer program product as claimed in claim 15, wherein the characteristic of the real-world reference includes at least one of:

a shape of the real-world reference, and a real location based on coordinates referenced in the marker.

19. The computer program product as claimed in claim 15, wherein the facilitating further comprises:

retrieving, via the AR device, content of the file system object based on the link created, the retrieving comprising;

detecting, by the AR device, that the marker exists in the database, the detecting being based on recognizing a user movement, the user movement causing the AR device to compare characteristics of the real-world reference to the marker and, based on the characteristics of the marker matching the real-world reference, enabling the user to open the file system object via the AR device; and

opening, via the AR device, the file system object on at least one of the AR device or the computer screen, the opening being based on the virtual icon appearing, based on the detecting, and the user selecting the virtual icon, the selecting activating the link to the file system object.