Cognitive display interface for augmenting display device content within a restricted access space based on user input

• US 10,831,261 B2
• Filed: 03/05/2019
• Issued: 11/10/2020
• Est. Priority Date: 03/05/2019
• Status: Active Grant

• Current Assignee: Kyndryl Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Mukherjea, Sougata, Kolluri Venkata Sesha, Saiprasad, Nagar, Seema, Dey, Kuntal
• Primary Examiner(s): Wills-Burns, Chineyere D
• Application Number:
  US16/292,626
• Publication Number:
  US 20200285303A1
• Time in Patent Office: 616 Days
• Field of Search: 345173
• CPC Class Codes: G06F 16/2457 with adaptation to user needs
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G09G 2358/00 Arrangements for display data security
• CPC Class Codes: G09G 2370/02 Networking aspects

Abstract

A cognitive display interface operatively couples to a display device and provides augmentation data for display to a user within a restricted access space based on user input. The interface receives user input and context-related data, with the user providing the user input within the space, and one or more sensors providing the context-related data. The interface analyzes the context-related sensor data to determine a location and an orientation of the user within the space at the time of providing the user input, resolves context of the user input based, at least in part, on the determined location and orientation of the user within the space, and determines a search query from the input and the resolved context. A response to the search query is generated based, at least in part, on searching a database(s), and the interface provides a response as augmentation data for rendering on the display device.

Claims

1. A method comprising:

providing a cognitive display interface to execute on a computer system and operatively couple to a display device and a user input component, the user input component being an electronic component separate from the display device, and the cognitive display interface to provide augmentation data to the display device for display to a user based on a user input, the cognitive display interface;

receiving a registration of the user input component and the display device for an interface session, and during the interface session, the cognitive display interface;

receiving the user input and context-related data associated with the user input, the user providing the user input using the user input component within a restricted access space, the restricted access space including one or more sensors, the one or more sensors providing, at least in part, the context-related data;

analyzing the context-related data to determine, at least in part therefrom, a location and an orientation of the user within the restricted access space at the time of the user providing the user input;

resolving context of the user input based, at least in part, on the determined location and orientation of the user within the restricted access space at the time of providing the user input;

determining a search query from, at least in part, the user input and the resolved context of the user input;

generating a response to the search query based, at least in part, on searching one or more databases; and

providing the response as augmentation data for rendering on the display device for display to the user.

2. The method of claim 1, wherein determining the search query by the cognitive display interface is further with reference to received data indicative of location of one or more objects within the restricted access space at the time of the user providing the user input.

3. The method of claim 1, wherein the user input component comprises an electronic textile, the electronic textile being worn by the user.

4. The method of claim 1, wherein the user input component includes a plurality of touch-sensitive electronic sensors, the user input being provided by user contact with one or more sensors of the plurality of touch-sensitive sensors while writing on the user input component.

5. The method of claim 1, wherein providing the response as augmentation data occurs in addition to display of an existing image on the display device.

6. The method of claim 1, wherein the display device lacks direct user control of display content, and is a device selected from the group consisting of an augmented content device, an augmented reality device, and a virtual reality device.

7. The method of claim 6, wherein the display device is worn by the user.

8. The method of claim 1, wherein the restricted access space is a data center installation.

9. A system comprising:

a memory; and

a processor in communication with the memory, wherein the system is configured to perform a method comprising;

providing a cognitive display interface to execute on a computer system and operatively couple to a display device and a user input component, the user input component being an electronic component separate from the display device, and the cognitive display interface to provide augmentation data to the display device for display to a user based on a user input, the cognitive display interface;

receiving a registration of the user input component and the display device for an interface session, and during the interface session, the cognitive display interface;

receiving the user input and context-related data associated with the user input, the user providing the user input using the user input component within a restricted access space, the restricted access space including one or more sensors, the one or more sensors providing, at least in part, the context-related data;

analyzing the context-related data to determine, at least in part therefrom, a location and an orientation of the user within the restricted access space at the time of the user providing the user input;

resolving context of the user input based, at least in part, on the determined location and orientation of the user within the restricted access space at the time of providing the user input;

determining a search query from, at least in part, the user input and the resolved context of the user input;

generating a response to the search query based, at least in part, on searching one or more databases; and

providing the response as augmentation data for rendering on the display device for display to the user.

10. The computer system of claim 9, wherein determining the search query by the cognitive display interface is further with reference to received data indicative of location of one or more objects within the restricted access space at the time of the user providing the user input.

11. The computer system of claim 9, wherein the user input component comprises an electronic textile, the electronic textile being worn by the user.

12. The computer system of claim 9, wherein the user input component includes a plurality of touch-sensitive electronic sensors, the user input being provided by user contact with one or more sensors of the plurality of touch-sensitive sensors while writing on the user input component.

13. The computer system of claim 9, wherein providing the response as augmentation data occurs in addition to display of an existing image on the display device.

14. The computer system of claim 9, wherein the display device lacks direct user control of display content, and is a device selected from the group consisting of an augmented content device, an augmented reality device, and a virtual reality device.

15. A computer program product comprising:

a non-transitory computer-readable storage medium readable by a processing circuit and storing instructions for execution by the processing circuit for performing a method comprising;

providing a cognitive display interface to execute on a computer system and operatively couple to a display device and a user input component, the user input component being an electronic component separate from the display device, and the cognitive display interface to provide augmentation data to the display device for display to a user based on a user input, the cognitive display interface;

receiving a registration of the user input component and the display device for an interface session, and during the interface session, the cognitive display interface;

receiving the user input and context-related data associated with the user input, the user providing the user input using the user input component within a restricted access space, the restricted access space including one or more sensors, the one or more sensors providing, at least in part, the context-related data;

analyzing the context-related data to determine, at least in part therefrom, a location and an orientation of the user within the restricted access space at the time of the user providing the user input;

resolving context of the user input based, at least in part, on the determined location and orientation of the user within the restricted access space at the time of providing the user input;

determining a search query from, at least in part, the user input and the resolved context of the user input;

generating a response to the search query based, at least in part, on searching one or more databases; and

providing the response as augmentation data for rendering on the display device for display to the user.

16. The computer program product of claim 15, wherein determining the search query by the cognitive display interface is further with reference to received data indicative of location of one or more objects within the restricted access space at the time of the user providing the user input.