Visually enhanced tactile feedback

• US 10,055,020 B2
• Filed: 08/27/2015
• Issued: 08/21/2018
• Est. Priority Date: 12/05/2014
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bombacino, Vinicio, Milanese, Nicola, Pizzutilo, Riccardo, Tortosa, Andrea
• Primary Examiner(s): Nabi, Reza
• Assistant Examiner(s): Rodriguez, Daniel
• Application Number:
  US14/837,257
• Publication Number:
  US 20160162024A1
• Time in Patent Office: 1,090 Days
• Field of Search: 715702
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G09B 21/005 Details of specially-adapted software to access information, e.g. to browse through hyperlinked information

Abstract

In an approach for visually enhancing tactile metadata, a computer receives an image on a first computing device. The computer selects an object from one or more objects depicted within the received image. The computer determines boundaries of the selected object. The computer assigns an object tag to the selected object within the determined boundaries, wherein the assigned object tag includes one or more keywords and terms describing the selected object. The computer assigns tactile metadata to the selected object within the determined boundaries based on one or more physical properties associated with the assigned object tag. The computer creates a visually enhanced image based on the assigned tactile metadata, wherein the assigned tactile metadata includes one or more physical properties associated with the assigned object tag capable of being represented visually.

Claims

1. A method for visually enhancing tactile metadata, the method comprising:

receiving, by one or more computer processors, an image on a first computing device;

selecting, by one or more computer processors, an object from one or more objects depicted within the received image;

determining, by one or more computer processors, boundaries of the selected object;

assigning, by one or more computer processors, an object tag to the selected object within the determined boundaries based on the determined type of object, wherein the assigned object tag includes one or more keywords and terms describing the selected object;

assigning, by one or more computer processors, tactile metadata to the selected object within the determined boundaries based on one or more physical properties associated with the assigned object tag;

creating, by one or more computer processors, a visually enhanced image based on the assigned tactile metadata, wherein the assigned tactile metadata includes the one or more physical properties associated with the assigned object tag capable of being represented visually, wherein the visually enhanced image is in a three dimensional plane, wherein the visually enhanced image includes a deformation that is a softness property that transforms a material due to an applied pressure, wherein a resumption includes a memory property that allows the material to resume a previous shape after the deformation in the three dimensional plane;

creating, by one or more computer processors, an executable application based on the created visually enhanced image, wherein the created executable application includes an image viewing application capable of displaying tactile metadata in response to user interaction with the created visually enhanced image;

receiving, by one or more computer processors, the created visually enhanced image;

extracting, by one or more computer processors, one or more objects within the received visually enhanced image with the assigned tactile metadata;

creating, by one or more computer processors, a table based on the extracted one or more objects within the received visually enhanced image with the assigned tactile metadata;

determining, by one or more computer processors, whether an object within the extracted one or more objects does not include assigned tactile metadata for a type of material;

responsive to determining that the object within the extracted one or more objects does not include assigned tactile metadata for the type of material, identifying, by one or more computer processors, an entry within the created table that corresponds to the object within the extracted one or more objects that does not include assigned tactile metadata for each physical property;

determining, by one or more computer processors, one or more types of materials of the object based on the assigned object tag and a list of materials;

analyzing, by one or more computer processors, a resolution of the object;

determining, by one or more computer processors, a type of material of the object from within the determined one or more types of materials based on the analyzed resolution of the object;

assigning, by one or more computer processors, the determined material to the object within the received created visually enhanced image and to an entry within the associated created table associated with the object;

identifying, by one or more computer processors, boundaries for each of the one or more objects within the created table for visual enhancements; and

assigning, by one or more computer processors, tactile metadata to interior pixels associated with the identified boundaries for each of the one or more objects within the created table for visual enhancements.

2. The method of claim 1, wherein determining the boundaries of the selected object further comprises one or more of the following:

receiving, by one or more computer processors, coordinates identifying the boundaries of the selected object; and

utilizing, by one or more computer processors, image processing techniques to determine the boundaries of the selected object.

3. The method of claim 1, wherein assigning tactile metadata to the selected object within the determined boundaries based on one or more physical properties associated with the assigned object tag further comprises:

receiving, by one or more computer processors, a selection of one or more physical properties associated with the assigned object tag, wherein the received selection of one or more physical properties includes one or more of the following;

a type of material, an elasticity, and a temperature; and

assigning, by one or more computer processors, the received selection of one or more physical properties associated with the assigned object tag to the selected object within the determined boundaries.

4. The method of claim 1 further comprising:

installing, by one or more computer processors, the created executable application on a second computing device;

displaying, by one or more computer processors, the created visually enhanced image on the second computing device through the installed executable application;

determining, by one or more computer processors, whether a selection is made within the created visually enhanced image;

responsive to determining the selection is made within the created visually enhanced image, determining, by one or more computer processors, a first location associated with the selection;

determining, by one or more computer processors, whether the determined first location of the selection includes tactile metadata; and

responsive to determining the first location of the selection includes the tactile metadata, displaying, by one or more computer processors, a shape around the determined first location based on one or more visual enhancements associated with the tactile metadata associated with the determined first location.

5. The method of claim 4, wherein displaying the shape around the determined first location based on one or more visual enhancements comprises one or more of the following:

displaying, by one or more computer processors, a temperature associated with the determined first location as one or more colors based on one or more temperature values included in the tactile metadata associated with the first location; and

displaying, by one or more computer processors, an elasticity associated with the determined first location as a deformation and resumption of the enhanced image based on the one or more elasticity values included in the tactile metadata associated with the determined first location.

6. The method of claim 4 further comprising:

determining, by one or more computer processors, whether the determined first location made within the created visually enhanced image includes a continuous movement from the determined first location to a second location;

responsive to determining the determined first location made within the created visually enhanced image includes a continuous movement from the determined first location to the second location, determining, by one or more computer processors, a trajectory corresponding to the continuous movement; and

displaying, by one or more computer processors, an animation in the determined trajectory based on tactile metadata between the determined first location and the second location.