Visually enhanced tactile feedback

• US 9,971,406 B2
• Filed: 12/05/2014
• Issued: 05/15/2018
• Est. Priority Date: 12/05/2014
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bombacino, Vinicio, Milanese, Nicola, Pizzutilo, Riccardo, Tortosa, Andrea
• Primary Examiner(s): Nabi, Reza
• Assistant Examiner(s): Rodriguez, Daniel
• Application Number:
  US14/561,490
• Publication Number:
  US 20160162023A1
• Time in Patent Office: 1,257 Days
• Field of Search: 715702
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G09B 21/005 Details of specially-adapted software to access information, e.g. to browse through hyperlinked information

Abstract

In an approach for visually enhancing tactile metadata, a computer receives an image on a first computing device. The computer selects an object from one or more objects depicted within the received image. The computer determines boundaries of the selected object. The computer assigns an object tag to the selected object within the determined boundaries, wherein the assigned object tag includes one or more keywords and terms describing the selected object. The computer assigns tactile metadata to the selected object within the determined boundaries based on one or more physical properties associated with the assigned object tag. The computer creates a visually enhanced image based on the assigned tactile metadata, wherein the assigned tactile metadata includes one or more physical properties associated with the assigned object tag capable of being represented visually.

Claims

1. A computer program product for visually enhancing tactile metadata, the computer program product comprising:

one or more computer readable storage media and program instructions stored on the one or more computer readable storage media, the program instructions comprising;

program instructions to receive an image on a first computing device;

program instructions to select an object from one or more objects depicted within the received image;

program instructions to determine boundaries of the selected object;

program instructions to determine a type of object associated with the selected object;

program instructions to assign an object tag to the selected object within the determined boundaries based on the determined type of object, wherein the assigned object tag includes one or more keywords and terms describing the selected object;

program instructions to assign tactile metadata to the selected object within the determined boundaries based on one or more physical properties associated with the assigned object tag;

program instructions to create a visually enhanced image based on the assigned tactile metadata, wherein the assigned tactile metadata includes the one or more physical properties associated with the assigned object tag capable of being represented visually, wherein the visually enhanced image is in a three dimensional plane, wherein the visually enhanced image includes a deformation that is a softness property that transforms a material due to an applied pressure, wherein a resumption includes a memory property that allows the material to resume a previous shape after the deformation in the three dimensional plane;

create an executable application based on the created visually enhanced image, wherein the created executable application includes an image viewing application capable of displaying tactile metadata in response to user interaction with the created visually enhanced image;

program instructions to receive the created visually enhanced image;

program instructions to extract one or more objects within the received visually enhanced image with the assigned tactile metadata;

program instructions to create a table based on the extracted one or more objects within the received visually enhanced image with the assigned tactile metadata;

program instructions to determine whether an object within the extracted one or more objects does not include assigned tactile metadata for a type of material;

responsive to determining that the object within the extracted one or more objects does not include assigned tactile metadata for a type of material, program instruction to identify an entry within the created table that corresponds to the object within the extracted one or more objects that does not include assigned tactile metadata for each physical property;

program instructions to determine one or more types of materials of the object based on the assigned object tag and a list of materials;

program instructions to analyze a resolution of the object;

program instructions to determine a type of material of the object from within the determined one or more types of materials based on the analyzed resolution of the object;

program instructions to assign the determined material to the object within the received created visually enhanced image and to an entry within the associated created table associated with the object;

program instructions to identify boundaries for each of the one or more objects within the created table for visual enhancements; and

program instructions to assign tactile metadata to interior pixels associated with the identified boundaries for each of the one or more objects within the created table for visual enhancements.

2. The computer program product of claim 1, wherein determining the boundaries of the selected object further comprises one or more of the following program instructions, stored on the one or more computer readable storage media, to:

receive coordinates identifying the boundaries of the selected object; and

utilize image processing techniques to determine the boundaries of the selected object.

3. The computer program product of claim 1, wherein assigning tactile metadata to the selected object within the determined boundaries based on one or more physical properties associated with the assigned object tag further comprises program instructions, stored on the one or more computer readable storage media, to:

receive a selection of one or more physical properties associated with the assigned object tag, wherein the received selection of one or more physical properties includes one or more of the following;

a type of material and an elasticity; and

assign the received selection of one or more physical properties associated with the assigned object tag to the selected object within the determined boundaries.

4. The computer program product of claim 1 further comprising program instructions, stored on the one or more computer readable storage media, to:

install the created executable application on a second computing device;

display the created visually enhanced image on the second computing device through the installed executable application;

determine whether a selection is made within the created visually enhanced image;

responsive to determining the selection is made within the created visually enhanced image, determine a first location associated with the selection;

determine whether the determined first location of the selection includes tactile metadata; and

responsive to determining the first location of the selection includes the tactile metadata, display a shape around the determined first location based on one or more visual enhancements associated with the tactile metadata associated with the determined first location.

5. The computer program product of claim 4, wherein displaying the shape around the determined first location based on one or more visual enhancements comprises program instructions, stored on the one or more computer readable storage media, to:

display an elasticity associated with the determined first location as the deformation and resumption of the enhanced image over a time based on one or more elasticity values included in the tactile metadata associated with the determined first location.

6. The computer program product of claim 4 further comprising program instructions, stored on the one or more computer readable storage media, to:

determine whether the determined first location made within the created visually enhanced image includes a continuous movement from the determined first location to a second location;

responsive to determining the determined first location made within the created visually enhanced image includes a continuous movement from the determined first location to the second location, determine a trajectory corresponding to the continuous movement; and

display an animation in the determined trajectory based on tactile metadata between the determined first location and the second location.

7. The computer program product of claim 1, wherein to assign the tactile metadata to the interior pixels associated with the identified boundaries for each of the one or more objects within the created table for visual enhancements further comprises program instructions, stored on the one or more computer readable storage media, to:

determine whether the object includes a range of values associated with the elasticity and the temperature;

responsive to determining that the object includes a range of values associated with the elasticity and the temperature, calculate a varying level of elasticity and temperature for the interior pixels within the identified boundaries; and

assign the varying level of elasticity and temperature to the interior pixels based on a starting point and an ending point.

8. A computer system for visually enhancing tactile metadata, the computer system comprising:

one or more computer readable storage media and program instructions stored on the one or more computer readable storage media, the program instructions comprising;

program instructions to receive an image on a first computing device;

program instructions to select an object from one or more objects depicted within the received image;

program instructions to determine boundaries of the selected object;

program instructions to determine a type of object associated with the selected object;

program instructions to assign an object tag to the selected object within the determined boundaries based on the determined type of object, wherein the assigned object tag includes one or more keywords and terms describing the selected object;

program instructions to assign tactile metadata to the selected object within the determined boundaries based on one or more physical properties associated with the assigned object tag;

program instructions to create a visually enhanced image based on the assigned tactile metadata, wherein the assigned tactile metadata includes the one or more physical properties associated with the assigned object tag capable of being represented visually, wherein the visually enhanced image is in a three dimensional plane, wherein the visually enhanced image includes a deformation that is a softness property that transforms a material due to an applied pressure, wherein a resumption includes a memory property that allows the material to resume a previous shape after the deformation in the three dimensional plane;

program instructions to create an executable application based on the created visually enhanced image, wherein the created executable application includes an image viewing application capable of displaying tactile metadata is response to user interaction with the created visually enhanced image;

program instructions to receive the created visually enhanced image;

program instructions to extract one or more objects within the received visually enhanced image with the assigned tactile metadata;

program instructions to create a table based on the extracted one or more objects within the received visually enhanced image with the assigned tactile metadata;

program instructions to determine whether an object within the extracted one or more objects does not include assigned tactile metadata for a type of material;

responsive to determining that the object within the extracted one or more objects does not include assigned tactile metadata for a type of material, program instructions to identify an entry within the created table that corresponds to the object within the extracted one or more objects that does not include assigned tactile metadata for each physical property;

program instructions to determine one or more types of materials of the object based on the assigned object tag and a list of materials;

program instructions to analyze a resolution of the object;

program instructions to determine a type of material of the object from within the determined one or more types of materials based on the analyzed resolution of the object;

program instructions to assign the determined material to the object within the received created visually enhanced image and to an entry within the associated created table associated with the object;

program instructions to identify boundaries for each of the one or more objects within the created table for visual enhancements; and

program instructions to assign tactile metadata to interior pixels associated with the identified boundaries for each of the one or more objects within the created table for visual enhancements.

9. The computer system of claim 8, wherein assigning tactile metadata to the selected object within the determined boundaries based on one or more physical properties associated with the assigned object tag further comprises program instructions, stored on the one or more computer readable storage media, to:

receive a selection of one or more physical properties associated with the assigned object tag, wherein the received selection of one or more physical properties includes one or more of the following;

a type of material and an elasticity; and

assign the received selection of one or more physical properties associated with the assigned object tag to the selected object within the determined boundaries.

10. The computer system of claim 8 further comprising program instructions, stored on the one or more computer readable storage media, to:

install the created executable application on a second computing device;

display the created visually enhanced image on the second computing device through the installed executable application;

determine whether a selection is made within the created visually enhanced image;

responsive to determining the selection is made within the created visually enhanced image, determine a first location associated with the selection;

determine whether the determined first location of the selection includes tactile metadata; and

responsive to determining the first location of the selection includes the tactile metadata, display a shape around the determined first location based on one or more visual enhancements associated with the tactile metadata associated with the determined first location.

11. The computer system of claim 10, wherein displaying the shape around the determined first location based on one or more visual enhancements program instructions, stored on the one or more computer readable storage media, to:

display an elasticity associated with the determined first location as the deformation and resumption of the enhanced image over a time based on one or more elasticity values included in the tactile metadata associated with the determined first location.

12. The computer system of claim 10 further comprising program instructions, stored on the one or more computer readable storage media, to:

determine whether the determined first location made within the created visually enhanced image includes a continuous movement from the determined first location to a second location;

responsive to determining the determined first location made within the created visually enhanced image includes a continuous movement from the determined first location to the second location, determine a trajectory corresponding to the continuous movement; and

display an animation in the determined trajectory based on tactile metadata between the determined first location and the second location.

13. The computer system of claim 8, wherein to assign the tactile metadata to the interior pixels associated with the identified boundaries for each of the one or more objects within the created table for visual enhancements further comprises program instructions, stored on the one or more computer readable storage media, to:

determine whether the object includes a range of values associated with the elasticity and the temperature;

responsive to determining that the object includes a range of values associated with the elasticity and the temperature, calculate a varying level of elasticity and temperature for the interior pixels within the identified boundaries; and

assign the varying level of elasticity and temperature to the interior pixels based on a starting point and an ending point.