SYSTEM AND METHOD FOR PERCEIVING IMAGES WITH MULTIMODAL FEEDBACK

• US 20140049491A1
• Filed: 07/30/2013
• Published: 02/20/2014
• Est. Priority Date: 08/20/2012
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Nagar, Abhishek, Josiam, Kaushik, Dickerson, Andrew, Thompson, Timothy
• Granted Patent: US 9,280,206 B2
• Time in Patent Office: Days
• US Class Current: 345/173
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis

Abstract

A method, system and computer-readable medium for providing feedback effects for an image. The method includes identifying one or more features in an area of the image. The method also includes mapping the one or more identified features to at least one of multiple modalities of feedback effects. Additionally, the method includes generating parameters for feedback effect output and storing the generated parameters in association with the image such that, in response to coordinates of an input of a user corresponding to the area of the image, a user interface associated with the image is configured to provide at least one of the multiple modalities of feedback effects to the user. The multiple modalities of feedback effects include one or more visual feedback effects, one or more haptic feedback effects and one or more audio feedback effects.

Claims

1. A method for providing feedback effects for an image, the method comprising:

identifying one or more features in an area of the image;

mapping the one or more identified features to at least one of multiple modalities of feedback effects; and

generating parameters for feedback effect output and storing the generated parameters in association with the image such that, in response to coordinates of an input of a user corresponding to the area of the image, a user interface associated with the image is configured to provide at least one of the multiple modalities of feedback effects to the user.

2. The method of claim 1, wherein the multiple modalities of feedback effects comprise one or more visual feedback effects, one or more haptic feedback effects, and one or more audio feedback effects.

3. The method of claim 1 further comprising:

displaying the image using a touchscreen enabled device;

identifying coordinates of the image corresponding to a touch input of the user;

identifying a feedback effect mapped to an area of the image; and

providing the feedback effect using the touchscreen enabled device.

4. The method of claim 1, wherein the one or more identified features comprise a texture, an intensity, and a color associated with the area of the image

5. The method of claim 2 further comprising:

identifying a constraint on providing one or more of the multiple modalities of feedback effects to the user; and

constraining mapping of the one or more identified features to only unconstrained modalities of feedback effects.

6. The method of claim 5, wherein:

the constraint is a visually-impaired mode of operation, and

constraining mapping of the one or more identified features comprises constraining mapping of the one or more identified features to at least one of one or more haptic feedback effects or one or more audio feedback effects.

7. The method of claim 5, wherein:

identifying the constraint comprises identifying, using a sensor, that an amount of light is below a threshold, and

constraining mapping of the one or more identified features comprises constraining mapping of the one or more identified features to at least one of one or more haptic feedback effects or one or more audio feedback effects.

8. The method of claim 5, wherein:

the constraint is a silent mode of operation, and

constraining mapping of the one or more identified features comprises constraining mapping of the one or more identified features to at least one of one or more visual feedback effects or one or more haptic feedback effects.

9. The method of claim 8, wherein the image is associated with a call being received, wherein different areas on the image correspond to different actions that can be taken in response to the call, and wherein the mapped feedback effects are haptic feedback effects indicating an action taken in response to a touch input from the user.

10. A system for providing feedback effects for an image, the system comprising:

at least one processor;

at least one storage device configured to store program code;

wherein the processor is configured to execute the program code to;

identify one or more features in an area of the image;

map the one or more identified features to at least one of multiple modalities of feedback effects; and

generate parameters for feedback effect output and store the generated parameters in association with the image such that, in response to coordinates of an input of a user corresponding to the area of the image, a user interface associated with the image is configured to provide at least one of the multiple modalities of feedback effects to the user.

11. The system of claim 10, wherein the multiple modalities of feedback effects comprise one or more visual feedback effects, one or more haptic feedback effects, and one or more audio feedback effects.

12. The system of claim 10, wherein the processor is further configured to execute the program code to:

identify coordinates of the image corresponding to a touch input of the user;

identify a feedback effect mapped to an area of the image; and

provide the feedback effect using a touchscreen enabled device.

13. The system of claim 10, wherein the one or more identified features comprise a texture, an intensity, and a color associated with the area of the image

14. The system of claim 11, wherein the processor is further configured to execute the program code to:

identify a constraint on providing one or more of the multiple modalities of feedback effects to the user; and

constrain mapping of the one or more identified features to only unconstrained modalities of feedback effects.

15. The system of claim 14, wherein:

the constraint is a visually-impaired mode of operation, and

the processor is configured to execute the program code to constrain mapping of the one or more identified features to at least one of one or more haptic feedback effects or one or more audio feedback effects.

16. The system of claim 14, wherein:

identifying the constraint comprises identifying, using a sensor, that an amount of light is below a threshold, and

the processor is configured to execute the program code to constrain mapping of the one or more identified features to at least one of one or more haptic feedback effects or one or more audio feedback effects.

17. The system of claim 14, wherein:

the constraint is a silent mode of operation, and

the processor is configured to execute the program code to constrain mapping of the one or more identified features to at least one of one or more visual feedback effects or one or more haptic feedback effects.

18. The system of claim 17, wherein the image is associated with a call being received, wherein different areas on the image correspond to different actions that can be taken in response to the call, and wherein the mapped feedback effects are haptic feedback effects indicating an action taken in response to a touch input from the user.

19. A non-transitory computer-readable medium embodying program code for providing feedback effects for an image, the computer-readable medium comprising program code for:

identifying one or more features in an area of the image;

mapping the one or more identified features to at least one of multiple modalities of feedback effects; and

generating parameters for feedback effect output and storing the generated parameters in association with the image such that, in response to coordinates of an input of a user corresponding to the area of the image, a user interface associated with the image is configured to provide at least one of the multiple modalities of feedback effects to the user.

20. The computer-readable medium of claim 19, wherein the multiple modalities of feedback effects comprise one or more visual feedback effects, one or more haptic feedback effects, and one or more audio feedback effects.