Apparatus and method for touch screen interaction based on tactile feedback and pressure measurement

• US 7,952,566 B2
• Filed: 07/31/2007
• Issued: 05/31/2011
• Est. Priority Date: 07/31/2006
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Maruyama, Shigeaki, Poupyrev, Ivan
• Primary Examiner(s): Mengistu; Amare
• Assistant Examiner(s): Bolotin; Dmitriy
• Application Number:
  US11/831,703
• Publication Number:
  US 20080024459A1
• Time in Patent Office: 1,400 Days
• Field of Search: 345173-178, 178 1801- 1811, 178 1901- 1907, 178 2001- 2004, 715700-702
• US Class Current: 345/173
• CPC Class Codes: G06F 2203/014 Force feedback applied to GUI
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

An apparatus includes a display section with a touch screen, and the touch screen is adapted to display at least one graphical user interface object and detect a touch position on the touch screen. The apparatus has a haptic feedback generating unit attached to the touch screen and is adapted to generating haptic feedback. A pressure sensing unit is attached to the touch screen and adapted to detect pressure applied to the touch screen. A controller section is adapted to control and drive the display section. The graphical user interface object displayed on the touch screen has a plurality of logical states. The controller section determines a current logical state of the graphical user interface object and a form of the haptic feedback to be generated depending on the detected touch position.

Claims

1. An apparatus including a display section with a touch screen, the touch screen being adapted to display at least one graphical user interface object and detect a touch position on the touch screen, the touch position being inputted with a user'"'"'s finger or a pointing device, the apparatus comprising:

a haptic feedback generating unit attached to the touch screen and adapted to generating haptic feedback;

a pressure sensing unit attached to the touch screen and adapted to detect pressure applied to the touch screen; and

a controller section adapted to control and drive the display section;

wherein

the graphical user interface object displayed at a display location on the touch screen has a plurality of logical states, the display location defined by a boundary of the graphical user interface object,

the controller section determines a current logical state of the graphical user interface object using a history of detected touch positions and a history of detected pressure values, the current logical state of the graphical user interface object includes at least one of a selected state and an actuated state,

the controller section further determines a form of the haptic feedback to be generated depending on (i) the detected touch position, (ii) the detected pressure value and (iii) the determined current logical state of the graphical user interface object, and

the haptic feedback generating unit further applies, to the touch screen corresponding to the display location of the graphical user interface object upon determination by the controller that at least the detected touch position is within the boundary of the graphical user interface object, a first predetermined force pattern to the touch screen upon determination that the current logical state of the graphical user interface object is the selected sate, and a second predetermined force pattern different from the first predetermined force pattern upon determination that current logical state of the graphical user interface object is the actuated state.

2. The apparatus according to claim 1, wherein:

the logical states of the graphical user interface object include at least a selected state and an actuated state;

the controller section determines that the graphical user interface object is in the actuated state if a pressing event is recognized; and

the controller section recognizes the pressing event using a history of the detected pressure value.

3. The apparatus according to claim 1, wherein:

the logical states of the graphical user interface object include at least a selected state and an actuated state; and

the controller section determines that the graphical user interface object is in the actuated state if;

(i) the touch position is inside of the graphical user interface object; and

(ii) the detected pressure is more than a preset actuation threshold value.

4. The apparatus according to claim 1, wherein:

the logical states of the graphical user interface object include at least a selected state and an actuated state; and

the controller section determines that the graphical user interface object is in the actuated state if;

(i) the touch position is inside of the graphical user interface object; and

(ii) a history of the detected pressure satisfies a preset actuation condition.

5. The apparatus according to claim 1, wherein:

the haptic feedback generation unit includes a single or plurality of piezoelectric elements; and

at least one of the piezoelectric elements is used for generating the haptic feedback and detecting the pressure applied by the user.

6. The apparatus according to claim 1, wherein:

the controller section further controls either a frequency of the haptic feedback, an amplitude of the haptic feedback or both amplitude and frequency of the haptic feedback simultaneously.

7. The apparatus according to claim 1, wherein:

the controller section controls the haptic feedback generating unit to generate a continuous haptic feedback as long as the touch position is inside of the graphical user interface object; and

the controller section changes the continuous tactile feedback in response to a change of the pressure applied to the touch screen, the change of the continuous haptic feedback depending on the current logical state of the graphical user interface object.

8. The apparatus according to claim 1, wherein:

the controller section controls the haptic feedback generating unit to generate a single burst of the haptic feedback when the touch position crosses over a hotspot predefined within the graphical user interface object.

9. The apparatus according to claim 1, wherein:

the controller section controls the haptic feedback generating unit to generate a single burst of the tactile feedback when the touch position or the detected pressure changes more than a preset threshold value.

10. The apparatus according to claim 1, wherein:

the graphical user interface object is formed with a plurality of sub-objects; and

the controller section controls the haptic feedback generating unit to generate different tactile feedbacks for different sub-objects thereof.

11. The apparatus according to claim 2, wherein:

the controller section determines that the graphical user interface object is in the activated state by using a plurality of pressure thresholds.

12. The apparatus according to claim 2, wherein:

the controller section differentiates a stronger push and a lighter push based on a noise level of a signal from the touch screen, the stronger push corresponding to the pressing event, the lighter push corresponding to sliding of the user'"'"'s finger or pointing device.

13. The apparatus according to claim 1, wherein:

the controller section controls the display section to generate visual feedback in correlation with the haptic feedback.

14. The apparatus according to claim 1, wherein:

the logical states of the graphical user interface object include at least a selected state and an actuated state, the selected state being a state where the graphical user interface object is selected but not actuated, the actuated state being a state where the graphical user interface object is actuated;

the logical state is allowed to change to the actuated state after the selected state.

15. A graphical user interface method for a touch screen, comprising:

displaying a graphical user interface object at a display location on the touch screen, the graphical user interface object having a plurality of logical states, the display location defined by a boundary of the graphical user interface object;

detecting a touch position on the touch screen, at which a user'"'"'s finger or a pointing device is touching;

detecting pressure applied on the touch screen when the touch position is detected; and

generating haptic feedback in response to the touching, a form of the haptic feedback being determined depending on (i) the detected touch position, (ii) the detected pressure value and (iii) a current logical state of the graphical user interface object;

wherein

the current logical state of the graphical user interface object is determined by using a history of detected touch positions and a history of detected pressure values, the current logical state of the graphical user interface object includes at least one of a selected state and an actuated state, and

the generated haptic feedback response is applied to the touch screen corresponding to the display location of the graphical user interface object upon determination that at least the detected touch position is within the boundary of the graphical user interface object, a first predetermined force pattern to the touch screen upon determination that the current logical state of the graphical user interface object is the selected state, and a second predetermined force pattern different from the first predetermined force pattern upon determination that current logical state of the graphical user interface object is the actuated state.

16. The graphical user interface method according to claim 15, further comprising:

determining the current logical state of the graphical user interaction object by detecting if there is a pressing event, the pressing event being recognized by using a history of the detected pressure value;

wherein the graphical user interface object is determined as in an actuated state if the pressing event is recognized.

17. The graphical user interface method according to claim 16, wherein:

the pressing event is determined if the history of the detected pressure value satisfies a preset pattern.

18. The graphical user interface method according to claim 15, wherein:

a single burst of the haptic feedback is generated when the detected touch position and/or the detected pressure changed more than a preset threshold value.