User interface device with touch pad enabling original image to be displayed in reduction within touch-input screen, and input-action processing method and program

• US 9,244,544 B2
• Filed: 06/29/2012
• Issued: 01/26/2016
• Est. Priority Date: 07/29/2011
• Status: Active Grant

• Current Assignee: KDDI Corporation
• Original Assignee: KDDI Corporation
• Inventors: Matsuki, Tomoaki
• Primary Examiner(s): Ngo, Tony N
• Application Number:
  US14/233,518
• Publication Number:
  US 20140160073A1
• Time in Patent Office: 1,306 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/04105 Pressure sensors for measuring the pressure or force exerted on the touch surface without providing the touch position
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

A process is disclosed for displaying an image on a display screen and recognizing a user'"'"'s input by the user'"'"'s finger touch on a touch screen, the process including: a first step of displaying an original image; a second step of forming a screen pad, such that a reference image, which is a reduced duplicate of the original image, is displayed on the screen pad within the display screen, such that the reference image is overlaid with the original image, in a translucent mode that renders the original image visible through the reference image, or an opaque mode where the original image is not visible. The reference image is obtained by reducing the original image with or without image simplification. A third step, in response to the user touching the screen pad, detects finger position on the screen pad, and recognizes the user'"'"'s input based on the detected finger position.

Claims

1. A user interface device configured to perform:

(a) a display function of displaying an image on a display screen, and (b) a touch panel function of outputting successive locations of the user'"'"'s finger touch point on a touch screen over time, the user interface device comprising;

a screen-pad controller configured to display, while displaying an original image on the display screen, a reference image on the display screen forming a screen pad, the screen pad being defined as an in-screen touch pad, such that the reference image has a reduced size with respect to the original image to partially overlay the reference image with the original image, the original image containing a plurality of user-selectable first objects, the reference image containing a plurality of user-selectable second objects respectively corresponding to at least each of the plurality of first objects; and

an operation controller configured to, in response to a user'"'"'s finger touch with the screen pad, (i) detect one of the plurality of second objects on the screen pad, which has been selected by the user'"'"'s finger touch, and (ii) recognize the user'"'"'s input based on one of the plurality of first objects, which corresponds to the selected second object.

2. The user interface device according to claim 1, wherein the operation controller is configured to (i) display a pointer on the original image being displayed on the display screen in order to indicate a location on the display screen that corresponds to a position at which the user has touched the finger to the screen pad, and (ii) invoke the same operation that will occur in the user interface device if the user touches the finger to the display screen at a same position as an initial position that the pointer took, and then swipes the finger across the original image being displayed on the display screen, along a same path that the pointer took after the initial position.

3. The user interface device according to claim 1, wherein the screen pad is displayed on the display screen at a position that allows the user, who is holding the user interface device using the user'"'"'s one hand in an attempt to manipulate the user interface device using the one hand of the user, to reach any location on the screen pad using a finger of the one hand of the user.

4. The user interface device according to claim 1, wherein

the operation controller is configured to, in response to an event occurring in which a stationary-touch duration during which a finger touch point is held substantially stationary on the screen pad is equal to or longer than a predetermined duration, to activate a touch focus mode for monitoring successive locations of the finger touch point on the screen pad, with the plurality of second objects held stationarily on the display screen when the finger moves on the screen pad.

5. The user interface device according to claim 4, wherein the operation controller is configured to:

(i) display a pointer on the original image being displayed on the display screen, to indicate a location on the display screen, which corresponds to a position at which the user has touched the finger to the screen pad, and (ii) in response to an event occurring in which, during the touch focus mode, the pointer indicates one of the plurality of first objects included in the original image, determine that the user has performed a focus motion to select the one first object, and then invoke an operation associated with the one selected first object.

6. The user interface device according to claim 4, wherein

the operation controller is configured to:

(i) display a pointer on the original image being displayed on the display screen, to indicate a location on the display screen, which corresponds to a position at which the user has touched the finger to the screen pad, and (ii) in response to an event occurring in which, during the touch focus mode, the pointer indicates one of the plurality of first objects included in the original image, and in which a pressing-force with which the finger presses the touch panel is equal to or larger than a predetermined threshold, determine that the user has performed a click motion to invoke an function pre-assigned to the one first object, and then invoke an operation associated with the one first object.

7. The user interface device according to claim 6, further comprising:

a haptic feedback arrangement configured to impart haptic feedback to the user'"'"'s finger that is in touch with the touch panel; and

a haptic feedback controller configured to control the haptic feedback arrangement to impart the haptic feedback to the finger via the touch panel.

8. The user interface device according to claim 1, wherein

the operation controller, in response to an event occurring in which, before a predetermined length of time elapses after the finger starts being touched to the screen pad, the finger starts moving in touch with the screen pad, and then the finger stops moving, is configured to determine that the user has performed a drag click motion in the form of a succession of a first motion for moving one of the plurality of first objects in coordination with a movement of the finger, and a second motion for further moving the one of the plurality of first objects at a rate variable as a function of a magnitude of the pressing force after the drag motion ends, and then invoke an operation associated with the drag click motion.

9. The user interface device according to claim 1, wherein the operation controller is configured to, in response to the user touching the finger to the display screen at a location away from a region of the screen pad being displayed, invoke a same operation that would be invoked in the user interface device if the user touched the finger to the display screen at a same location as the away location in an absence of the screen pad on the display screen.

10. The user interface device according to claim 1, wherein

the screen-pad controller is configured to change between (i) a display mode allowing the screen pad to be displayed, and (ii) a non-display mode not allowing the screen pad to be displayed, depending on the user'"'"'s manipulation, and

in the non-display mode, the user is allowed to directly select one of the plurality of first objects by touching the finger to the original image.

11. The user interface device according to claim 1, wherein the screen-pad controller is configured to adjust a reduction scale factor with which the original image is to be reduced to create the reference image of the screen pad, depending on the user'"'"'s manipulation.

12. The user interface device according to claim 1, wherein

the plurality of first objects include a plurality of user-selectable original icons, and

the plurality of second objects include a plurality of user-selectable miniature icons obtained by reducing the corresponding respective original icons with or without image simplification.

13. A process of processing a user'"'"'s inputs on a user interface device configured to:

(i) display an image on a display screen, and (ii) recognize the user'"'"'s input by the user'"'"'s finger touch on a touch screen in association with the displayed image, the process comprising;

displaying an original image on the display screen, the original image containing a plurality of user-selectable first objects;

forming a screen pad defined as an in-screen touch pad, such that a reference image having a reduced size with respect to the original image being displayed is displayed on the screen pad within the display screen, in a manner that the reference image is partially overlaid with the original image, in a translucent mode that renders the original image visible through the reference image, or an opaque mode that fails to render the original image visible through the reference image, the reference image containing a plurality of user-selectable second objects respectively corresponding to at least each of the plurality of first objects; and

in response to the user touching the finger to the screen pad, detecting one of the plurality of second objects on the screen pad, which has been selected by the user'"'"'s finger touch, as a selected second object, and recognizing the user'"'"'s input based on one of the plurality of first objects, which corresponds to the selected second object.

14. A non-transitory computer readable storage medium configured to store a program that when executed by a computer performs the process according to claim 13.

15. The process according to claim 13, wherein

the process includes changing between (i) a display mode allowing the screen pad to be displayed, and (ii) a non-display mode not allowing the screen pad to be displayed, depending on the user'"'"'s manipulation, and

in the non-display mode, the user is allowed to directly select one of the plurality of first objects by touching the finger to the original image.

16. The process according to claim 13, further including adjusting a reduction scale factor with which the original image is to be reduced to create the reference image of the screen pad, depending on the user'"'"'s manipulation.

17. The process according to claim 13, wherein

the plurality of first objects include a plurality of user-selectable original icons, and

the plurality of second objects include a plurality of user-selectable miniature icons obtained by reducing the corresponding respective original icons with or without image simplification.