Apparatus for manipulating an object displayed on a display device by using a touch screen

• US 7,345,675 B1
• Filed: 09/08/1998
• Issued: 03/18/2008
• Est. Priority Date: 10/07/1991
• Status: Expired due to Fees

• Current Assignee: Apple Inc.
• Original Assignee: Fujitsu Limited
• Inventors: Okuyama, Satoshi, Minakuchi, Yu, Kamata, Hajime, Fukue, Akiko
• Primary Examiner(s): Hjerpe, Richard
• Assistant Examiner(s): FATAHI YAR, MAHMOUD
• Application Number:
  US09/149,216
• Time in Patent Office: 3,479 Days
• Field of Search: 345/173, 345/127, 345/341, 345/339, 395/800
• US Class Current: 345/173
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

Apparatus and a method of operation manipulates the display of an object on the display surface of a display device in accordance with sensing touching contact on a touch panel, superimposed on the display surface and simulating movement of the displayed object in accordance with the characteristics of the touching contact on the touch panel adjacent to the displayed object image. Further, object data defining the display of the object image for each of a plurality of different states of the object, display information specifying the shape and physical properties of the object and the current display position thereof and file information relating to the stored data are stored and accessed for interpreting the simulated manipulation of the object by the characteristics of the touching contact therewith, for correspondingly manipulating and displaying the thus manipulated object image.

Claims

1. A method for simulating manipulation of an object utilizing a displayed image of the object, said method comprising the steps of:

sensing touching contact relative to the displayed image of the object, the touching contact simulating a manipulation of the object, and outputting touch information corresponding to the sensed touching contact and changes of the touching contact;

detecting, from the output touch information, characteristics of said touching contact including the selected location on the object image of the touching contact and changes of the touching contact and recognizing therefrom the corresponding object manipulation simulated thereby; and

changing the display of the object image in accordance with the recognized, simulated manipulation of the object and object information for the displayed object image, including data relative to a type of the displayed object, specifying physical properties of the object.

2. A method according to claim 1, further comprising, in response to detected characteristics indicating that two touching contacts are located at opposite sides of the object image and move and stop with a distance therebetween:

controlling the display of the object image on the display device so that the object moves on a display surface of the display device from where the two touching contacts are located at opposite sides of the object image to where the two touching contacts stop with the distance therebetween.

3. A method according to claim 1, further comprising, in response to the detected characteristics indicating that the touching contact position is located at the object and moves while keeping in contact with the object and the object is sufficiently large relative to, and extends beyond, the display or only a portion of the object image is displayed on the display device:

controlling the display of the object by scrolling the object image while displaying same.

4. A method according to claim 3, further comprising in response to the detected characteristics indicating that a moving, touching contact stops moving:

controlling the display of the object image on the display device so that the scrolling of the object image stops on the display.

5. A method as recited in claim 1, further comprising storing information specifying a position of a portion of the object image being displayed on the display device, relative to the whole object image.

6. A method according to claim 1, further comprising, in response to the detected characteristics indicating that the touching contact is located at a geometric center or at a center of gravity of the object and moves and stops while maintaining contact with the object:

controlling the display of the object image so that the object image moves from where the touching contact is located at the geometric center or the center of gravity to where the moving, touching contact stops moving.

7. A method according to claim 1, further comprising in response to the detected characteristics indicating that the touching contact is located at a position off a geometric center or off a center of gravity of the object and moves and stops while maintaining contact with the object:

controlling the display of the object image so that the object image moves while rotating, from a position where the touching contact begins to a position where the touching contact stops.

8. A method according to claim 1, further comprising, in response to the detected characteristics indicating that the touching contact touches the object after moving in a direction toward the object, from a position apart therefrom and at a first speed higher than a predetermined speed:

controlling the display of the object so that the object image moves a distance in the direction and at a second speed proportional to the first speed with which the touching contact touches the object.

9. A method according to claim 1, further comprising, in response to the object information specifying the displayed object to be of a rollable type:

displaying the object image in a rolling condition.

10. A method according to claim 9, further comprising, in response to the detected characteristics indicating that the touching contact touches the object image:

moving and stopping while maintaining contact with the object, displaying the object image in a rolling condition.

11. A method according to claim 9, further comprising, in response to the detected characteristics indicating that the touching contact moves on the object while maintaining said touching contact with the object:

displaying the object image in a rolling condition and changing a positional relationship between the object image and the touching contact on the object image in accordance with the movement of the touching contact on the object.

12. A method according to claim 9, further comprising:

displaying the object image in a rolling condition so as to move the object image in a direction of the movement of the touching contact on the object.

13. A method according to claim 1, further comprising:

displaying the object image in a state of distortion and/or restoration, according to the object information specifying the displayed object to be of an elastic type.

14. A method according to claim 1, further comprising:

sensing an amount of pressure applied to the object image and, when the detected characteristic of the touching contact exceeds a specific amount of pressure, displaying the object image in a degree of distortion and/or restoration according to the amount of, and changes in, pressure of the touching contact.

15. An image display and manipulating method, comprising:

displaying an image of an object;

sensing touching contact relative to the displayed object image and outputting corresponding touch information;

detecting, from the received touch information, characteristics of the touching contact, the characteristics including the position on the object image of the input touching contact and changes of the position of the input touching contact; and

changing the display of the object images in accordance with the characteristics and object information including an object type, which specifies physical properties of the object.

16. A method according to claim 15, further comprising detecting changes of the position of the input touching contact.

17. A method of manipulating a displayed image of an object, comprising:

displaying an image of the object;

detecting a position of touching contact relative to the object image and changes of the touching contact; and

changing the display of the object image in accordance with detected characteristics and object information including at least an object type which specifies physical properties of the object and position information which specifies a position of the displayed object image.

18. A method according to claim 17, further comprising detecting changes of the touching contact.

19. A method of manipulating a displayed image of an object, comprising:

sensing characteristics of a touching contact relative to the displayed image of the object, the characteristics including at least a position of the touching contact and changes therein, and outputting corresponding touch information representing a simulated movement of the object responsive to the characteristics of the touching contact; and

recognizing a type of manipulation of the object image in accordance with the information representing a simulated movement of the object and an object type specifying physical properties of the object and, correspondingly, manipulating and displaying the object image.

20. A method according to claim 19, further comprising sensing changes in the position of the touching contact.

21. A method as recited in claim 20, further comprising storing information specifying a position of a portion of the object being displayed, relative to the whole object.

22. A display method comprising:

producing a display of an image of an object;

sensing touching contact relative to the displayed object image, simulating an actual touching contact with an actual object corresponding to the displayed image of the object;

detecting characteristics of the touching contact including a position of the touching contact relatively to the object image and changes of the position of the touching contact; and

changing the display of the object in accordance with object information, specifying physical properties of the object, and the detected characteristics.

23. A method according to claim 22, further comprising detecting changes of the position of the touching contact.

24. A computer readable medium storing therein a computer program affording simulated manipulation of an object in accordance with an image of the object displayed on a display surface of a display device, said computer program comprising:

a first function of responding to sensed touching contact relative to the displayed image of the object, simulating a manipulation of the object, and to changes in the touching contact for producing and outputting corresponding touch information;

a second function of detecting, from the output touch information, characteristics of said touching contact including the selected location on the object image of the touching contact and changes of the touching contact and recognizing therefrom the corresponding object manipulation simulated thereby; and

a third function of changing the display of the object image in accordance with the detected characteristics of the touching contact, the recognized, simulated manipulation of the object and object information for the displayed object image including data relative to a type of the displayed object specifying physical properties of the object.

25. A computer readable medium according to claim 24, wherein, in response to said second function detecting characteristics indicating that two touching contacts are located at opposite sides of the object image and, further, move and stop with a distance therebetween, said third function changes the display of the object image so that the object moves on a display surface of the display device from where the two touching contacts are located at opposite sides of the object image to where the two touching contacts stop with the distance therebetween.

26. A computer program product for controlling a computer, the program product comprising:

a recording medium readable by the computer;

means responsive to touching contact, and to changes in the touching contact, relative to an object displayed on a display surface of a display device, simulating manipulation of the object, for producing and outputting touch information corresponding to the sensed touching contact and representing the simulated manipulation of the object;

first subroutine means, responsive to the output touch information, for detecting the characteristics of the touching contact and recognizing therefrom the corresponding object manipulation simulated thereby; and

second subroutine means for changing the display of the object image in accordance with the detected characteristics of the touching contact, the recognized, simulated manipulation of the object and object information for the displayed object image including data relative to a type of the displayed object image specifying physical properties of the object.

27. An apparatus affording simulated manipulation of an object in accordance with an image of the object display surface of a displayed on a display device, comprising:

a display device displaying an image of an object;

an input device responsive to a touching contact relative to the displayed object image, the touching contact simulating a manipulation of the object, and outputting corresponding touch information;

a storage unit storing a plurality of object information, each object information specifying a responsive manipulation type of each object; and

a display controller changing the display of the object image in accordance with the touch information and the stored object information specifying a manipulation type of the object which specifies physical properties of the object.

28. An apparatus according to claim 27, wherein:

said storage unit stores an object information specifying that the manipulation type is “

rollable”

; and

in response to the touch information and said object information, said display controller displays the object image in a rolling condition.

29. An apparatus according to claim 27, wherein:

said storage unit stores an object information specifying a manipulation type of “

elastic”

; and

in response to the touch information and said object information, the display controller displays the object image in a distortion and/or restoration condition.

30. A method for simulating manipulation of an object using a displayed image of the object, comprising:

displaying an image of an object;

responding to a touching contact relative to the displayed object image, the touching contact simulating a manipulation of the object, and outputting corresponding touch information; and

changing the display of the object image in accordance with the touch information and at least one of plural object information, each object information specifying a responsive manipulation type of the object which specifies physical properties of the object.

31. A method according to claim 30, wherein:

said one of plural object information specifies that the manipulation type is “

rollable”

; and

comprises

in response to the touch information and said object information, displaying the object image in a rolling condition.

32. A method according to claim 30, wherein said one of plural object information specifies that the manipulation type is elastic, further comprising:

in response to the touch information and said object information, displaying the object image respectively in a distorted or a restored condition.

33. A storage medium storing a process displaying an image of an object by sensing touching contact relative to the displayed object image and outputting corresponding touch information;

detecting, from the received touch information, characteristics of the touching contact, the characteristics including the position on the object image of the input touching contact and any changes of the position of the input touching contact; and

changing the display of the object image in accordance with the characteristics and an object information including an object type, which specifies physical properties of the object.

34. A storage medium storing a process displaying an image of the object by detecting a position of touching contact relative to the object image and any changes of the touching contact and changing the display of the object image in accordance with the detected characteristics and object information including at least an object type which specifies physical properties of the object and position information which specifies a position of the displayed object image.

35. A storage medium storing a process displaying an image of an object by sensing characteristics of a touching contact relative to the displayed image of the object, the characteristics including at least a position of the touching contact and any changes therein, and outputting corresponding touch information representing a simulated movement of the object responsive to the characteristics of the touching contact;

and recognizing a type of manipulation of the object image in accordance with the information representing a simulated movement of the object and an object type specifying physical properties of the object and, correspondingly, manipulating and displaying the object image.

36. A storage medium as recited in claim 35, storing information specifying a position of a portion of the object being displayed, relative to the whole object.

37. A storage medium producing a display of an image of an object by sensing touching contact relative to the displayed object image, simulating an actual touching contact with an actual object corresponding to the displayed image of the object;

detecting characteristics of the touching contact including a position of the touching contact relatively to the object image and any changes of the position of the touching contact; and

changing the display of the object in accordance with the object information specifying physical properties of the object, and the detected characteristics.

38. A computer readable medium storing therein a computer program affording simulated manipulation of an object using a displayed image of the object, said computer program comprising:

a first function of responding to a touching contact relative to the displayed object image, the touching contact simulating a manipulation of the object, and outputting corresponding touch information; and

a second function of changing the display of the object image in accordance with the touch information and at least one of plural object information, each said object information specifying a responsive manipulation type of the corresponding object which specifies physical properties of the object.

39. A controller for use with a display device to manipulate an object displayed on the display device and an input device responsive to a touching contact relative to the displayed object and simulating said manipulation, the input device producing corresponding touch information and the controller comprising:

a detector detecting, from the touch information, characteristics of said touching contact including the selected location on the displayed object of the touching contact and changes of the touching contact and recognizing therefrom the corresponding object manipulation simulated thereby; and

a display controller controlling to change the display of the object in accordance with the recognized, simulated manipulation of the object and object information for the displayed object, including data relative to a type of the displayed object which specifies physical properties of the object.

40. A controller for use with a display device to manipulate an object displayed on the display device and an input device responsive to a touching contact relative to the displayed object, the touching contact simulating said manipulation and the input device producing corresponding touch information;

the controller comprising;

a storage unit storing a plurality of object information, each object information specifying a responsive manipulation type of each object; and

a display controller controlling to change the display of the object in accordance with the touch information and the stored object information specifying a manipulation type of the object which specifies physical properties of the object.

41. An apparatus for use with a display device to manipulate an object displayed on the display device, the apparatus comprising:

an input device responding to a manipulation to the displayed object and producing corresponding input information, wherein

said input device is responsive to a touching contact relative to the displayed object, the touching contact simulating said manipulation, and produces corresponding touch information;

a storage unit storing object information specifying an object and its center of gravity;

a display controller controlling to change the display of the object in accordance with the input information and the object information specifying physical properties of the object, wherein

said display controller controls the display of the object in accordance with the touch information and the object information.

42. An apparatus according to claim 41, wherein, in response to touch information indicating a movement such that the touching contact touches the object at its center or its center of gravity, moves and stops while keeping in contact with the object, said display controller recognizes the movement as a push manipulation and controls the display of the object so that the object moves from where the touching touches the object to where the touching stops.

43. An apparatus according to claim 41, wherein in response to touch information indicating a movement such that the touching contact touches said object at a position off the center or the center of gravity thereof, moves and stops on said touch panel while keeping in contact with the object, said display controller controls the object on the display device so that the object moves while rotating from where the touching contact touches the object to where the touching contact stops.

44. An apparatus for use with a display device to manipulate an object displayed on the display device, the apparatus comprising:

an input device inputting a manipulation to the displayed object and producing corresponding input information;

a display controller controlling to change the display of the object on the display device so that the object moves on the display device from where two touches touch respective, opposite sides of the object to where the two touches stop, in accordance with the input information indicating a movement such that the two touches touch respective, opposite sides of the object and move.

45. An apparatus according to claim 44, wherein said input device is responsive to the two touches relative to the displayed object and simulating said manipulation, and outputs corresponding said input information;

and

the display controller controlling to change the display of the object on the display device so that the displayed object moves on the display device from where the two touches touch respective, opposite sides of the displayed object to where the two touches stop touching the object, in accordance with the touch information indicating a movement such that the two touches touch the respective, opposite sides of the object and move.

46. An apparatus of claim 45, further comprising:

a storage unit storing position information which specifies the position where the object is displayed on the display device;

wherein the display controller, in accordance with the touch information and display position information, recognizes a movement such that the two touches touch the respective, opposite sides of the object and move, and controls the object on the display device so that the object moves on the display device from where the two touching touch both sides of the object to where the two touching stop.

47. A controller for use with a display device to manipulate an object displayed on the display device and an input device inputting a manipulation to the displayed object and producing corresponding input information, the controller comprising:

a display controller controlling to change the display of the object on the display device so that the object moves on the display device from where two touches touch respective, opposite sides of the object to where the two touches stop such touching, in accordance with the input information indicating a movement such that the two touches touch the respective, opposite sides of the object and move.

48. A method for simulating manipulation of a displayed object, comprising:

displaying an object;

responding to a manipulation to the displayed object at an input device which produces corresponding input information;

controlling to change the display of the object on the display device so that the object moves on the display device from where two touches touch the respective, opposite sides of the object to where the two touches stop such touching, in accordance with the input information indicating a movement such that the two touches touch the respective, opposite sides of the object and move.

49. A computer readable medium storing therein a computer program affording simulated manipulation of displayed object, said computer program comprising:

a first function of responding to a manipulation to the displayed object at an input device which produces corresponding input information;

a second function of controlling to change the display of the object on the display device so that the object moves on the display device from where two touches touch the respective, opposite sides of the object to where the two touches stop such touching, in accordance with the input information indicating a movement such that the two touches touch the respective, opposite sides of the object and move.

50. An apparatus for use with a display device to manipulate an object displayed on the display device, the apparatus comprising:

an input device responsive to a touching contact, relative to the displayed object, and producing corresponding input information;

a storage unit storing object information about a large object extending beyond a display area of the display device; and

a display controller, in accordance with the object information and the input information indicating a movement such that the touching contact moves a distance more than a predetermined distance or the touching contact moves at a speed higher than a predetermined speed, controlling to display the object on the display device in a scroll condition.

51. An apparatus according to claim 50, wherein said display controller controls to start the scroll condition at a first speed and gradually decreases the scroll condition to a second, lower speed.

52. An apparatus according to claim 51, wherein said first speed of the scroll condition depends on said speed at which, or said distance at once which, said touching contact moves.

53. An apparatus according to claim 50, wherein said display controller controls to decrease the speed of the scroll condition in a case where another touching contact does not occur.

54. An apparatus according to claim 51, wherein said display controller controls to continue the scroll condition in a case where another touching contact occurs before the scroll condition steps.

55. An apparatus according to claim 50, wherein said display controller controls to stop the scroll condition in a case where the touching contact stops in the scroll condition of the displayed object.

56. A controller for use with a display device to manipulate an object displayed on the display device and an input device responsive to a touching contact relative to the displayed object and producing corresponding input information, the controller comprising:

a storage unit storing object information about a large object extending beyond a display area of the display device; and

a display controller, in accordance with the object information and the input information indicating a movement, controlling to display the object on the display device in a scroll condition, such that the touching contact moves a distance more than a predetermined distance or the touching contact moves at a speed higher than a predetermined speed.

57. A method for simulating manipulation of a displayed object, comprising:

displaying a part of an object which extends beyond a display area;

responding to a touching contact to the displayed object, at an input device responsive to a touching contact to the displayed object and producing corresponding input information; and

controlling to display the object on the display device in a scroll condition, in accordance with the input information indicating a movement such that the touching contact moves a distance more than a predetermined distance or the touching contact moves at a speed higher than a predetermined speed.

58. A computer readable medium storing therein a computer program affording simulated manipulation of displayed object, said computer program comprising:

a first function of responding a manipulation to a displayed object from an input device responsive to a touching contact relative to the displayed object and producing corresponding input information, said displayed object extending beyond a display area; and

a second function of controlling to display the object on the display device in scroll condition, in accordance with the input information indicating a movement such that the touching contact moves a distance more than a predetermined distance or the touching contact moves at a speed higher than a predetermined speed.

59. An apparatus controlling a display of an object in accordance with a simulated manipulation of the object produced by a corresponding touching contact on an input device, relative to the displayed object, the input device producing corresponding touch information and the apparatus comprising:

a detector detecting, from the touch information, characteristics of said touching contact including the selected location on the displayed object of the touching contact and changes of the touching contact and recognizing therefrom the corresponding object manipulation simulated thereby; and

a display controller controlling to change the display of the object in accordance with the recognized, simulated manipulation of the object and object information for the displayed object, including data relative to a type of the displayed object, which specifies physical properties of the object.

60. An apparatus controlling a display of an object in accordance with a simulated, selected manipulation of the object produced by corresponding touching contact on an input device, relative to the displayed object, the input device outputting corresponding touch information and the apparatus comprising:

a storage unit storing a plurality of object information, each object information specifying a responsive manipulation type of each object; and

a display controller controlling to change the display of the object in accordance with the touch information and stored object information specifying a manipulation type of the object, which specifies physical properties of the object.