INPUT DEVICE, DISPLAY DEVICE, INPUT METHOD, DISPLAY METHOD, AND PROGRAM

• US 20090146968A1
• Filed: 11/17/2008
• Published: 06/11/2009
• Est. Priority Date: 12/07/2007
• Status: Abandoned Application

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Matsuda, Kouichi, NARITA, Tomoya, Tsuzaki, Ryoichi, Miyashita, Ken
• Application Number:
  US12/272,196
• Publication Number:
  US 20090146968A1
• Time in Patent Office: Days
• US Class Current: 345/173
• CPC Class Codes: G06F 2203/04108 Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/0421 by interrupting or reflecting a light beam, e.g. optical touch-screen
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: H04M 2250/22 including a touch pad, a touch sensor or a touch detector digitizers for touch screens G06F3/041

Abstract

An image acquisition section 105 obtains an image of a finger by bringing the finger into touch with a display 13 formed of a device including an image pickup device and picture elements or by bringing the finger closer to the display without touching the display. A computing section 120 determines a center-of-gravity position of the finger, based on the obtained image of the finger. An information generating section 125 generates information for operating the cellular phone based on a displacement of the center-of-gravity position of the finger r. A determination section 115 determines whether the finger has touched the screen or not, based on the touch area between the finger and the screen. A display section 130 displays a virtual sign 14 indicating a reference point for subsequent movement of the finger, at a first position that the finger has touched the screen.

Claims

1. An input device comprising:

an image acquisition section that obtains an image of an object for supplying information to a display device of an apparatus, the image being obtained by bringing the object into touch with a screen of a display device of an apparatus or by bringing the object closer to the screen of the display device to a position capable of detecting the object without touching the screen, the display device being formed of a device including an image pickup device and picture elements;

a computing section that computes a center-of-gravity position of the object using the image of the object obtained by the image acquisition section; and

an information generating section that generates information for operating the apparatus based on a displacement of the center-of-gravity position of the object corresponding to movement of the object, as the input information from the object.

2. The input device according to claim 1, further comprising:

a determination section that determines whether the object has approached the display device to the position capable of being detected without touching the screen or has touched the display device, based on the image of the object obtained by the image acquisition section.

3. The input device according to claim 1, wherein

while the determination section determines that the object is continuously approaching the position capable of being detected without touching the screen or is continuously in touch with the display device, the computing section repeats computation of the center-of-the gravity position of the object;

and

the information generating section keeps on generating the information for operating the apparatus, based on a displacement of the center-of-the-gravity position of the object repeatedly computed.

4. The input device according to claim 3, wherein

when the determination section determines that the object has separated from the display device by a predetermined distance or more, the computing section stops computing the center-of-the-gravity position of the object;

and

the information generating section stops generating the information for operating the apparatus.

5. The input device according to claim 2, wherein when the determination section determines that a plurality of the objects have approached the display device to positions capable of being detected at a plurality of points of the screen without touching the screen, or are continuously in touch with the display device at the points, the information generating section generates the information for operating the apparatus, based on a relative relationship between displacements of the center-of-the-gravity portions of the objects at the points.

6. The input device according to claim 2, wherein when he determination section determines that a plurality of the objects have approached the display device to positions capable of being detected at a plurality of points of the screen without touching the screen, or are continuously in touch with the display device at the points, the information generating section generates the information for executing a different function of the apparatus, based on a displacement of the center-of-the-gravity portion of each of the objects at the respective points.

7. The input device according to claim 2, wherein when the determination section determines that a plurality of the objects have approached the display device to positions capable of being detected at a plurality of points of the screen without touching the screen, or are continuously in touch with the display device, the information generating section generates the information for operating the apparatus, based on a sum of displacements of the center-of-the-gravity portions of the objects at the respective points.

8. The input device according to claim 2, wherein the determination section determines whether the object has approached the display device to the position capable of being detected without touching the screen or has touched the display device, based on brightness of the obtained image of the object.

9. The input device according to claim 8, further comprising:

a selecting section that specifies an arbitrary range including a touch position of the object on the display device based on the brightness of the image of the object, and selects an image within the specified range when the determination section determines that the object has approached the display device to the position capable of being detected without touching the screen and then determines that the object has touched the display device based on the image of the object obtained by the image acquisition section.

10. The input device according to claim 1, wherein the information for operating the apparatus is used for one of controls of scrolling, zooming, and tilting an image displayed on the display device.

11. The input device according to claim 10, wherein the information generating section generates the information for operating the apparatus so that an amount of scrolling, zooming, or tilting the image displayed on the display device is changed based on brightness of the image of the object.

12. The input device according to claim 1, wherein the information indicating the displacement of the center-of-gravity information of the object includes at least one of a difference between arbitrary two points on a moving trajectory of the object, a moving direction of the object, a moving speed of the object, and an acceleration of the object.

13. The input device according to claim 1, wherein the apparatus is a portable-type apparatus.

14. The input device according to claim 13, wherein the object is a finger of a user who holds the portable-type apparatus.

26. A program for causing a computer to implement functions of the input device according to claims 1 through 15.

15. A display device comprising:

an image acquisition section that obtains an image of an object for supplying information to a display device of an apparatus, the image being obtained by bringing the object into touch with a screen of the display device or by bringing the object closer to the screen of the display device to a position capable of detecting the object, without touching the screen, the display device being formed of a device including an image pickup device and picture elements;

a computing section that computes a center-of-gravity position of the object based on the image of the object obtained by the image acquisition section;

an information generating section that generates information for operating the apparatus, based on a displacement of the center-of-gravity position of the object corresponding to movement of the object;

a determination section that determines whether the object has approached the display device to the position capable of being detected without touching the screen or has touched the display device, based on brightness of the image of the object obtained by the image acquisition section; and

a display section that displays a virtual sign at a position first determined by the determination section that the object has touched the display device or at the position first determined by the determination section that the object has approached the display device without touching the screen, as a reference point for subsequent movement of the object.

16. The display device according to claim 15, wherein when the determination section determines that the object has separated from the display device by a predetermined distance or more, the display section stops display of the virtual sign.

17. The display device according to claim 15, wherein

the determination section determines whether the object has approached the display device to the position capable of being detected without touching the screen or has touched the display device, based on the brightness of the image of the object obtained by the image acquisition section;

and

the display section displays an image which is closer to the position approached by the object or the position touched by the object to be more distant from the object.

27. A program for causing a computer to implement functions of the display device according to claims 16 through 23.

18. A display device comprising:

a display section that displays at least one fixed virtual sign at a fixed position on a screen of the display device formed of a device including an image pickup device and picture elements, the at least one fixed virtual sign serving as a reference point when an object for supplying information approaches or touches the display device;

an image acquisition section that obtains an image of the object by bringing the object into touch with the at least one fixed virtual sign or by bringing the object closer to the at least one fixed virtual sign to a position capable of detecting the object, without touching the screen;

a computing section that computes a center-of-gravity position of the object based on the image of the object obtained by the image acquisition section; and

an information generating section that generates information for operating the apparatus, based on a displacement of the center-of-gravity position of the object corresponding to movement of the object.

19. The display device according to claim 18, wherein the at least one fixed virtual sign is projected onto the screen of the display device, and functions as a pseudo input device where the information for operating the apparatus is generated according to a state where the object touches the at least one virtual sign.

20. The display device according to claim 18, wherein

the at least one fixed virtual sign virtually rotates centering on an axis thereof, in response to the movement of the object;

and

the information generating section generates the information for operating the apparatus based on a relative or absolute displacement in a rotating direction of the at least one fixed virtual sign.

21. The display device according to claim 20, wherein the information generating section generates the information for operating the apparatus when the relative or absolute displacement in the rotating direction of the at least one fixed virtual sign exceeds a predetermined threshold value.

22. The display device according to claim 20, wherein the information for operating the apparatus is used to increase or decrease a desired numerical value, based on the relative or absolute displacement in the rotating direction of the at least one fixed virtual sign.

23. The display device according to claim 22, wherein the information generating section generates the information for operating the apparatus so that a change amount of the numerical value differs according to a touch position between the object and the at least one fixed virtual sign.

24. An information input method comprising the steps of:

bringing an object for supplying information into touch with a screen of a display device or bringing the object closer to the screen of the display device to a position capable of detecting the object without touching the screen, thereby obtaining an image of the object, the display device being formed of a device including an image pickup device and picture elements;

computing a center-of-gravity position of the object based on the obtained image of the object; and

generating information for operating an apparatus including the display device, as input information from the object, based on a displacement of the center-of-gravity position of the object corresponding to movement of the object.

25. A display method comprising the steps of:

bringing an object for supplying information into touch with a screen of a display device or bringing the object closer to the screen of the display device to a position capable of detecting the object without touching the screen, thereby obtaining an image of the object, the display device being formed of a device including an image pickup device and picture elements;

computing a center-of-gravity position of the object based on the obtained image of the object;

generating information for operating an apparatus including the display device, based on a displacement of the center-of-gravity position of the object corresponding to movement of the object;

obtaining brightness of the obtained image of the object and determining whether the object has approached the display device without touching the screen or has touched the display device, based on the obtained brightness; and

displaying a virtual sign at a position first determined that the object has touched the display device or at the position first determined that the object has approached the display device without touching the screen, as a reference point for subsequent movement of the object.