Portable communication terminal, information display device, control input device and control input method

• US 6,943,774 B2
• Filed: 04/02/2001
• Issued: 09/13/2005
• Est. Priority Date: 04/02/2001
• Status: Expired due to Fees

• Current Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Original Assignee: Matsushita Electric Industrial Company Limited (Panasonic Holdings Corporation)
• Inventors: Horiki, Toshio
• Primary Examiner(s): Lefkowitz, Sumati
• Assistant Examiner(s): Jorgensen, Leland R.
• Application Number:
  US09/824,397
• Publication Number:
  US 20020140667A1
• Time in Patent Office: 1,625 Days
• Field of Search: 455/79, 455/566, 455/574, 345/166, 345 7- 9, 345156-158, 345/1.3, 178/18.09, 178/18.01, 178/345, 178/18.04, 348/14.05, 348/14.08, 348/14.09, D14/130, D14/137, D14/138, D14/144, D14/345, D14/240, D14/248, D14/247, D14/243, D14/343
• US Class Current: 345/158
• CPC Class Codes: G06F 1/1616 with folding flat displays, e.g. laptop computers or notebooks having a clamshell configuration, with body parts pivoting to an open position around an axis parallel to the plane they define in closed position
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1647 including at least an additional display G06F1/1692 takes precedence
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 1/169 the I/O peripheral being an integrated pointing device, e.g. trackball in the palm rest area, mini-joystick integrated between keyboard keys, touch pads or touch stripes G06F1/1643 takes precedence; constructional details of pointing devices G06F3/033
• CPC Class Codes: G06F 2200/1614 Image rotation following screen orientation, e.g. switching from landscape to portrait mode
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G09G 2330/021 Power management, e.g. power saving
• CPC Class Codes: G09G 2340/0492 Change of orientation of the displayed image, e.g. upside-down, mirrored
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T
• CPC Class Codes: H04M 1/0214 Foldable telephones, i.e. with body parts pivoting to an open position around an axis parallel to the plane they define in closed position combined with translation movement H04M1/0235
• CPC Class Codes: H04M 1/0218 The hinge comprising input and/or output user interface means
• CPC Class Codes: H04M 1/0247 comprising more than two body parts
• CPC Class Codes: H04M 1/233 including a pointing device, e.g. roller key, track ball, rocker switch or joystick constructional details of pointing devices per se G06F3/033
• CPC Class Codes: H04M 2250/12 including a sensor for measuring a physical value, e.g. temperature or motion telephones including GPS signal receivers H04M2250/10
• CPC Class Codes: H04M 2250/16 including more than one display unit
• CPC Class Codes: H04N 2007/145 Handheld terminals
• CPC Class Codes: H04N 7/142 : Constructional details of the terminal equipment, e.g. arrangements of the camera and the display

Abstract

A portable communications terminal has

• • display part having a display surface for displaying an image;
  • main body having at least an antenna for transmitting and receiving radio waves for wireless communications, a speaker for outputting voice, and a pointing device for controlling at least a cursor displayed on said display surface; and
  • joining part for joining said display part to said main body in an angularly movable fashion, and wherein:
  • said speaker is disposed on one main surface of said main body, and
  • said pointing device is disposed on the other main surface of said main body opposite from said one main surface.

Claims

1. An information display method comprising the steps of:

imaging and capturing an image of an object;

processing the image captured by said imaging step;

displaying prescribed information on a projection micro-display, separately obtained from the imaging step;

displaying a designated pointer on said display and controlling the pointer using only the same hand that holds the projection micro-display; and

controlling information on the display comprising the steps of;

(a) storing as a control input a sequential change, first (i) of a first hand pattern, formed of at least one finger, and then second (ii) of a second hand pattern, formed of at least one finger, the second hand pattern different from the first hand pattern;

(b) imaging, by the camera, the first hand pattern;

(c) imaging, by the camera, the second hand pattern;

(d) recognizing the first hand pattern imaged in step (b);

(e) recognizing the second hand pattern imaged in step (c);

(f) comparing a sequential change of first step (d) and then second step (e) to the stored control input;

(g) controlling the information on the display, after the comparing of step (f);

(h) positioning the display between an eye of a user and a hand of the user;

(i) viewing the at least one finger of the hand on the display, while forming the first and second hand patterns; and

(j) visually aligning the eye of the user, the display and the first and second hand patterns.

2. The information display method according to claim 1, wherein the image captured by said imaging step is a fingertip, a first hand pattern and a second hand pattern.

3. The information display method according to claim 1, wherein said display step makes said display surface be equal in size to a region within which said imaging step captures said image, or be smaller than said capture region.

4. The information display method according to claim 1, wherein said image processing step extracts a contour of said image, and

a position detecting step detects the position of said image on a screen from said extracted contour.

5. The information display method according to claim 1, wherein said image processing steps performs processing on portions of said image that are designated by a specific color and/or a specific temperature, and/or on portions of said image that lie within a focal length of said imaging means.

6. The control input method comprising:

the information display method according to anyone of claims 1 to 5; and

input step of carrying out a control input on an object pointed to by said designated pointer on said display surface.

7. The control input method according to claim 6, wherein said designated pointer is displayed by detecting a fingertip as said image.

8. The control input method according to claim 6, wherein said position detecting step compares the image captured by said imaging step or the image extracted by said image processing step with a plurality of image patterns corresponding to said control inputs respectively, and when said captured or extracted image matches any one of said image patterns, said input step carries out a control input that corresponds to said matched image pattern.

9. The control input method according to claim 6, wherein said position detecting step compares the images captured by said imaging step or the images extracted by said image processing step with a combination of a plurality of image patterns corresponding to one of said control input, and when said captured or extracted image match any one of said combination of image patterns, said input step carries out a control input that corresponds to said matched combination of image patterns.

10. The information display method according to any one of claims 1 to 5, wherein said display is used for a portable communication terminal or a portable telephone.

11. A portable telephone having camera and a display, a method of controlling information on the display comprising the steps of:

(a) storing as a control input a sequential change, first (i) of a first hand pattern, formed of at least one finger, and then second (ii) of a second hand pattern, formed of at least one finger, the second hand pattern different from the first hand pattern;

(b) imaging, by the camera, the first hand pattern;

(c) imaging, by the camera, the second hand pattern;

(d) recognizing the first hand pattern imaged in step (b);

(e) recognizing the second hand pattern imaged in step (c);

(f) comparing a sequential change of first step (d) and then second step (e) to the stored control input;

(g) controlling the information on the display, after the comparing of step (f);

(h) positioning the display between an eye of a user and a hand of the user;

(i) viewing the at least one finger of the hand on the display, while forming the first and second hand patterns; and

(j) visually aligning the eye of the user, the display and the first and second hand patterns.

12. The method of controlling information on the display of claim 11, wherein

step (a) includes storing as a further control input a further sequential change first (i) of the second hand pattern, and then second (ii) of the first hand pattern;

step (b) includes imaging another first hand pattern;

step (c) includes imaging another second hand pattern;

step (d) includes recognizing the other first hand pattern imaged in step (b);

step (e) includes recognizing the other second hand pattern imaged in step (c);

comparing a further sequential change of first step (e) and then second step (d) to the stored further control input; and

controlling the information on the display, based on the further sequential change.

13. The method of controlling information on the display of any one of claims 11-12 wherein

controlling the information on the display includes controlling the information on a projection micro-display, and

superimposing a cursor on the display surface of the projection micro display to control the information;

the method further including;

visually observing both the cursor and the one finger forming the first hand pattern and the second hand pattern, and

moving both the cursor and the one finger imaged on the display to control the information on the display.