COMMUNICATION TERMINALS WITH SUPERIMPOSED USER INTERFACE

• US 20090254855A1
• Filed: 04/08/2008
• Published: 10/08/2009
• Est. Priority Date: 04/08/2008
• Status: Abandoned Application

• Current Assignee: Sony Ericsson Mobile Communications AB (Sony Group Corp.)
• Original Assignee: Sony Ericsson Mobile Communications AB (Sony Group Corp.)
• Inventors: Kretz, Martin, Gajdos, Tom
• Application Number:
  US12/099,541
• Publication Number:
  US 20090254855A1
• Time in Patent Office: Days
• US Class Current: 715/800
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1684 Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
• CPC Class Codes: G06F 1/169 the I/O peripheral being an integrated pointing device, e.g. trackball in the palm rest area, mini-joystick integrated between keyboard keys, touch pads or touch stripes G06F1/1643 takes precedence; constructional details of pointing devices G06F3/033
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

An electronic device includes a user input device and a display screen. A moving picture representative of a pointing object that is external to the electronic device is superimposed onto the display screen, and a plurality of features of the pointing object are interpreted as selection pointers so that a movement of the pointing object relative to the display screen may be interpreted as movement of a plurality of selection pointers.

Claims

1. An electronic device comprising:

a display screen;

a controller that is coupled to the display screen and that is configured to display an object on the display screen and to superimpose an image representative of a pointing object that is external to the electronic device onto the display screen wherein at least one of the object or the image representative of the pointing object is at least partially transparent; and

a user input management unit that is coupled to the controller and that is configured to interpret a plurality of features of the pointing object as selection pointers so that a movement of the pointing object relative to the display screen is interpreted by the user input management unit as movement of the selection pointers.

2. The electronic device of claim 1, wherein the user input management unit is configured to interpret movement of the plurality of selection pointers relative to one another as a selection command.

3. The electronic device of claim 1, wherein the user input management unit is configured to interpret movement of two of the plurality of selection pointers into contact with each other as a selection command.

4. The electronic device of claim 1, wherein the user input management unit is configured to interpret a change in size of the image representative of the pointing object as a zoom command, and to increase a magnification of an image on the display screen in response to the change in size of the image representative of the pointing object.

5. The electronic device of claim 1, wherein the controller is configured to magnify a portion of the object on the display screen in response to a selection region between the selection pointers being moved over the object on the display screen.

6. The electronic device of claim 5, wherein a magnification level of the portion of the object on the display screen is determined in response to a spacing between two of the selection pointers.

7. The electronic device of claim 1, wherein portions of the display screen around the object are magnified, and wherein a level of magnification of portions of the display screen around the object is proportional to a distance from the object.

8. The electronic device of claim 1, wherein the user input management unit comprises a software object implemented by the controller.

9. The electronic device of claim 1, wherein the electronic device comprises a housing including a front side and a reverse side opposite the front side, wherein the display screen is positioned on the front side and the camera includes a lens is positioned on the reverse side, opposite the front side, at a point on the reverse side corresponding to a center of the display screen.

21. The electronic device of claim 1, wherein the image representative of the pointing object comprises an image of a user'"'"'s hand.

22. The electronic device of claim 1, wherein the image representative of the pointing object is superimposed onto the display screen so that the image representative of the pointing object appears to be above the object, and wherein the image representative of the pointing object is at least partially transparent.

23. The electronic device of claim 1, wherein the image representative of the pointing object is superimposed onto the display screen so that the image representative of the pointing object appears to be below the object, and wherein the object is at least partially transparent.

24. The electronic device of claim 1, wherein the controller is configured to perform an operation in response to a change in size of the image representative of the selection pointer.

25. The electronic device of claim 9, wherein the user input management unit is configured to capture an image of a pointing object that is positioned within a field of view of the camera and to superimpose the image of the pointing object onto the display screen so that the pointing object appears to be visible through the display screen.

26. The electronic device of claim 25, wherein the user input management unit is configured to detect relative movement of the pointing object toward and/or away from the camera in response to a change in size of the image of the pointing object.

10. A method of operating an electronic device including a user input device and a display screen, the method comprising:

superimposing a moving picture of a pointing object that is external to the electronic device onto the display screen wherein the picture of the pointing object is at least partially transparent; and

interpreting a plurality of features of the pointing object as selection pointers so that a movement of the pointing object relative to the display screen is interpreted as movement of a plurality of selection pointers.

11. The method of claim 10, further comprising interpreting movement of the plurality of selection pointers relative to one another as a selection command.

12. The method of claim 10, further comprising interpreting movement of two of the plurality of selection pointers into contact with each other as a selection command.

13. The method of claim 10, further comprising interpreting magnification of the pointing object as a zoom command, and increasing a magnification of an image on the display screen in response to magnification of the pointing object.

14. The method of claim 10, further comprising magnifying a portion of the object on the display screen in response to a selection region between the selection pointers being moved over the object on the display screen.

15. The method of claim 14, wherein a magnification level of the portion of the object on the display screen is determined in response to a spacing between the selection pointers.

16. The method of claim 10, further comprising magnifying portions of the display screen around the object, wherein a level of magnification of portions of the display screen around the object is proportional to a distance from the object.

17. A computer program product for operating a portable electronic device including a user input device and a display screen, the computer program product comprising:

a computer readable storage medium having computer readable program code embodied in said medium, said computer readable program code comprising;

computer readable program code configured to superimpose a moving picture of a pointing object that is external to the electronic device onto the display screen wherein the picture of the pointing object is at least partially transparent; and

computer readable program code configured to interpret a plurality of features of the pointing object as selection pointers so that a movement of the pointing object relative to the display screen is interpreted by the user input management unit as movement of a plurality of selection pointers.

18. The computer program product of claim 17, further comprising computer readable program code configured to interpret relative movement of the plurality of selection pointers as a selection command.

19. The computer program product of claim 17, further comprising computer readable program code configured to interpret movement of two of the plurality of selection pointers into contact with each other as a selection command.

20. The computer program product of claim 17, further comprising computer readable program code configured to interpret magnification of the pointing object as a zoom command, and to increase a magnification of an image on the display screen in response to magnification of the pointing object.