Image control

• US 20060146009A1
• Filed: 01/22/2003
• Published: 07/06/2006
• Est. Priority Date: 01/22/2003
• Status: Abandoned Application

• Current Assignee: Nokia Corporation
• Original Assignee: Nokia Corporation
• Inventors: Van Walstijn, Bartholomeus Gerardus Gosewinu, Krischeu, Klaus, Syrbe, Hanno, Koivunen, Rami Arto
• Application Number:
  US10/540,052
• Publication Number:
  US 20060146009A1
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: A63F 13/00 Video games, i.e. games using an electronically generated display having two or more dimensions gaming systems which provide a financial reward G07F17/32
• CPC Class Codes: A63F 13/80 Special adaptations for executing a specific game genre or game mode
• CPC Class Codes: A63F 13/92 Video game devices specially adapted to be hand-held while playing
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/204 the platform being a handheld device
• CPC Class Codes: G06F 1/1616 with folding flat displays, e.g. laptop computers or notebooks having a clamshell configuration, with body parts pivoting to an open position around an axis parallel to the plane they define in closed position
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1684 Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 1/1698 the I/O peripheral being a sending/receiving arrangement to establish a cordless communication link, e.g. radio or infrared link, integrated cellular phone details of antennas disposed inside a computer H01Q1/2266
• CPC Class Codes: G06F 2200/1614 Image rotation following screen orientation, e.g. switching from landscape to portrait mode
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 3/0317 in co-operation with a patterned surface, e.g. absolute position or relative movement detection for an optical mouse or pen positioned with respect to a coded surface
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: H04M 1/72427 for supporting games or graphical animations
• CPC Class Codes: H04M 2250/12 including a sensor for measuring a physical value, e.g. temperature or motion telephones including GPS signal receivers H04M2250/10
• CPC Class Codes: H04M 2250/52 including functional features of a camera constructional features concerning the integration of cameras in portable phones H04M1/0264
• CPC Class Codes: H04N 2007/145 Handheld terminals
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q

Abstract

The present invention relates to the field of hand-held devices that are equipped wit a processor and a digital camera for capturing motion video or still images. Images captured by the camera are used to determine motion of the hand-held device. A resulting motion signal is used as input to a user interface. Displayed images can be scrolled, zoomed or rotated by moving the hand-held device. The motion signal can also be used as input for graphical user interface to move a cursor or other object of the graphical user interface over the display. The invention relates further to a hand-held device provided with means for sensing motion, a display, a keypad with at least a first- and a second key, and a graphical user interface with objects and a cursor. The hand-held device also includes means for transforming the sensed motion of the handheld device into a signal suitable for moving the cursor over the display.

Claims

1. A hand-held device comprising a processor, a digital camera for capturing motion video or still images, and means for transforming a signal from the camera into a motion signal indicative of the motion of the hand-held device.

2. A hand-held device according to claim 1, further comprising a user interface in which motion of the hand-held device is—

through the motion signal derived thereof—

used as a user input.

3. A hand-held device according to claim 1, further comprising a display suitable for displaying captured images.

4. A hand-held device according to claim 3, in which motion of a given type of the hand-held device is used to manipulate images shown at least in part on the display, preferably by moving the images in a manner substantially corresponding to the movement of the hand-held device.

5. A hand-held device according to claim 4, in which a given type of motion the hand-held device is used to move, and/or zoom, and/or expand/collapse and/or rotate images displayed on the display.

6. A hand-held device according to claim 5, in which motion substantially parallel to the plane of the display of the hand-held device is used to scroll an image displayed on the display, and/or motion substantially perpendicular to the plane of the display is used to zoom an image displayed on the display and/or rotational motion of the hand-held device is used to rotate an image displayed on the display.

7. A hand-held device according to claim 4, in which the images are images previously captured by the camera.

8. A hand-held device according to claim 4, in which movement of image is inverted with respect to motion of the hand-held device.

9. A hand-held device according to claim 2, in which the user interface comprises a graphical user interface, and wherein motion of the hand-held device is used as an input to the graphical user interface.

10. A hand-held device according to claim 9, in which motion of the hand-held device is used to manipulate an object displayed by the graphical user interface, preferably by moving the object in a manner substantially corresponding to the motion or to the inverted motion of the hand-held device, whereby the object displayed by the graphical user interface can be, an icon, a dialogue box, a window, a menu or a pointer.

11. A hand-held device according to claim 9, in which motion of a given type of the hand-held device is used to move, and/or zoom, and/or expand/collapse and/or rotate objects displayed by the graphical user interface.

12. A hand-held device according to claim 11, in which motion substantially parallel to the plane of the display of the hand-held device is used to scroll an object displayed by the graphical user interface, and/or motion substantially perpendicular to the plane of the display is used to zoom an object displayed by the graphical user interface and/or rotational motion of the hand-held device is used to rotate an object displayed by the graphical user interface.

13. A hand-held device according to claim 2, in which the digital camera is detachable.

14. A hand-held device according to claim 2, in which the digital camera is movable relative to the hand-held device.

15. A hand-held device according to claim 2, in which the means for transforming a signal from the camera into a motion signal derives the motion signal from changes between succeeding images, or parts of succeeding images captured by the camera.

16. A hand-held device according to claim 2, in which the camera has an autofocus system, whereby the focusing setting of the autofocus system is used for detecting movement in the camera direction.

17. A hand-held device according to claim 2, further comprising at least one key, wherein the functionality of a motion type is dependent on the state of the at least one key.

18. A hand-held device according to claim 2, in which rotational motion of the hand-held device about an axis substantially perpendicular to the display results in an inverse rotational movement of the image or graphical user interface object relative to the display, preferably in a manner such that the image or object is static with respect to the fixed coordinate system in which the hand-held device is situated.

19. A hand-held device according to claim 2, in which the motion signal is used to adjust device settings, the device settings preferably comprising sound settings and display settings.

20. A hand-held device according to claim 9, further comprising a keypad with at least a first- and a second key and the graphical user interface comprises a cursor, whereby motion of the hand-held device is used to position the cursor over an object of the graphical user interface and primary functions associated with the object concerned are activated by pressing the first key and secondary functions associated with the object of the concerned are activated by pressing the second key.

21. A hand-held device according to claim 20, in which the functionality of the first key is associated with selection and activation of objects of the graphical user interface, and in which the functionality of the second key is preferably associated with calling up a context-sensitive menu.

22. A hand-held device according to claim 21, in which selection of the object concerned is performed by pressing and releasing the first key, and activation of the object concerned is preferably performed by pressing and releasing the first key twice in rapid succession.

23. A hand-held device according to claim 21, in which moving or resizing of the object concerned is performed by holding down the first key while moving the hand-held device to move the cursor.

24. A hand-held device according to claim 20, in which the first key and the second key are softkeys whereby the current functionality of the softkeys is shown in the display, preferably in dedicated fields of the display.

25. A hand-held device according to claim 24, in which the first key is placed below the display on the left side of the latter, preferably proximate to lower edge of the display, and the second key is placed below the display on the right side of the latter, preferably proximate to lower edge of the display.

26. A hand-held device according to claim 1, further comprising at least one gravity based tilt sensor, and whereby the signal from the at least one tilt sensor is used in combination with the signal from the camera for creating the motion signal.

27. A hand-held device according to claim 26, wherein a tilt sensor is associated with the X-axis and/or a tilt sensor is associated with the z-axis.

28. A hand-held device according to claim 27, wherein the signal form the at least one tilt sensor is used to determined the absolute orientation of the handheld device relative to the direction of the gravitational pull.

29. A hand-held device according to claim 1, further comprising means for sending the motion signal to another terminal via cable, infrared waves or radio frequency waves.

30. A system comprising a hand-held device according to claim 29 and a terminal capable of displaying imaginary three-dimensional objects on a two-dimensional screen, said terminal comprising means to change the orientation of the displayed object in response to signals received from the handheld device, whereby orientation changes of the hand-held device are translated to corresponding orientation changes of the displayed object.

31. A system according to claim 30, in which position changes of the handheld device are translated to position changes of the displayed object.

32. A system comprising a hand-held device according to claim 29 and a terminal capable of displaying an imaginary three-dimensional space on a two-dimensional screen, said terminal comprising means to change the viewing position in the imaginary three-dimensional space in response to signals received from the handheld device, whereby positional changes of the hand-held device are translated to corresponding changes in the viewing position.

33. A system according to claim 30, in which orientation changes of the handheld device are translated into corresponding changes in the viewing direction in the imaginary three-dimensional space.

34. A method for creating user input for a hand-held device that has a processor, a user interface and a digital camera for capturing motion video or still images comprising the steps of:

determining motion of the hand-held device from the camera signal;

using the determined motion of the hand-held device as an input for the user interface.

35. Use of a digital camera for capturing motion video or still images of a hand-held device that has a processor to produce a motion signal indicative of motion of the hand-held device.

36. A hand-held device comprising a processor, means for sensing motion of the hand-held device, a display, a keypad with at least a first- and a second key, a graphical user interface with objects and a cursor, and means for transforming the sensed motion of the handheld device into a signal suitable for moving the cursor over the display.

37. A hand-held device according to claim 36, in which motion of the hand-held device is used to position the cursor over an object of the graphical user interface and primary functions associated with the object concerned are activated by pressing the first key and secondary functions associated with the object concerned are activated by pressing the second key.

38. A hand-held device according to claim 37, in which the functionality of the first key is associated with selection and activation of objects of the graphical user interface, and in which the functionality of the second key is preferably associated with calling up a context-sensitive menu.

39. A hand-held device according to claim 38, in which selection of the object concerned is performed by pressing and releasing the first key, and activation of the object concerned is preferably performed by pressing and releasing the first key twice in rapid succession.

40. A hand-held device according to claim 38, in which moving or resizing of the object concerned is performed by holding down the first key while moving the hand-held device to move the cursor and the object concerned in unison therewith.

41. A hand-held device according to claim 36, in which the first key and the second key are softkeys whereby the current functionality of the softkeys is shown in the display, preferably in dedicated fields of the display.

42. A hand-held device according to claim 41, in which the first key is placed below the display on the left side of the latter, preferably proximate to lower edge of the display, and the second key is placed below the display on the right side of the latter, preferably proximate to lower edge of the display.

43. A hand-held device according to claim 36, in which said means for transforming motion of the handheld device into a signal suitable for moving the cursor over the display comprises a tilt sensor and/or an image capturing device and/or an accelerometer.

44. A hand-held device according to claim 43, in which said image capturing device is a motion video or still image digital camera.

45. A hand-held device according to claim 36, further comprising means to transform a signal from the camera and/or tilt sensor and/or accelerometer into a position signal for the cursor.

46. A hand-held device according to claim 45, in which said means for transforming a signal from the camera into a motion signal derives the motion signal from the changes between succeeding images, or parts of succeeding images captured by the camera.

47. A hand-held device according to claim 44, in which the camera has an autofocus system, whereby the focusing setting of the autofocus system is used for detecting movement in the camera direction.

48. A hand-held device according to claim 36, in which the graphical user interface includes one or more of the following object types:

icons, dialogue boxes, windows, menus, pointers.