Electronic Device and a Method for Controlling the Functions of the Electronic Device as Well as Program Product for Implementing the Method

• US 20080297474A1
• Filed: 08/16/2005
• Published: 12/04/2008
• Est. Priority Date: 08/17/2004
• Status: Active Grant

• Current Assignee: Conversant Wireless Licensing S.à.r.l. (f/k/a Core Wireless Licensing S.a.r.l.) (MOSAID Technologies Inc. (f/k/a Conversant Intellectual Property Management))
• Original Assignee: 2011 Intellectual Property Asset Trust (Nokia Corporation)
• Inventors: Blomqvist, Mikko, Laine, Tommi, Onkalo, Mila
• Granted Patent: US 8,970,490 B2
• Time in Patent Office: Days
• US Class Current: 345/158
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: H04M 1/72427 for supporting games or graphical animations

Abstract

The invention relates to an electronic device, which includes a display component, in which at least one controllable element is arranged to be visualized in its entirety, the control of which element is arranged to be based on determining a change (M) relating to the attitude or position of the device and camera means arranged to form image frames (IMAGE1, IMAGE2), and in which, in order to control the functions of the device, such as, for example, to control the element, the change is arranged to be determined, using optical data-flow algorithms, from the image frames formed by the camera means, which image frames are arranged to be compared to each other in a set manner, and on the basis of which comparison the said change is arranged to be determined. The image frames are arranged to be segmented in a selected manner, from which at least one movement vector (S_R, S_L, S_Up, S_Down) determining the said change is arranged to be determined from at least some of the segments (SEGMENT1, SEGMENT2).

Claims

1. Electronic device, which includes

a display component, in which at least one controllable element is arranged to be visualized in its entirety, the control of which element is arranged to be based on determining a change (M) relating to the attitude or position of the device relative to at least one selected fixed point and

camera means arranged to form consecutive image frames (IMAGE1, IMAGE2),

and in which, in order to control the functions of the device, such as, for example, to control the said element, the said change (M) is arranged to be determined, using optical data-flow algorithms, from the consecutive image frames (IMAGE1, IMAGE2) formed by the camera means, which consecutive image frames are arranged to be compared to each other in a set manner, and on the basis of which comparison the said change (M) is arranged to be determined, wherein the consecutive image frames (IMAGE1, IMAGE2) are arranged to be segmented in a selected manner, from which at least one movement vector (S_R, S_L, S_Up, S_Down) defining the said change (M) is arranged to be determined from at least some of the segments (SEGMENT1, SEGMENT2).

2. Device according to claim 1, where said camera means include at least two camera units, which are arranged on opposite sides of the device.

3. Device according to claim 1, where at least one fixed point is arranged to be determined from the image frames (IMAGE1, IMAGE2), for which fixed point the said change (M) is arranged to be determined from the location change (S, S_R, S_L, S_Up, S_Down) determined from the image frame (IMAGE1, IMAGE2).

4. Device according to claim 1, where using at least some of the movement vectors (S_R, S_L, S_Up, S_Down) are arranged to be determined their mean value.

5. Device according to claim 1, where the definition frequency of the change (M)/the imaging frequency of the camera means is arranged to be adjustable according to the current operating situation of the device.

6. Device according to claim 1, where the camera means are arranged to form image frames (IMAGE1, IMAGE2) substantially less frequently compared to the conventional formation frequency.

7. Method for controlling the operations of an electronic device, in which the operations of the device, such as, for example, controlling of at least one element arranged to be visualized in its entirety and controlled on the display component, is based on determining a change (M) relating to the attitude or position of the device (10) relative to at least one selected fixed point, and in which method camera means are used to form consecutive image frames (IMAGE1, IMAGE2) in order to determine the said change (M), using optical data-flow algorithms, which are compared to each other in a set manner and on the basis of which comparison the said change (M) is determined, where the consecutive image frames (IMAGE1, IMAGE2) are segmented in a set manner, for at least some of which segments (SEGMENT1, SEGMENT2) at least one movement vector (S_R, S_L, S_Up, S_Down) defining the said change (M) is determined.

8. Method according to claim 7, where the image frames (IMAGE1, IMAGE2) are formed using the camera means, which include at least two camera units arranged on opposite sides of the device.

9. Method according to claim 7, where from the image frames (IMAGE1, IMAGE2) at least one fixed point is defined, for which the said change (M) is determined on the basis of the change in location (S, S_R, S_L, S_Up, S_Down) determined from the image frames (IMAGE1, IMAGE2).

10. Method according to claim 7, where using at least some of the movement vectors (S_R, S_L, S_Up, S_Down) are determined their mean value.

11. Method according to claim 7, where the frequency of the determining of the change (M)/imaging frequency of the camera means is adjusted according to the current operating situation of the device.

12. Method according to claim 7, where the image frames (IMAGE1, IMAGE2) are formed substantially less frequently compared to a conventional formation frequency.

13. Program product for implementing the method according to the invention in an electronic device according to the invention, which program product includes a memory medium (MEM) and program code to be executed using processor means and written on the memory medium (MEM), which program code is arranged to control the functions of the device, such as, for example, at least one element arranged to be visualized in its entirety and controlled on the display component, based on defining a change (M) relating to the attitude or position of the device relative to at least one selected fixed point, and in which the program code includes

code means configured to determine the said change (M) from the consecutive image frames (IMAGE1, IMAGE2) formed by camera means using the optical data-flow algorithms and

which program code is configured to compare the consecutive image frames (IMAGE1, IMAGE2) with each other in a set manner and, on the basis of which comparison, the program code is arranged to be determined the change (M) in the attitude or position of the device relative to at least one fixed point selected from the consecutive image frames (IMAGE1, IMAGE2), where the program code additionally includes code means configured to segment the consecutive image frames (IMAGE1, IMAGE2) in a selected manner, from at least some of the segments (SEGMENT1, SEGMENT2) the change (M), relating to the attitude or position of the device (10) relative to the selected fixed point, is arranged to be determined as at least one movement vector (S, S_R, S_L, S_Up, S_Down).

14. Program product according to claim 13, where the program code includes code means configured to analyse the image information (IMAGE1, IMAGE2, IMAGE1″

, IMAGE2″

) formed on the opposite sides of the device.

15. Program product according to claim 13, where the program code includes code means configured to determine at least one fixed point from the image frames (IMAGE1, IMAGE2), for which fixed point the change (M) relating to the attitude or position of the device relative to at least one selected fixed point is arranged to be determined from the location change (S, S_R, S_L, S_Up, S_Down) determined from the image frame (IMAGE1, IMAGE2).

16. Program product according to claim 13, where the said code means is configured to determine a mean value using at least some of the movement vectors (S_R, S_L, S_Up, S_Down).

17. Program product according to claim 13, where the program code includes code means for adjusting the frequency of the definition of the change (M)/the imaging frequency of the camera means according to the current operating situation of the device.