VIEWING IMAGES WITH TILT CONTROL ON A HAND-HELD DEVICE

• US 20100171691A1
• Filed: 01/25/2008
• Published: 07/08/2010
• Est. Priority Date: 01/26/2007
• Status: Active Grant

• Current Assignee: Motionip LLC
• Original Assignee: Motionip LLC
• Inventors: Mölne, Anders L., Carsanaro, Joseph A., Cook, Ralph
• Granted Patent: US 8,994,644 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 1/1613 for portable computers cooling arrangements therefor G06F1/203; constructional details or arrangements for pocket calculators, electronic agendas or books G06F15/0216; constructional details of portable telephone sets: with several bodies H04M1/0202
• CPC Class Codes: G06F 1/1637 Details related to the display arrangement, including those related to the mounting of the display in the housing
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G06T 3/60 Rotation of whole images or parts thereof
• CPC Class Codes: G09G 2340/04 Changes in size, position or resolution of an image
• CPC Class Codes: G09G 5/34 for rolling or scrolling

Abstract

A user interface suitable for use in cellular phones and personal digital assistants (PDAs), PC Tablets, as well as laptops, PCs, office equipment, medical equipment, or any other hand-held electronic device, that allows control of the image on the device display by tilting the device to either change the view in perspective, change the magnification, or both, concurrently, by moving the device. Thus, the tilt of the device controls the angle of view of the image, and moving the device perpendicular to the screen controls the magnification.

Claims

1. A handheld electronic device comprising:

a processor with memory;

a display panel in data communication with said processor;

a user-input assembly in data communication with said processor;

one or more motion-sensing devices in data communication with said processor for sensing movement of the handheld electronic device and outputting a motion signal indicative thereof;

software resident in said memory for interpreting said motion signal and for controlling display of an object being viewed on said display panel as a function of motion of said handheld electronic device, said function interpreting motion along an axis as zoom to emulate magnification as if said object were being viewed through a magnifying glass, and said function interpreting motion about another axis as scrolling, thereby eliminating need for separate controls for scrolling and magnification.

2. The handheld electronic device according to claim 1, wherein said one or more motion-sensing devices further comprise:

at least one accelerometer to measure gravitational forces and inertial acceleration along three orthogonal axes; and

at least one optical sensor for capturing multiple consecutive images at known time intervals.

3. The handheld electronic device according to claim 2, further comprising software for comparing overlap in said multiple consecutive images and for determining relative orientation changes over time.

4. The handheld electronic device according to claim 3, wherein said software correlates said relative orientation changes over time with force data measured by said at least one accelerometer to generate measurements of orientation with respect to a horizontal plane and inertial acceleration in three dimensions.

5. The handheld electronic device according to claim 4, wherein the relative orientation changes over time detected by said optical sensor comprise rotation around an axis substantially aligned with the total force vector comprised of gravitation and inertia, to compensate for inherent inability of the accelerometer to measure such motion.

6. The handheld electronic device according to claim 4, wherein the relative orientation changes over time detected by said optical sensor are compared with the changes in total force measured along three axes by the accelerometer in order to calculate an approximate direction of the gravitational vector, which is then used to subtract an assumed local gravitational force from total force measurements, resulting in independent measurements of the gravitational and inertial forces along three axes.

7. The handheld electronic device according to claim 1, wherein said one or more motion-sensing devices further comprise one or more from among a group comprising:

accelerometer(s) that measure the combined forces of gravitational acceleration and inertial acceleration along two or more axes;

optical sensor(s) that capture multiple consecutive images of the sensor'"'"'s environment at known time intervals and to approximate relative changes in device orientation or position based on the overlap observed in consecutive images;

gyroscope(s) that measure relative changes in device orientation; and

an Inertial Measurement Unit for measuring both orientation and inertial acceleration.

8. The handheld electronic device according to claim 4, wherein when a user moves the handheld device along a line perpendicular to said display panel closer or away from the user, the software interprets said motion signal and controls magnification of an object being viewed on said display panel as if said object were being viewed with greater or lesser magnification, respectively.

9. The handheld electronic device according to claim 8, wherein moving the display panel closer to the user causes a decrease in magnification, and moving the display further away from the user causes an increase in magnification.

10. A method for controlling a display of a handheld electronic device

comprising, the steps of:

placing the handheld electronic device in a “

tilt mode”

;

sensing movement of said handheld electronic device when in said tilt mode and outputting a motion signal indicative thereof;

interpreting said motion signal and controlling magnification of an object being viewed on said display panel as if said object were being viewed through a virtual magnifying glass controlled by said motion of said handheld electronic device.

11. The method for controlling a display of a handheld electronic device according to claim 10, wherein said step of sensing movement further comprises accepting inputs from the device indicating that the device has been tilted around either or both of the axes parallel to the top and side edges of the screen.

12. The method for controlling a display of a handheld electronic device according to claim 11, wherein said step of interpreting said motion signal further comprises responding to said input signal by tilting the angle of view of the object being viewed as though through a camera looking at the object.

13. The method for controlling a display of a handheld electronic device according to claim 11, wherein said tilting the angle of view comprises tilting the device to the left along the axis parallel to the edge of the view screen causes the display to change so that (1) the center of the field of view moves to the left, (2) there is more of the viewed image visible to the left, and more of the image visible left of the line from the viewpoint to the center of the field of view, than there was before the device was tilted.

14. The method for controlling a display of a handheld electronic device according to claim 13, wherein a portion of the image that is made closer to the user by virtue of the tilt appears larger than the part of the image made further away from the user by virtue of the tilt.

15. The method for controlling a display of a handheld electronic device according to claim 10, wherein there is a “

maximum tilt angle”

used by the software for tilting the viewed image, and wherein, if the user holds the display within a certain tolerance of that maximum tilt angle, the software scrolls the perspective display of the viewed image, while holding the perspective aspect of the view constant.

16. The method for controlling a display of a handheld electronic device according to claim 10, wherein the method defines a user motion to “

center”

the current view, whereupon said user motion makes the current center of the field of view a normalized centerpoint for a new view.

17. The method for controlling a display of a handheld electronic device according to claim 16, wherein the user motion to “

center”

the current view comprises any one from among a group comprising a pre-defined motion of the device, a button push, a screen tap, and a voice command.

18. The method for controlling a display of a handheld electronic device according to claim 10, wherein the method defines a motion of the handheld device to change the viewpoint in relation to a 3 dimensional image being viewed.

19. The method for controlling a display of a handheld electronic device according to claim 10, wherein said step of interpreting movement further comprises responding to said input signal by changing said viewpoint as if orbiting relative to a 3 dimensional image being viewed.

20. The method for controlling a display of a handheld electronic device according to claim 10, wherein said step of interpreting movement further comprises maintaining a user'"'"'s view of a virtual surface being observed as stationary when the center of the field of view is perpendicular to the virtual surface being observed, and scrolling with increasing velocity toward the center of the user'"'"'s field of view as a tilt angle of said handheld device increases.