Viewing images with tilt-control on a hand-held device

• US 9,507,431 B2
• Filed: 03/26/2015
• Issued: 11/29/2016
• Est. Priority Date: 01/26/2007
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Cook, Ralph, Mlne, Anders L., Carsanaro, Joseph A.
• Primary Examiner(s): PATEL, PREMAL R
• Application Number:
  US14/670,276
• Publication Number:
  US 20150199026A1
• Time in Patent Office: 614 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 1/1613 for portable computers cooling arrangements therefor G06F1/203; constructional details or arrangements for pocket calculators, electronic agendas or books G06F15/0216; constructional details of portable telephone sets: with several bodies H04M1/0202
• CPC Class Codes: G06F 1/1637 Details related to the display arrangement, including those related to the mounting of the display in the housing
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G06T 3/60 Rotation of whole images or parts thereof
• CPC Class Codes: G09G 2340/04 Changes in size, position or resolution of an image
• CPC Class Codes: G09G 5/34 for rolling or scrolling

Abstract

A user interface suitable for use in cellular phones and personal digital assistants (PDAs), PC Tablets, as well as laptops, PCs, office equipment, medical equipment, or any other hand-held electronic device, that allows control of the image on the device display by tilting the device to either change the view in perspective, change the magnification, or both, concurrently, by moving the device. Thus, the tilt of the device controls the angle of view of the image, and moving the device perpendicular to the screen controls the magnification.

Claims

1. A method comprising:

determining, by a motion-sensing device of a handheld electronic device, orientation changes of the handheld electronic device over time;

correlating, by said handheld electronic device, the orientation changes with force data measured by an accelerometer to generate measurements of orientation with respect to a horizontal plane and inertial acceleration in three dimensions; and

based at least in part on the measurements of orientation, controlling, by said handheld electronic device, display of an object on a display panel of the handheld electronic device as a function of motion of said handheld electronic device, said function interpreting motion along a first axis as a first function that affects the display of the object on said display panel, and said function interpreting motion about a second axis distinct from the first axis as a second function that affects the display of the object on said display panel, wherein the first function is different from the second function.

2. The method of claim 1 wherein the orientation changes of the handheld electronic device are determined at least in part by comparing overlap in multiple sequential images captured by at least one optical sensor of the handheld electronic device.

3. The method of claim 2, wherein the accelerometer measures gravitational forces and inertial acceleration along three orthogonal axes to generate the force data and wherein the at least one optical sensor captures the multiple sequential images at known time intervals.

4. The method of claim 1, further comprising:

decreasing a magnification of the object on said display panel in response to moving the handheld electronic device in a first direction along the first axis, and increasing the magnification of the object on said display panel in response to moving the handheld electronic device in a second direction, opposite the first direction, along the first axis.

5. The method of claim 1, further comprising:

defining a threshold for a maximum tilt angle; and

in accordance with a determination that a tilt angle of the handheld electronic device has increased beyond the threshold for the maximum tilt angle, scrolling a display of the object on said display panel.

6. The method of claim 1, further comprising:

scrolling a display of the object on said display panel with increasing velocity as a tilt angle of said handheld electronic device increases.

7. The method of claim 6, wherein the scrolling of the display of the object on said display panel comprises orbiting around a perspective view of the object.

8. The method of claim 6, further comprising:

while scrolling the display of the object on said display panel, centering a current perspective view of the object on said display panel in response to a user action.

9. The method of claim 8, wherein the user action is a pre-defined motion of the device, a button push, a screen tap, or a voice command.

10. The method of claim 1, wherein the first function comprises a zoom function and wherein the second function comprises a change in perspective function.

11. A handheld electronic device comprising:

a processor;

a display panel in data communication with said processor;

a motion-sensing device in data communication with said processor for sensing movement of the handheld electronic device, the motion-sensing device including an accelerometer;

a memory device coupled to said processor, said memory device including instructions to be executed, wherein said instructions, when executed by said processor, cause said processor to;

determine, by the motion-sensing device, orientation changes of the handheld electronic device over time;

correlate, by said handheld electronic device, the orientation changes with force data measured by the accelerometer to generate measurements of orientation with respect to a horizontal plane and inertial acceleration in three dimensions; and

based at least in part on the measurements of orientation, control, by said handheld electronic device, display of an object on said display panel as a function of motion of said handheld electronic device, said function interpreting motion along a first axis as a first function that affects the display of the object on said display panel, and said function interpreting motion about a second axis distinct from the first axis as a second function that affects the display of the object on said display panel, wherein the first function is different from the second function.

12. The handheld electronic device of claim 11, wherein the handheld electronic device includes at least one optical sensor in data communication with said processor and the orientation changes of the handheld electronic device are determined at least in part by comparing overlap in multiple sequential images captured by the at least one optical sensor.

13. The handheld electronic device of claim 12, wherein the accelerometer measures gravitational forces and inertial acceleration along three orthogonal axes to generate the force data and wherein the at least one optical sensor captures the multiple sequential images at known time intervals.

14. The handheld electronic device of claim 11, wherein the instructions further cause the processor to:

decrease a magnification of the object on said display panel in response to moving the handheld electronic device in a first direction along the first axis, and increase the magnification of the object on said display panel in response to moving the handheld electronic device in a second direction, opposite the first direction, along the first axis.

15. The handheld electronic device of claim 11, wherein the instructions further cause the processor to:

define a threshold for a maximum tilt angle; and

in accordance with a determination that a tilt angle of the handheld electronic device has increased beyond the threshold for the maximum tilt angle, scroll a display of the object on said display panel.

16. The handheld electronic device of claim 11, wherein the instructions further cause the processor to:

scroll a display of the object on said display panel with increasing velocity as a tilt angle of said handheld electronic device increases.

17. The handheld electronic device of claim 16, wherein the scrolling of the display of the object on said display panel comprises orbiting around a perspective view of the object.

18. The handheld electronic device of claim 16, wherein the instructions further cause the processor to:

while scrolling the display of the object on said display panel, center a current perspective view of the object on said display panel in response to a user action.

19. The handheld electronic device of claim 18, wherein the user action is a pre-defined motion of the device, a button push, a screen tap, or a voice command.

20. The handheld electronic device of claim 11, wherein the first function comprises a zoom function and wherein the second function comprises a change in perspective function.

21. A non-transitory memory storing executable instructions that, when executed by an electronic device with a processor, a display panel in data communication with said processor, and a motion-sensing device in data communication with said processor for sensing movement of the handheld electronic device, the motion-sensing device including an accelerometer, cause the handheld electronic device to:

determine, by the motion-sensing device, orientation changes of the handheld electronic device over time;

correlate, by said handheld electronic device, the orientation changes with force data measured by the accelerometer to generate measurements of orientation with respect to a horizontal plane and inertial acceleration in three dimensions; and

based at least in part on the measurements of orientation, control, by said handheld electronic device, display of an object on said display panel as a function of motion of said handheld electronic device, said function interpreting motion along a first axis as a first function that affects the display of the object on said display panel, and said function interpreting motion about a second axis distinct from the first axis as a second function that affects the display of the object on said display panel, wherein the first function is different from the second function.

22. The non-transitory memory of claim 21, wherein the handheld electronic device includes at least one optical sensor in data communication with said processor and the orientation changes of the handheld electronic device are determined at least in part by comparing overlap in multiple sequential images captured by the at least one optical sensor.

23. The non-transitory memory of claim 22, wherein the accelerometer measures gravitational forces and inertial acceleration along three orthogonal axes to generate the force data and wherein the at least one optical sensor captures the multiple sequential images at known time intervals.

24. The non-transitory memory of claim 21, wherein the executable instructions further cause the handheld electronic device to:

decrease a magnification of the object on said display panel in response to moving the handheld electronic device in a first direction along the first axis, and increase the magnification of the object on said display panel in response to moving the handheld electronic device in a second direction, opposite the first direction, along the first axis.

25. The non-transitory memory of claim 21, wherein the executable instructions further cause the handheld electronic device to:

define a threshold for a maximum tilt angle; and

in accordance with a determination that a tilt angle of the handheld electronic device has increased beyond the threshold for the maximum tilt angle, scroll a display of the object on said display panel.

26. The non-transitory memory of claim 21, wherein the instructions further cause the handheld electronic device to:

scroll a display of the object on said display panel with increasing velocity as a tilt angle of said handheld electronic device increases.

27. The non-transitory memory of claim 26, wherein the scrolling of the display of the object on said display panel comprises orbiting around a perspective view of the object.

28. The non-transitory memory of claim 26, wherein the instructions further cause the handheld electronic device to:

while scrolling the display of the object on said display panel, center a current perspective view of the object on said display panel in response to a user action.

29. The non-transitory memory of claim 28, wherein the user action is a pre-defined motion of the device, a button push, a screen tap, or a voice command.

30. The non-transitory memory of claim 21, wherein the first function comprises a zoom function and wherein the second function comprises a change in perspective function.