Motion-based interface control on computing device

• US 8,977,987 B1
• Filed: 06/14/2010
• Issued: 03/10/2015
• Est. Priority Date: 06/14/2010
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Korn, Jeffrey, Held, David
• Primary Examiner(s): PITARO, RYAN F
• Application Number:
  US12/815,096
• Time in Patent Office: 1,730 Days
• Field of Search: 715/864, 715/802, 715/829, 715/831, 715/863, 715/702
• US Class Current: 715/863
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

A computer-implemented user interface method include displaying an image on a graphical user interface of a mobile computing device, receiving a first user input indicating an intent to perform a zooming operation on the graphical user interface, transitioning the mobile computing device into a zooming mode in response to the user input, sensing motion of the mobile computing device in one or more of a plurality of directions, correlating the sensed motion in one or more of a plurality of directions with either a zooming in or a zooming out zooming direction, and changing a zoom level of the display of the image on the graphical user interface to correspond to the correlated zooming direction.

Claims

1. A computer-implemented user interface method, comprising:

displaying an image on a graphical user interface of a mobile computing device;

receiving a first user input on a touchscreen of the mobile computing device;

sensing motion of the mobile computing device in one or more of a plurality of directions, wherein the sensed motion comprises a sensed tilting of the mobile computing device about an axis of the mobile computing device;

determining whether to interpret the sensed motion as indicating a zooming input based on the first user input, wherein the sensed motion is interpreted as indicating a zooming input when the user is determined to have been contacting the touchscreen while the sensed motion was detected by the mobile computing device;

based on the determination of whether to interpret the sensed motion as indicating a zooming input, correlating the sensed motion with either a zooming in or a zooming out zooming direction;

determining a tilt angle of the mobile computing device;

identifying the tilt angle as being within a particular tilt angle range from among a plurality of tilt angle ranges, wherein each respective tilt angle range from among a plurality of tilt angle ranges, wherein each respective tilt angle range from among the plurality of tilt angle ranges is associated with a distinct zoom level of a plurality of zoom levels;

identifying a particular zoom level of the plurality of zoom levels as being associated with the particular tilt angle range; and

changing a display of the image on the graphical user interface to correspond to the particular zoom level in the correlated zooming direction.

2. The method of claim 1, wherein the motion is sensed by using an accelerometer in the mobile computing device.

3. The method of claim 1, further comprising providing on the touchscreen a zooming icon, and wherein receiving a user input indicating an intent to perform a zooming function comprises receiving a user selection of the zooming icon.

4. The method of claim 1, wherein the image comprises a geographical map.

5. The method of claim 1, wherein the image comprises a rendered web page.

6. The method of claim 1, wherein the image comprises a digital image displayed in an image viewing application executing on the mobile computing device.

7. The method of claim 1, wherein, when the sensed motion is correlated with the zooming in direction, the zoom level of the display is increased and is centered based on a location on the touchscreen at which the user touched the touchscreen.

8. The method of claim 1, further comprising:

receiving an indication that a user of the mobile computing device has stopped touching the touchscreen of the mobile computing device; and

in response to receiving the indication, removing the correlation of the sensed motion with either the zooming in or the zooming out zooming direction, wherein removing the correlation causes the zoom level of the display to stop being changed.

9. The method of claim 1, further comprising:

based on the determination of whether to interpret the sensed motion as indicating a zooming input, displaying a first label positioned near an edge of the graphical user interface and a second label positioned near an opposite edge of the graphical user interface, wherein the first label indicates that motion of the mobile computing device in a direction toward the edge of the graphical user interface will cause a zoom in operation to be performed, and wherein the second label indicates that motion of the mobile computing device in an opposite direction toward the opposite edge of the graphical user interface will cause a zoom out operation to be performed.

10. A computer-implemented user interface method, comprising:

displaying an image on a graphical user interface of a mobile computing device;

receiving a first user input on a touchscreen of the mobile computing device that indicates that a user of the mobile computing device has contacted the touchscreen;

sensing motion of the mobile computing device in one or more of a plurality of directions;

determining whether to interpret the sensed motion as indicating a zooming input based on the first user input, wherein the sensed motion is interpreted as indicating a zooming input when the user is determined to have been contacting the touchscreen while the sensed motion was detected by the mobile computing device;

based on the determination of whether to interpret the sensed motion as indicating a zooming input, correlating the sensed motion in one or more of a plurality of directions with either a zooming in or a zooming out zooming direction;

changing a zoom level of the display of the image on the graphical user interface to correspond to the correlated zooming direction;

while changing the zoom level of the display, receiving a second user input that indicates an intent to perform a panning operation on the graphical user interface, wherein the second user input comprises a dragging input defined by a user maintaining contact with the touchscreen while moving a point of contact on the touchscreen;

correlating motion of the dragging input with one or more panning directions; and

panning the content in a direction of the one or more correlated panning directions;

wherein the image comprises a list of files, sensed tilting of the device in a direction that is aligned with the list causes the list to scroll, and sensed motion of the device in a direction that is not aligned with the list causes the list to zoom.

11. The method of claim 10, wherein zooming of the list comprises shifting from a listing of single alphanumeric characters to a listing of each item in the list in an alphanumeric order.

12. A computer-implemented user interface method, comprising:

displaying an image on a graphical user interface of a mobile computing device;

receiving a first user input on a touchscreen of the mobile computing device that indicates that a user of the mobile computing device has contacted the touchscreen;

sensing motion of the mobile computing device in one or more of a plurality of directions;

determining whether to interpret the sensed motion as indicating a zooming input based on the first user input, wherein the sensed motion is interpreted as indicating a zooming input when the user is determined to have been contacting the touchscreen while the sensed motion was detected by the mobile computing device;

based on the determination of whether to interpret the sensed motion as indicating a zooming input, correlating the sensed motion in one or more of a plurality of directions with either a zooming in or a zooming out zooming direction;

changing a zoom level of the display of the image on the graphical user interface to correspond to the correlated zooming direction;

while changing the zoom level of the display, receiving a second user input that indicates an intent to perform a panning operation on the graphical user interface, wherein the second user input comprises a dragging input defined by a user maintaining contact with the touchscreen while moving a point of contact on the touchscreen;

correlating motion of the dragging input with one or more panning directions; and

panning the content in a direction of the one or more correlated panning directions;

wherein the sensed motion comprises lateral motion of the mobile computing device and the zooming occurs in a direction that corresponds to a direction of the lateral motion.

13. An article comprising a non-transitory computer-readable data storage medium storing program code that is operable to cause one or more machines to perform operations, the operations comprising:

displaying an image on a graphical user interface of a mobile computing device;

receiving a first user input on a touchscreen of the mobile computing device;

sensing motion of the mobile computing device in one or more of a plurality of directions, wherein the sensed motion comprises a sensed tilting of the mobile computing device about an axis of the mobile computing device;

determining whether to interpret the sensed motion as indicating a zooming input based on the first user input, wherein the sensed motion is interpreted as indicating a zooming input when the user is determined to have been contacting the touchscreen while the sensed motion was detected by the mobile computing device;

based on the determination of whether to interpret the sensed motion as indicating a zooming input, correlating the sensed motion with either a zooming in or a zooming out zooming direction;

determining a tilt angle of the mobile computing device;

identifying the tilt angle as being within a particular tilt angle range from among a plurality of tilt angle ranges, wherein each respective tilt angle range from among the plurality of tilt angle ranges is associated with a distinct zoom level of a plurality of zoom levels;

identifying a particular zoom level of the plurality of zoom levels as being associated with the particular tilt angle range; and

changing a display of the image on the graphical user interface to correspond to the particular zoom level in the correlated zooming direction.

14. The article of claim 13, wherein the operations further comprise providing a zooming icon on the touchscreen, and wherein receiving a user input indicating an intent to perform a zooming function comprises receiving a user selection of the zooming icon.

15. The article of claim 13, wherein the image comprises a list of files, sensed tilting of the device in a direction that is aligned with the list causes the list to scroll, and sensed motion of the device in a direction that is not aligned with the list causes the list to zoom.

16. The article of claim 15, wherein zooming of the list comprises shifting from a listing of single alphanumeric characters to a listing of each item in the list in an alphanumeric order.

17. A mobile computing device, comprising:

a touch input manager to receive and interpret user inputs on a touch input device of a computing device, wherein the touch input manager is arranged to receive first user input that indicates that a user of the mobile computing device has contacted a touchscreen of the mobile computing device and a second user input comprising a dragging input defined by the user maintaining contact with the touchscreen while moving a point of contact on the touchscreen;

one or more sensors arranged to sense tilting motion of the mobile computing device about an axis of the mobile computing device;

one or more computing applications stored on the mobile computing device; and

an input method editor programmed to;

receive information indicating that tilting motion of the mobile computing device was sensed by the one or more motion sensors;

determine whether to interpret the sensed tilting motion as indicating zooming input or other input based on the first user input on the touchscreen of the mobile computing device;

determine a tilt angle of the mobile computing device,

identify the tilt angle as being within a particular tilt angle range from among a plurality of tilt angle ranges, wherein each respective tilt angle range from among the plurality of tilt angle ranges is associated with a distinct zoom level of a plurality of zoom levels;

identify a particular zoom level of the plurality of zoom levels as being associated with the particular tilt angle range; and

provide data relating to the received information to a plurality of different applications that include the one or more applications if the sensed tilting motion is interpreted as indicating a zooming input,

wherein the sensed tilting motion is interpreted as indicating a zooming input when the user is determined to have been contacting the touchscreen while the sensed tilting motion was detected by the mobile computing device, and

wherein the one or more applications are programmed to convert the data from the input method editor into commands for changing a display of an image on the computing device to correspond to the particular zoom level.

18. The device of claim 17, further comprising an application programmed to obtain one or more applications from an in-line application store, wherein the obtained applications include interfaces for interacting with the input method editor.

19. A computer-implemented user interface method, comprising:

displaying an image on a graphical user interface of a mobile computing device;

receiving a first user input on a touchscreen of the mobile computing device that indicates that a user of the mobile computing device has contacted the touchscreen;

sensing motion of the mobile computing device in one or more of a plurality of directions;

determining whether to interpret the sensed motion as indicating a zooming input based on the first user input, wherein the sensed motion is interpreted as indicating a zooming input when the user is determined to have been contacting the touchscreen while the sensed motion was detected by the mobile computing device;

based on the determination of whether to interpret the sensed motion as indicating a zooming input, correlating the sensed motion in one or more of a plurality of directions with either a zooming in or a zooming out zooming direction;

changing a zoom level of the display of the image on the graphical user interface to correspond to the correlated zooming direction;

while changing the zoom level of the display, receiving a second user input that indicates an intent to perform a panning operation on the graphical user interface, wherein the second user input comprises a dragging input defined by a user maintaining contact with the touchscreen while moving a point of contact on the touchscreen;

correlating motion of the dragging input with one or more panning directions;

panning the content in a direction of the one or more correlated panning directions; and

based on the determination of whether to interpret the sensed motion as indicating a zooming input, displaying a first label positioned near an edge of the graphical user interface and a second label positioned near an opposite edge of the graphical user interface, wherein the first label indicates that motion of the mobile computing device in a direction toward the edge of the graphical user interface will cause a zoom in operation to be performed, and wherein the second label indicates that motion of the mobile computing device in an opposite direction toward the opposite edge of the graphical user interface will cause a zoom out operation to be performed.