MOBILE DEVICE USER INTERFACE COMBINING INPUT FROM MOTION SENSORS AND OTHER CONTROLS

• US 20110316888A1
• Filed: 06/28/2011
• Published: 12/29/2011
• Est. Priority Date: 06/28/2010
• Status: Abandoned Application

• Current Assignee: Invensense Incorporated (TDK Corporation)
• Original Assignee: Invensense Incorporated (TDK Corporation)
• Inventors: NASIRI, Steven S., SACHS, David
• Application Number:
  US13/171,298
• Publication Number:
  US 20110316888A1
• Time in Patent Office: Days
• US Class Current: 345/667
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

Various embodiments provide user interfaces for mobile devices which combine input from motion sensors and other input controls. In one aspect, a handheld electronic device includes a display operative to display an image, an input control operative to sense a contact motion of the user with the device, a set of motion sensors sensing rotational rate of the device around at least three axes of the device and linear acceleration along at least three axes of the device, and a subsystem capable of facilitating interaction with the device based on combined sensor data. The combined sensor data includes motion data derived from at least one of the motion sensors and contact data derived from the contact motion sensed by the input control.

Claims

1. A handheld electronic device, the device comprising:

a display operative to display an image;

an input control operative to sense a contact motion of the user with the device;

a set of motion sensors sensing rotational rate of the device around at least three axes of the device and linear acceleration along at least three axes of the device; and

a subsystem capable of facilitating interaction with the device based on combined sensor data, the combined sensor data including motion data derived from at least one of the motion sensors and contact data derived from the contact motion sensed by the input control.

2. The electronic device of claim 1, wherein the input control is a touchscreen sensor of the display, and wherein the contact motion is motion of a user contacting a surface of the display.

3. The electronic device of claim 1, wherein the input control is one of the following:

a capacitive sensor strip, a trackball, an optical sensor, a mechanical wheel, or a series of buttons.

4. The electronic device of claim 1, wherein the motion sensors sensing rotational rate are gyroscopes and the motion sensors sensing linear acceleration are accelerometers.

5. The electronic device of claim 1, wherein the combined sensor data including the motion data and the contact data is derived from simultaneous contact motion and device motion provided by the user.

6. The electronic device of claim 2, wherein the display is controlled to display at least one control region that is operative to provide the contact data based on the touch of the user in the at least one control region.

7. The electronic device of claim 6 wherein the at least one control region is operative to enable the motion data to be included in the combined sensor data to facilitate the interaction with the device.

8. The electronic device of claim 6, wherein the control region is at least one strip region approximately at a side of the display.

9. The electronic device of claim 6, wherein user sliding contact on the surface of the display in the control region is operative to control a zooming function for an image displayed by the display.

10. The electronic device of claim 9, wherein rotation of the device around an x-axis of the device as sensed by the motion sensors is operative to control a first panning function for the image displayed by the display.

11. The electronic device of claim 10, wherein rotation of the device around a y-axis of the device as sensed by the motion sensors is operative to control a second panning function for the image, the second panning function operating along a different axis of the displayed image than the first panning function.

12. The electronic device of claim 1, wherein

the motion data is used to control a panning function for an image displayed on the display, and

the contact data is used to control a zooming function for the image displayed on the display.

13. The electronic device of claim 1, wherein

the motion data is used to control a zooming function for an image displayed on the display, and

the contact data is used to control a panning function for the image displayed on the display.

14. The electronic device of claim 9, wherein a displayed element of the image is selected in response to a predetermined zoom level of the zooming function being reached.

15. A method for providing interaction with a handheld electronic device, the method comprising:

sensing contact motion input from a user using an input control of the handheld electronic device;

sensing motion of the device around at least one axis of the device using a set of motion sensors, the set of motion sensors operative to sense rotational rate around at least three axes of the device and linear acceleration along at least three axes of the device; and

providing interaction with the device based on combined sensor data, the combined sensor data including motion data derived from at least one of the motion sensors and contact data derived from the contact motion sensed by the input control.

16. The method of claim 15, wherein providing interaction with the device includes modifying or selecting at least a portion of an image displayed by a display of the handheld electronic device based on the combined sensor data.

17. The method of claim 16, wherein the input control is a touchscreen sensor included in the display, and wherein the contact motion is motion of a user contacting a surface of the display.

18. The method of claim 17, wherein the display is controlled to display at least one control region that senses the contact motion input from the user and is operative to provide the contact data based on the touch of the user and to enable the motion data to be included in the combined sensor data to facilitate the interaction with the device.

19. The method of claim 15, wherein

the motion data is used to control a panning function for an image displayed on the display, and

the contact data is used to control a zooming function for the image displayed on the display,

wherein the panning function and the zooming function are provided simultaneously to create the combined sensor data.

20. A storage medium including a software program, the software program capable of running at least partially on a handheld electronic device,

wherein the device comprises:

a touchscreen display operative to display an image and to sense contact on a surface of the touchscreen display; and

a set of motion sensors sensing rotational rate around at least three axes and linear acceleration along at least three axes,

wherein the software program is capable of facilitating interaction with the device based on combined sensor data, the combined sensor data including motion data derived from at least one of the motion sensors and contact data derived from the contact sensed by the touchscreen display.