CONTROLLING AND ACCESSING CONTENT USING MOTION PROCESSING ON MOBILE DEVICES

• US 20090262074A1
• Filed: 03/04/2009
• Published: 10/22/2009
• Est. Priority Date: 01/05/2007
• Status: Abandoned Application

• Current Assignee: Invensense Incorporated (TDK Corporation)
• Original Assignee: Invensense Incorporated (TDK Corporation)
• Inventors: SACHS, DAVID, NASIRI, STEVEN S.
• Application Number:
  US12/398,156
• Publication Number:
  US 20090262074A1
• Time in Patent Office: Days
• US Class Current: 345/158
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/285 Generating tactile feedback signals via the game input device, e.g. force feedback
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/533 for prompting the player, e.g. by displaying a game menu
• CPC Class Codes: A63F 13/5378 for displaying an additional top view, e.g. radar screens or maps using two or more virtual cameras concurrently A63F13/5252
• CPC Class Codes: A63F 13/71 using secure communication between game devices and game servers, e.g. by encrypting game data or authenticating players
• CPC Class Codes: A63F 13/92 Video game devices specially adapted to be hand-held while playing
• CPC Class Codes: A63F 2300/1006 having additional degrees of freedom
• CPC Class Codes: A63F 2300/1037 being specially adapted for converting control signals received from the game device into a haptic signal, e.g. using force feedback
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/204 the platform being a handheld device
• CPC Class Codes: A63F 2300/209 characterized by low level software layer, relating to hardware management, e.g. Operating System, Application Programming Interface
• CPC Class Codes: A63F 2300/307 for displaying an additional window with a view from the top of the game field, e.g. radar screen
• CPC Class Codes: A63F 2300/308 Details of the user interface
• CPC Class Codes: A63F 2300/401 Secure communication, e.g. using encryption or authentication
• CPC Class Codes: A63F 2300/532 using secure communication, e.g. by encryption, authentication
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: G03B 17/00 Details of cameras or camera bodies; Accessories therefor lens hoods or caps G03B11/04
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 2200/1636 Sensing arrangement for detection of a tap gesture on the housing
• CPC Class Codes: G06F 2200/1637 : Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 2203/04806 : Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/0236 : using selection techniques to select from displayed items
• CPC Class Codes: G06F 3/0346 : with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0481 : based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04817 : using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 : Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04845 : for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0485 : Scrolling or panning
• CPC Class Codes: H04M 2250/12 : including a sensor for measuring a physical value, e.g. temperature or motion telephones including GPS signal receivers H04M2250/10
• CPC Class Codes: H04W 12/06 : Authentication
• CPC Class Codes: H04W 12/68 : Gesture-dependent or behaviour-dependent
• CPC Class Codes: H04W 88/02 : Terminal devices

Abstract

Various embodiments provide systems and methods capable of facilitating interaction with handheld electronics devices based on sensing rotational rate around at least three axes and linear acceleration along at least three axes. In one aspect, a handheld electronic device includes a subsystem providing display capability, a set of motion sensors sensing rotational rate around at least three axes and linear acceleration along at least three axes, and a subsystem which, based on motion data derived from at least one of the motion sensors, is capable of facilitating interaction with the device.

Claims

1. A handheld electronic device, the device comprising:

a subsystem providing display capability;

a set of motion sensors sensing rotational rate around at least three axes and linear acceleration along at least three axes; and

a subsystem which, based on motion data derived from at least one of the motion sensors, is capable of facilitating interaction with the device.

2. The electronic device of claim 1, wherein the motion sensors sensing rotational rate are gyroscopes and the motion sensors sensing linear acceleration are accelerometers.

3. The electronic device of claim 1, wherein the motion sensors sensing rotational rate are implemented based on one or more of the following technologies:

Micro Electro Mechanical Systems, piezoelectric, hemispherical resonator, tuning fork, quartz, carbon nanotubes.

4. The electronic device of claim 1, wherein the subsystem providing display capability consists of one of the following:

a display attached to the device;

a set of modules capable of producing an image substantially adapted for display on an external display;

or

a set of modules capable of projecting an image.

5. The electronic device of claim 1, wherein the subsystem providing display capability includes a display attached to the device, and wherein the interaction with the device includes movement of a visual indicator on the display.

6. The electronic device of claim 1, wherein the subsystem providing display capability includes a display attached to the device, and wherein the interaction with the device includes interaction with a visual element displayed on the screen.

7. The electronic device of claim 6, wherein the interaction with the visual element consists of one or more of the following:

selecting the element, highlighting the element, opening the element, moving the element, reordering the element in a list of elements, or starting an application associated with the element.

8. The electronic device of claim 1, wherein the interaction with the device includes interaction with an application.

9. The electronic device of claim 1, wherein the subsystem providing display capability includes a display attached to the device, and wherein the interaction with the device consists of one or more of the following:

movement of a cursor on the display in response to movement of the device;

selection of a menu in response to movement of the device;

activation of a menu in response to movement of the device;

selection of an item included in a menu in response to movement of the device;

activation of an item included in a menu in response to movement of the device;

selection of an element shown on the display in response to movement of the device;

activation of an element shown on the display in response to movement of the device;

starting an application through an application-related element shown on the display in response to movement of the device;

switching between at least two applications in response to movement of the device;

interaction with an application through an application-related interface shown on the display in response to movement of the device;

or

exiting an application in response to movement of the device.

10. The electronic device of claim 1, wherein the subsystem providing display capability includes a display attached to the device, and wherein the interaction with the device consists of one or more of the following:

movement of a visual indicator on the screen in a substantially-linear direction in response to rotational movement of the electronic device about one of the following;

a roll, pitch or yaw axis of the electronic device;

selection of a visual element on the screen in a substantially-linear direction in response to rotational movement of the electronic device about one of the following;

a roll, pitch or yaw axis of the electronic device;

movement of a visual element on the screen in a substantially-linear direction in response to rotational movement of the electronic device about one of the following;

a roll, pitch or yaw axis of the electronic device;

movement of a visual indicator on the screen along a segmented, freeform or curved path in response to rotation of the device along at least two of the following;

a roll, pitch or yaw axis of the electronic device;

selection of a visual element on the screen along a segmented, freeform or curved path in response to rotation of the device along at least two of the following;

a roll, pitch or yaw axis of the electronic device;

or

movement of a visual element on the screen along a segmented, freeform or curved path in response to rotation of the device along at least two of the following;

a roll, pitch or yaw axis of the electronic device.

11. The electronic device of claim 1, wherein the subsystem providing display capability includes a display attached to the device, and wherein the interaction with the device includes rotation of a visual element on the display in two or three dimensions in response to rotation of the device along at least one of a roll, pitch or yaw axis.

12. The electronic device of claim 1, wherein the subsystem providing display capability includes a display attached to the device, and wherein the interaction with the device consists of one or more of the following:

panning an image on the display in response to movement of the device;

zooming an image on the display in response to movement of the device;

zooming a camera application running at least partially on the device in response to movement of the device;

changing a page on the display in response to movement of the device;

adjusting the location of at least one element on the display in response to movement of the device, wherein the at least one element is part of an array;

navigating to a menu screen in response to movement of the device;

selecting a key on a keypad displayed on the display in response to movement of the device;

selecting a key on a telephone dial pad displayed on the display in response to movement of the device;

selecting a character corresponding to the identify of a contact in response to movement of the device;

initiating a telephone call in response to movement of the device;

terminating a telephone call in response to movement of the device;

recognizing a character in response to movement of the device;

recognizing a plurality of characters in response to movement of the device;

displaying an element on the display as part of an auto-complete operation in response to movement of the device;

rotating a map on the display in response to movement of the device;

zooming a map on the display in response to movement of the device;

panning a map on the display in response to movement of the device;

tilting a map on the display in response to movement of the device;

scrolling through a set of elements, wherein at least one element included in the set of elements is displayed on the display, in response to movement of the device;

interpreting a movement of the device depending on a context of the device;

switching orientation of an image on the display between portrait and landscape modes in response to movement of the device;

activating an application in response to movement of the device, where the movement corresponds to switching between two different orientations;

providing visual, audio or vibration feedback in connection with interpretation of a movement of the device;

authenticating a user based on movement of the device;

transposing movement of the device into a tangible authentication mark;

controlling an external system based on movement of the device;

authorizing a commercial transaction based on movement of the device;

measuring caloric expenditure of a user based on movement of the device;

assessing a sports-related motion based on movement of the device;

counting a sports-related motion based on movement of the device;

assessing a game-related motion based on movement of the device;

controlling a power-saving feature of the device based on at least one of the following;

orientation of the device, movement of the device, or absence of movement of the device;

or

recognizing an instruction to be transmitted to an external system based on movement of the device.

13. The electronic device of claim 1, wherein the subsystem providing display capability includes a display attached to the device, and wherein the interaction with the device consists of one or more of the following:

triggering a function in response to a set of taps applied to the device;

displaying a symbol in response to a set of taps applied to the device to provide visual feedback relating to the tap;

triggering a function of the device in response to a set of shakes applied to the device;

or

displaying a symbol in response to a set of shakes applied to the device to provide visual feedback relating to the tap.

14. A handheld electronic device, the device comprising:

a display attached to the device;

a set of motion sensors sensing rotational rate around at least three axes and linear acceleration along at least three axes, where the motion sensors are integrated in a single module; and

a subsystem which, based on motion data derived from at least one of the motion sensors, is capable of facilitating interaction with the device.

15. The electronic device of claim 14, wherein the module is in a single package.

16. The electronic device of claim 14, wherein the module comprises a first subsystem and a second subsystem, the first subsystem comprising the motion sensors sensing rotational rate, the second subsystem comprising the motion sensors sensing linear acceleration, and wherein the first subsystem is coupled to the second subsystem via a sub-module data bus.

17. The electronic device of claim 14, wherein the module comprises a first subsystem and a second subsystem, the first subsystem comprising the motion sensors sensing rotational rate, the second subsystem comprising the motion sensors sensing linear acceleration, wherein the first subsystem is coupled to the second subsystem via a sub-module data bus, and wherein sub-module data bus is isolated from an external device bus connecting the module to at least one external device of the electronic device.

18. The electronic device of claim 14, wherein the module comprises a first subsystem, a second subsystem and a third subsystem, the first subsystem comprising at least one motion sensor sensing rotational rate, the second subsystem comprising at least two motion sensors sensing rotational rate, and the third subsystem comprising the motion sensors sensing linear acceleration, and wherein the first, second and third subsystems are coupled to a sub-module data bus.

19. A storage medium including a software program, the software program capable of running at least partially on a handheld electronic device,

wherein the device comprises:

a subsystem providing display capability; and

a set of motion sensors sensing rotational rate around at least three axes and linear acceleration along at least three axes; and

wherein the software program is capable of facilitating interaction with the device based on motion data derived from at least one of the motion sensors.

20. The storage medium of claim 19, wherein the software program is an operating system running on the handheld electronic device.

21. The storage medium of claim 19, wherein the software program is a software application running on the handheld electronic device.

22. A set of motion sensors sensing rotational rate around at least three axes and linear acceleration along at least three axes, wherein:

the motion sensors are capable of being integrated in a handheld electronic device; and

wherein the device comprises;

a subsystem providing display capability; and

a subsystem which, based on motion data derived from at least one of the motion sensors, is capable of facilitating interaction with the device.

23. The electronic device of claim 22, wherein the motion sensors sensing rotational rate are gyroscopes and the motion sensors sensing linear acceleration are accelerometers.

24. The electronic device of claim 22, wherein the motion sensors sensing rotational rate are implemented based on one or more of the following technologies:

Micro Electro Mechanical Systems, Piezo-Electric, quartz, or carbon nanotubes.

25. A handheld electronic device, the device comprising:

a subsystem providing display capability;

a motion function trigger;

a set of motion sensors sensing rotational rate around at least three axes and linear acceleration along at least three axes; and

a subsystem which, based on motion data derived from at least one of the motion sensors, is capable of facilitating interaction with the device.

26. The electronic device of claim 25, wherein the subsystem providing display capability consists of one of the following:

a display attached to the device;

a set of modules capable of producing an image substantially adapted for display on an external display;

or

a set of modules capable of projecting an image.

27. The electronic device of claim 25, wherein the subsystem providing display capability includes a display attached to the device, and wherein the interaction with the device includes movement of a visual indicator on the display.

28. The electronic device of claim 25, wherein the subsystem providing display capability includes a display attached to the device, and wherein the interaction with the device includes interaction with a visual element displayed on the screen.

29. The electronic device of claim 28, wherein the interaction with the visual element consists of one or more of the following in response to a signal received from the motion function trigger:

selecting the element, highlighting the element, opening the element, moving the element, reordering the element in a list of elements, or starting an application associated with the element.

30. The electronic device of claim 25, wherein the interaction with the device includes interaction with an application in response to a signal received from the motion function trigger.

31. The electronic device of claim 25, wherein the subsystem providing display capability includes a display attached to the device, and wherein the interaction with the device consists of one or more of the following in response to a signal received from the motion function trigger:

movement of a cursor on the display in response to movement of the device;

selection of a menu in response to movement of the device;

activation of a menu in response to movement of the device;

selection of an item included in a menu in response to movement of the device;

activation of an item included in a menu in response to movement of the device;

selection of an element shown on the display in response to movement of the device;

activation of an element shown on the display in response to movement of the device;

starting an application through an application-related element shown on the display in response to movement of the device;

switching between at least two applications in response to movement of the device;

interaction with an application through an application-related interface shown on the display in response to movement of the device;

or

exiting an application in response to movement of the device.

32. The electronic device of claim 25, wherein the subsystem providing display capability includes a display attached to the device, and wherein the interaction with the device consists of one or more of the following in response to a signal received from the motion function trigger:

movement of a visual indicator on the screen in a substantially-linear direction in response to rotational movement of the electronic device about one of the following;

a roll, pitch or yaw axis of the electronic device;

selection of a visual element on the screen in a substantially-linear direction in response to rotational movement of the electronic device about one of the following;

a roll, pitch or yaw axis of the electronic device;

movement of a visual element on the screen in a substantially-linear direction in response to rotational movement of the electronic device about one of the following;

a roll, pitch or yaw axis of the electronic device;

movement of a visual indicator on the screen along a segmented, freeform or curved path in response to rotation of the device along at least two of the following;

a roll, pitch or yaw axis of the electronic device;

selection of a visual element on the screen along a segmented, freeform or curved path in response to rotation of the device along at least two of the following;

a roll, pitch or yaw axis of the electronic device;

or

movement of a visual element on the screen along a segmented, freeform or curved path in response to rotation of the device along at least two of the following;

a roll, pitch or yaw axis of the electronic device.

33. The electronic device of claim 25, wherein the subsystem providing display capability includes a display attached to the device, and wherein the interaction with the device includes rotation of a visual element on the display in two or three dimensions in response to rotation of the device along at least one of a roll, pitch or yaw axis and in response to a signal received from the motion function trigger.

34. The electronic device of claim 25, wherein the subsystem providing display capability includes a display attached to the device, and wherein the interaction with the device consists of one or more of the following in response to a signal received from the motion function trigger:

panning an image on the display in response to movement of the device;

zooming an image on the display in response to movement of the device;

zooming a camera application running at least partially on the device in response to movement of the device;

changing a page on the display in response to movement of the device;

adjusting the location of at least one element on the display in response to movement of the device, wherein the at least one element is part of an array;

navigating to a menu screen in response to movement of the device;

selecting a key on a keypad displayed on the display in response to movement of the device;

selecting a key on a telephone dial pad displayed on the display in response to movement of the device;

selecting a character corresponding to the identify of a contact in response to movement of the device;

initiating a telephone call in response to movement of the device;

terminating a telephone call in response to movement of the device;

recognizing a character in response to movement of the device;

recognizing a plurality of characters in response to movement of the device;

displaying an element on the display as part of an auto-complete operation in response to movement of the device;

rotating a map on the display in response to movement of the device;

zooming a map on the display in response to movement of the device;

panning a map on the display in response to movement of the device;

tilting a map on the display in response to movement of the device;

scrolling through a set of elements, wherein at least one element included in the set of elements is displayed on the display, in response to movement of the device;

interpreting a movement of the device depending on a context of the device;

switching orientation of an image on the display between portrait and landscape modes in response to movement of the device;

activating an application in response to movement of the device, where the movement corresponds to switching between two different orientations;

providing visual, audio or vibration feedback in connection with interpretation of a movement of the device;

authenticating a user based on movement of the device;

transposing movement of the device into a tangible authentication mark;

controlling an external system based on movement of the device;

authorizing a commercial transaction based on movement of the device;

measuring caloric expenditure of a user based on movement of the device;

assessing a sports-related motion based on movement of the device;

counting a sports-related motion based on movement of the device;

assessing a game-related motion based on movement of the device;

controlling a power-saving feature of the device based on at least one of the following;

orientation of the device, movement of the device, or absence of movement of the device;

or

recognizing an instruction to be transmitted to an external system based on movement of the device.

35. The electronic device of claim 25, wherein the subsystem providing display capability includes a display attached to the device, and wherein the interaction with the device consists of one or more of the following in response to a signal received from the motion function trigger:

triggering a function in response to a set of taps applied to the device;

displaying a symbol in response to a set of taps applied to the device to provide visual feedback relating to the tap;

triggering a function of the device in response to a set of shakes applied to the device; and

displaying a symbol in response to a set of shakes applied to the device to provide visual feedback relating to the tap.

36. A handheld electronic device, the device comprising:

a display attached to the device;

a motion function trigger;

a set of motion sensors sensing rotational rate around at least three axes and linear acceleration along at least three axes, where the motion sensors are integrated in a single module; and

a subsystem which, based on motion data derived from at least one of the motion sensors and activation of the motion function trigger, is capable of facilitating interaction with the device.

37. The electronic device of claim 36, wherein the module is integrated in a single package.

38. The electronic device of claim 36, wherein the module comprises a first subsystem and a second subsystem, the first subsystem comprising the motion sensors sensing rotational rate, the second subsystem comprising the motion sensors sensing linear acceleration, and wherein the first subsystem is coupled to the second subsystem via a sub-module data bus.

39. The electronic device of claim 36, wherein the module comprises a first subsystem, a second subsystem and a third subsystem, the first subsystem comprising at least one motion sensor sensing rotational rate, the second subsystem comprising at least two motion sensors sensing rotational rate, and the third subsystem comprising the motion sensors sensing linear acceleration, and wherein the first, second and third subsystems are coupled to a sub-module data bus.