Sensor Fusion Algorithm

• US 20130232280A1
• Filed: 05/14/2012
• Published: 09/05/2013
• Est. Priority Date: 03/02/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Perek, David R., Schwager, Michael A., Drasnin, Sharon, Seilstad, Mark J.
• Granted Patent: US 8,548,608 B2
• Time in Patent Office: Days
• US Class Current: 710/14
• CPC Class Codes: E05D 11/1064 with a coil spring perpendicular to the pivot axis
• CPC Class Codes: E05F 5/08 with springs
• CPC Class Codes: E05Y 2201/46 Magnets
• CPC Class Codes: F16M 11/38 by folding , e.g. pivoting or scissors tong mechanisms
• CPC Class Codes: G05B 11/01 electric
• CPC Class Codes: G06F 1/1616 with folding flat displays, e.g. laptop computers or notebooks having a clamshell configuration, with body parts pivoting to an open position around an axis parallel to the plane they define in closed position
• CPC Class Codes: G06F 1/1618 the display being foldable up to the back of the other housing with a single degree of freedom, e.g. by 360° rotation over the axis defined by the rear edge of the base enclosure
• CPC Class Codes: G06F 1/1637 Details related to the display arrangement, including those related to the mounting of the display in the housing
• CPC Class Codes: G06F 1/1654 the display being detachable, e.g. for remote use
• CPC Class Codes: G06F 1/1656 Details related to functional adaptations of the enclosure, e.g. to provide protection against EMI, shock, water, or to host detachable peripherals like a mouse or removable expansions units like PCMCIA cards, or to provide access to internal components for maintenance or to removable storage supports like CDs or DVDs, or to mechanically mount accessories mounting of accessories to a computer display G06F1/1607; display hoods G06F1/1603; cooling arrangements for portable computers G06F1/203
• CPC Class Codes: G06F 1/166 related to integrated arrangements for adjusting the position of the main body with respect to the supporting surface, e.g. legs for adjusting the tilt angle
• CPC Class Codes: G06F 1/1662 Details related to the integrated keyboard
• CPC Class Codes: G06F 1/1669 Detachable keyboards
• CPC Class Codes: G06F 1/1681 Details related solely to hinges hinge details related to the transmission of signals or power are classified in G06F1/1683
• CPC Class Codes: G06F 1/1683 for the transmission of signal or power between the different housings, e.g. details of wired or wireless communication, passage of cabling
• CPC Class Codes: G06F 1/1684 Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 11/3089 Monitoring arrangements determined by the means or processing involved in sensing the monitored data, e.g. interfaces, connectors, sensors, probes, agents software debugging using additional hardware using a specific debug interface G06F11/3656; performance evaluation by tracing or monitoring G06F11/3466
• CPC Class Codes: G06F 13/102 where the programme performs an interfacing function, e.g. device driver G06F13/105 takes precedence; scheduling within device drivers G06F9/52; contention policies within device drivers G06F9/4881
• CPC Class Codes: G06F 3/002 Specific input/output arrangements not covered by G06F3/01 - G06F3/16 other optical apparatus G02B27/00
• CPC Class Codes: G06F 3/01 : Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: G06F 3/02 : Input arrangements using manually operated switches, e.g. using keyboards or dials
• CPC Class Codes: G06F 3/0202 : Constructional details or processes of manufacture of the input device
• CPC Class Codes: G06F 3/0219 : Special purpose keyboards
• CPC Class Codes: G06F 3/023 : Arrangements for converting discrete items of information into a coded form, e.g. arrangements for interpreting keyboard generated codes as alphanumeric codes, operand codes or instruction codes
• CPC Class Codes: G06F 3/0414 : using force sensing means to determine a position
• CPC Class Codes: G06F 3/0416 : Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/0487 : using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
• CPC Class Codes: G06F 3/0488 : using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04886 : by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06F 9/541 : via adapters, e.g. between incompatible applications
• CPC Class Codes: H01H 11/00 : Apparatus or processes specially adapted for the manufacture of electric switches processes specially adapted for manufacture of rectilinearly movable switches having a plurality of operating members associated with different sets of contacts, e.g. keyboards, H01H13/88
• CPC Class Codes: H01H 13/14 : Operating parts, e.g. push-button
• CPC Class Codes: H01H 13/702 : with contacts carried by or formed from layers in a multilayer structure, e.g. membrane switches
• CPC Class Codes: H01H 13/703 : characterised by spacers between contact carrying layers
• CPC Class Codes: H01H 13/704 : characterised by the layers, e.g. by their material or structure H01H13/703 takes precedence
• CPC Class Codes: H01H 13/78 : characterised by the contacts or the contact sites
• CPC Class Codes: H01H 13/785 : characterised by the material of the contacts, e.g. conductive polymers
• CPC Class Codes: H01H 13/79 : characterised by the form of the contacts, e.g. interspersed fingers or helical networks
• CPC Class Codes: H01H 13/807 : characterised by the spatial arrangement of the contact sites, e.g. superimposed sites
• CPC Class Codes: H01H 13/82 : characterised by contact space venting means
• CPC Class Codes: H01H 2201/036 : Variable resistance
• CPC Class Codes: H01H 2203/02 : Interspersed fingers
• CPC Class Codes: H01H 2203/036 : to solve particular problems
• CPC Class Codes: H01H 2203/058 : Contact area function of position on layered keyboard
• CPC Class Codes: H01H 2205/006 : mounted on spacer
• CPC Class Codes: H01H 2211/004 : Adhesive
• CPC Class Codes: H01H 2211/006 : Individual areas
• CPC Class Codes: H01H 2213/016 : in adhesive layer
• CPC Class Codes: H01H 2217/004 : Larger or different actuating area
• CPC Class Codes: H01H 2217/006 : Different feeling for different switch sites
• CPC Class Codes: H01H 2217/01 : Off centre actuation
• CPC Class Codes: H01H 2227/032 : Operating force
• CPC Class Codes: H01H 9/26 : for interlocking two or more switches H01H13/568 takes precedence; by a detachable member H01H9/28 ; for electromagnetic relays H01H50/323
• CPC Class Codes: H04M 1/0216 : Foldable in one direction, i.e. using a one degree of freedom hinge
• CPC Class Codes: H04M 1/0245 : using open/close detection
• CPC Class Codes: H04M 1/0254 : comprising one or a plurality of mechanically detachable modules
• CPC Class Codes: H04M 1/72409 : by interfacing with external accessories hands-free H04M1/60
• CPC Class Codes: H05K 5/0226 : Hinges
• CPC Class Codes: H05K 5/0234 : Feet; Stands; Pedestals, e.g. wheels for moving casing on floor
• CPC Class Codes: Y02D 10/00 : Energy efficient computing, e.g. low power processors, power management or thermal management
• CPC Class Codes: Y10T 29/49826 : Assembling or joining

Abstract

Sensor fusion algorithm techniques are described. In one or more embodiments, behaviors of a host device and accessory devices are controlled based upon an orientation of the host device and accessory devices, relative to one another. A combined spatial position and/or orientation for the host device may be obtained based on raw measurements that are obtained from at least two different types of sensors. In addition, a spatial position and/or orientation for an accessory device is ascertained using one or more sensors of the accessory device. An orientation (or position) of the accessory device relative to the host computing device may then be computed based on the combined spatial position/orientation for the host computing device and the ascertained spatial position/orientation for the accessory device. The relative orientation that is computed may then be used in various ways to control behaviors of the host computing device and/or accessory device.

Claims

1. One or more computer readable storage media comprising instructions that, when executed by a host computing device, cause the host computing device to perform operations including:

determining a combined spatial position for the host computing device based upon multiple raw spatial positions for the host computing device calculated using at least using at least two different types of sensors of the host computing device;

ascertaining a spatial position for an accessory device connected to the host computing device using one or more sensors of the accessory device; and

computing an orientation of the accessory device relative to the host computing device based on the combined spatial position for the host computing device and the ascertained spatial position for the accessory device.

2. One or more computer readable storage media as described in claim 1, wherein the instructions further cause the host computing device to perform operations including exposing the computed orientation for use by one or more applications of the host computing device via an application programming interface (API).

3. One or more computer readable storage media as described in claim 1, wherein determining the combined spatial position for the host computing device comprises using a gyroscope to compute one of the multiple raw spatial positions.

4. One or more computer readable storage media as described in claim 1, wherein determining the combined spatial position for the host computing device comprises using an accelerometer and a magnetometer in combination to compute one of the multiple raw spatial positions.

5. One or more computer readable storage media as described in claim 1, wherein the spatial position for the accessory device is ascertained via an accelerometer of the accessory device.

6. One or more computer readable storage media as described in claim 1, wherein the different types of sensors include a gyroscope, an accelerometer, and a magnetometer of the host computing device.

7. One or more computer readable storage media as described in claim 1, wherein determining the combined spatial position comprises interpolating between the multiple raw spatial positions to reduce inaccuracy of the combined spatial position.

8. One or more computer readable storage media as described in claim 1, wherein the instructions are executable via one or more microcontrollers of the host computing device to implement a sensor fusion module at least partially in hardware.

9. One or more computer readable storage media as described in claim 1, wherein the instructions further cause the host computing device to perform operations including controlling one or more behaviors of the host computing device based on the orientation that is computed.

10. One or more computer readable storage media as described in claim 9, wherein controlling the one or more behaviors of the host computing device comprises selectively adjusting sensitivity of one or more sensors based on the orientation that is computed.

11. One or more computer readable storage media as described in claim 9, wherein controlling the one or more behaviors of the host computing device comprises selectively activating or deactivating one or more applications based on the orientation that is computed.

12. One or more computer readable storage media as described in claim 1, wherein the instructions further cause the host computing device to perform operations including controlling one or more behaviors of the accessory device based on the orientation that is computed.

13. One or more computer readable storage media as described in claim 12, wherein controlling the one or more behaviors of the accessory device comprises modifying functionality that is activated for the accessory device based on the orientation that is computed.

14. One or more computer readable storage media as described in claim 12, wherein controlling the one or more behaviors of the accessory device comprises selectively enabling or disabling one or more sensors of the accessory device based on the orientation that is computed.

15. A host computing device comprising:

one or more microcontroller devices;

an interface to physically and communicatively connect an accessory device to the host computing device;

a sensor fusion module implemented at least partially via the microcontroller devices and configured to perform operations including;

computing an orientation of the accessory device relative to the host computing device when the accessory device is connected to the host computing device via the interface based upon;

a combined spatial position for the host computing device obtained by interpolation of at least two independent raw spatial positions for the host computing device calculated using multiple sensors of the host computing device; and

a spatial position for the accessory device ascertained using one or more sensors of the accessory device; and

controlling one or more behaviors of the host computing device and the accessory device based upon the orientation that is computed.

16. The host computing device as described in claim 15, wherein the interface comprises magnetic coupling devices to magnetically couple the host computing device to the accessory device.

17. The host computing device as described in claim 16, wherein the interface further comprises mechanical coupling portions to form a mechanical physical connection between the accessory device and the host computing device.

18. The host computing device as described in claim 16, wherein the interface further comprises communication contacts to form a communicative coupling through which information regarding the spatial position of the accessory device is obtained from the accessory device.

19. A method implemented by a microcontroller of a host computing device comprising:

ascertaining a spatial position of an accessory device physically and communicatively coupled to the host computing device via an interface that enables removable attachment of the accessory device to the host computing device using position information from one or more sensors of the accessory device obtained via the interface;

computing an orientation of the accessory device relative to the host computing device to which the accessory device is attached based on at least two independent computations of a spatial position for the host computing device that are calculated using an array of sensors for the host computing device and the ascertained spatial position for the accessory device ascertained using the position information from the one or more sensors of the accessory device;

exposing an application programming interface (API) operable by one or more applications to obtain the computed orientation; and

supplying the computed orientation to an application in response to receiving a request from the application via the API.

20. The method as described in claim 19, wherein the application programming interface (API) is further operable by the one or more applications to register with a notification system to receive notification messages regarding updates to the computed orientation that are sent by the notification system to registered applications.