SENSOR FUSION TO COMBINE SENSOR INPUT DATA FROM MULTIPLE DEVICES INTO ONE INPUT STREAM

• US 20100271302A1
• Filed: 12/31/2008
• Published: 10/28/2010
• Est. Priority Date: 12/31/2008
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: PERING, TREVOR, Lvons, Kenton M., Want, Roy, Rosario, Barbara
• Granted Patent: US 9,662,569 B2
• Time in Patent Office: Days
• US Class Current: 345/161
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/212 using sensors worn by the player, e.g. for measuring heart beat or leg activity
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/235 using a wireless connection, e.g. infrared or piconet
• CPC Class Codes: A63F 13/335 using Internet
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/92 Video game devices specially adapted to be hand-held while playing
• CPC Class Codes: A63F 2300/1012 involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: G06F 2203/0382 Plural input, i.e. interface arrangements in which a plurality of input device of the same type are in communication with a PC
• CPC Class Codes: G06F 2203/0384 Wireless input, i.e. hardware and software details of wireless interface arrangements for pointing devices
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0383 Signal control means within the pointing device

Abstract

Software gaming applications may be written to accept an input from one or more standard controllers such a joystick with input buttons. However, multiple wearable sensors or hand held MID devices may enhance the gaming experience. These sensors may include devices such as accelerometers to detect movement of the extremity they are attached to and wirelessly communicate this information to a receiver. Embodiments are directed to using sensor fusion to combine sensor data from multiple wireless input devices, such as wearable sensors and MID devices together to form one logical input stream that is presented to an application and which the application sees are a standard controller.

Claims

1. An apparatus, comprising:

a plurality of inputs, each of the inputs indicative a user'"'"'s physical movements; and

a unified data steam output comprising a fusion of all of the plurality of inputs combined into a format usable to a software component expecting the unified data stream.

2. The apparatus as recited in claim 1 wherein the software component comprises a gaming application.

3. The apparatus as recited in claim 2, further comprising:

a plurality of wearable sensors designed to be worn on various parts of a user'"'"'s body, wherein the plurality of wearable sensors provide the plurality of inputs.

4. The apparatus as recited in claim 3 wherein the wearable sensors are wireless.

5. The apparatus as recited in claim 1 further comprising:

a wireless hand held device equipped with motion sensors, wherein the hand held device provides at least one of the plurality of inputs.

6. The apparatus from claim 1 where the unified data stream is represented to the software component as a physical controller including one of a joystick, keyboard, mouse, dance pad, or other peripheral device.

7. The apparatus as recited in claim 1 wherein at least one of the inputs comprises a camera.

8. A method, comprising:

receiving multiple inputs from a plurality of sensors, the sensors indicative of a user'"'"'s movements;

combining the multiple inputs into a single data stream, the single data stream in a format compatible with a software application written to accept a single data stream.

9. The method as recited in claim 8, wherein the software application is a gaming application and the single data stream format comprises a joystick controller.

10. The method as recited in claim 8 wherein the plurality of sensors comprise wearable sensors designed to be worn on various parts of a user'"'"'s body.

11. The method as recited in claim 8 wherein a wireless hand-held device comprises at least one of the plurality of sensors.

12. The method as recited in claim 8 where the unified data stream is represented to the software application as a physical controller including one of a joystick, keyboard, mouse, dance pad, or other peripheral device.

13. The method as recited in claim 8 wherein some of the wearable sensors are wireless.

14. The method as recited in claim 11 wherein the hand-held device includes accelerometers, a camera, or other physical sensing device.

15. A system, comprising:

a gaming application to run on a computer, the gaming application expecting as an input a data stream from a joystick controller;

a plurality of inputs from a plurality of sensors each if the plurality of inputs indicative of a user'"'"'s movements;

a combiner to combine each of the plurality of inputs into a single input data stream in a format compatible with the data stream from a joystick controller.

16. The system as recited in claim 15 wherein the plurality of sensors comprises wearable sensors designed to be worn on various parts of a user'"'"'s body.

17. The system as recited in claim 15 wherein a wireless hand-held device comprises at least one of the plurality of sensors.

18. The system as recited in claim 17 wherein the hand-held device comprises a mobile Internet device (MID).

19. The system as recited in claim 16 wherein the wearable sensors are wireless.

20. The system as recited in claim 18 wherein the mobile internet device (MID) includes accelerometers.