Systems and methods for supporting generalized motion recognition

• US 8,195,585 B1
• Filed: 04/29/2010
• Issued: 06/05/2012
• Est. Priority Date: 07/14/2006
• Status: Active Grant

• Current Assignee: AiLive, Inc.
• Original Assignee: AiLive, Inc.
• Inventors: Tu, Xiaoyuan, KAWANO, Yoichiro, Musick, JR., Charles, Powers, III, William Robert, Reynolds, Stuart, Wilkinson, Dana, Wright, Ian, Yen, Wei
• Primary Examiner(s): Chaki, Kakali
• Assistant Examiner(s): Rifkin, Ben
• Application Number:
  US12/770,609
• Time in Patent Office: 768 Days
• Field of Search: 706/14
• US Class Current: 706/14
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/2145 the surface being also a display device, e.g. touch screens
• CPC Class Codes: A63F 13/215 comprising means for detecting acoustic signals, e.g. using a microphone
• CPC Class Codes: A63F 13/235 using a wireless connection, e.g. infrared or piconet
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/67 adaptively or by learning from player actions, e.g. skill level adjustment or by storing successful combat sequences for re-use
• CPC Class Codes: A63F 13/92 Video game devices specially adapted to be hand-held while playing
• CPC Class Codes: A63F 2300/1043 being characterized by constructional details
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/6027 using adaptive systems learning from user actions, e.g. for skill level adjustment
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: G06F 2203/0384 Wireless input, i.e. hardware and software details of wireless interface arrangements for pointing devices
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry

Abstract

Techniques for supporting generalized motion recognition are disclosed. A set of motion recognizers created from training sets of labeled processed motion signals is provided, each of the component outputs is transformed. When a motion signal including two or more component outputs is received, the component outputs are transformed into device-independent motion signals, where each of the component outputs describes a different component of a motion made by a user. The motion recognizers are applied to the motion signal to build generalized motion recognizers responsive to the motion sensitive device that has generated the motion signal.

Claims

1. A method for processing motion-related input streams to support a generalized motion recognition, the method comprising:

accessing a set of motion recognizers created from training sets of labeled processed motion signals;

receiving a motion signal including two or more component outputs, wherein each of the component outputs describes a different component of a motion made by a user, and at least one of the component outputs are generated from a set of sensors;

transforming each of the component outputs of the motion signal into device-independent motion signals by an operation including virtually changing locations of the sensors to corresponding locations of sensors used to generate the training sets of labeled processed motion signals; and

computing a motion recognition signal from the device-independent motion signals, wherein the motion recognition signal is applied to an object being controlled by the user.

2. The method as recited in claim 1, wherein the two or more component outputs are from inertial sensors in a motion sensitive device, and one or more of (i) buttons, dpads, sticks or triggers on the hand-held device, (ii) one or more touch screen sensors on the motion sensitive device, (iii) one or more video cameras tracking body part location and orientation of the user, and (iv) one or more microphones.

3. The method as recited in claim 2, further comprising converting the motion signal to parameters in a domain designed to work with a wide range of devices and inputs.

4. The method as recited in claim 2, further comprising:

filtering adaptively the motion signal with an adaptor specific to each component of the motion signal, so that some components are rarely filtered or modified, and others are frequently filtered or modified.

5. The method as recited in claim 1, wherein a processing unit is configured to include a recognition runtime library associating an elasticity parameter with each of the two or more component outputs, and a subsequent time warp-related score is responsive to the elasticity parameter.

6. The method as in claim 1, wherein the motion recognizers and the motion recognition signal are responsive to both static poses and dynamic motions of the user.

7. The method as in claim 1, further comprising applying the motion recognizers to the motion signal to build generalized motion recognizers responsive to a second motion sensitive device.

8. The method as in claim 7, wherein the motion is made by the user without reference to a predefined set of allowed motions and/or without restriction on how to execute the motion to create the generalized motion recognizers responsive to the second motion sensitive device.

9. A system for processing motion-related input streams to support generalized motion recognition, the method comprising:

at least two types of sensors generating sensing signals about a motion made by a user; and

a processing unit loaded with a set of motion recognizers that are created in advance, the processing unit receiving a motion signal including the sensing signals and configured to transform each of the sensing signals into device-independent motion signals by an operation including virtually changing locations of one of the at least two types of sensors to corresponding locations of sensors used to generate a training set of motion signals for at least or all of the motion recognizers, and determine a motion recognition signal from the motion signal using the set of motion recognizers, wherein the motion recognition signal is applied to an object being controlled by the user.

10. The system as recited in claim 9, wherein the at least two types of sensors include inertial sensors in a motion sensitive device, and one or more of (i) buttons, dpads, sticks or triggers on the motion sensitive device, (ii) one or more touch screen sensors on the hand-held device, (iii) one or more video cameras tracking body part location and orientation of the user, and (iv) one or more microphones.

11. The system as recited in claim 10, wherein the processing unit is configured to convert the motion signal to parameters in a domain designed to work with a wide range of devices and inputs.

12. The system as recited in claim 10, wherein the processing unit includes an adaptor configured to filter the motion signal adaptively, the adaptor being specific to each component of the motion signal, so that some components are rarely filtered or modified, and others are frequently filtered or modified.

13. The system as recited in claim 10, wherein the processing unit is configured to include a recognition runtime library associating an elasticity parameter with each of the two or more outputs, and a subsequent time warp-related score is responsive to the elasticity parameter.

14. The system as in claim 13, wherein the motion recognizers and the motion recognition signal are responsive to both static poses and dynamic motions of the user.

15. The system as in claim 10, wherein the motion recognizers are, used to build generalized motion recognizers responsive to the motion sensitive device.

16. The method as in claim 15, wherein the motion was made by the user without reference to a predefined set of allowed motions and/or without restriction on how to execute the motion to create generalized motion recognizers responsive to the motion sensitive device.

17. A method for processing motion-related input streams to support generalized motion recognition, the method comprising:

loading in a processing unit a set of the motion recognizers that are created in advance by a training set of motion signals from at least one first motion sensitive device embedded with a first set of sensors;

receiving in the processing unit a motion signal with at least two component outputs from at least one second motion sensitive device embedded with a second set of sensors some of which are different from some of the first set of sensors;

converting the component outputs into device-independent motion signals by an operation including virtually changing locations of the second set of sensors to locations of the first set of sensors, and

applying the motion recognizers to the device-independent motion signals to provide a generalized motion recognition capability.

18. The method as recited in claim 17, further comprising determining a final motion recognition signal from the motion signal using the generalized motion recognition capability.

19. The method as recited in claim 18, further comprising transforming each of the component outputs of the motion signal into one or more of a pseudo-linear acceleration, a pseudo-angular velocity, or a pseudo-button.

20. The method as recited in claim 19, wherein the first motion sensitive device is different from the second motion sensitive device in terms of its ability to fully describe various human motions.

21. The method as recited in claim 17, wherein the processing unit includes an interface to facilitate registration of motion sensitive devices to inform the processing unit of sufficient statistics of new motion sensitive devices to be used with the generalized motion recognition capability.

22. A system for processing motion-related input streams to support generalized motion recognition, the system comprising:

a processing unit loaded with a set of the motion recognizers that are created in advance by a training set of motion signals from at least one first motion sensitive device embedded with a set of sensors;

a second motion sensitive device producing inertial sensor signals and other sensor signals describing various motions made by a user;

the processing unit receiving a motion signal from the second motion sensitive device, the motion signal including the inertial sensor and other sensor signals, the processing unit configured to convert the inertial sensor and other sensor signals into device-independent motion signals by an operation including virtually changing locations of sensors in the second motion sensitive device to respective locations of the sensors in the at least one first motion sensitive device, and apply the motion recognizers to the motion signal to provide a generalized motion recognition service responsive to the second motion sensitive device.

23. The system as recited in claim 22, wherein the processing unit is configured to determine a final motion recognition signal from the motion signal using the generalized motion recognition service.

24. The system as recited in claim 23, wherein the processing unit is further configured to transform each of the inertial sensor and other sensor signals into one or more of a pseudo-linear acceleration, a pseudo-angular velocity, or a pseudo-button.

25. The system as recited in claim 22, wherein the first motion sensitive device is different from the second motion sensitive device in terms of sensibility.

26. The system as recited in claim 22, wherein the processing unit includes a interface to facilitate registration of motion sensitive devices to inform the processing unit of sufficient statistics of new motion sensitive devices to be used with the generalized motion recognition service.