Method and system for tuning motion recognizers by a user using a set of motion signals

• US 7,899,772 B1
• Filed: 03/02/2010
• Issued: 03/01/2011
• Est. Priority Date: 07/14/2006
• Status: Active Grant

• Current Assignee: AiLive, Inc.
• Original Assignee: AiLive, Inc.
• Inventors: Bererton, Curt, Dobson, Daniel, Funge, John, Musick, JR., Charles, Reynolds, Stuart, Tu, Xiaoyuan, Wright, Ian, Yen, Wei
• Primary Examiner(s): Sparks, Donald
• Assistant Examiner(s): Rifkin, Ben M
• Application Number:
  US12/715,397
• Time in Patent Office: 364 Days
• Field of Search: 706/46
• US Class Current: 706/46
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/67 adaptively or by learning from player actions, e.g. skill level adjustment or by storing successful combat sequences for re-use
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/6009 for importing or creating game content, e.g. authoring tools during game development, adapting content to different platforms, use of a scripting language to create content
• CPC Class Codes: A63F 2300/6027 using adaptive systems learning from user actions, e.g. for skill level adjustment
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

Techniques for using motion recognizers are described. The motion recognizers are created or generated in advance by trained users. The motion recognizers are then loaded into a processing unit that receives motion signals from one or more motion sensitive devices being manipulated by one or more end users to control one or more objects in a virtual environment. Depending on implementation, the virtual environment may represent a remote scene or a video game, where objects in the virtual environment can be controlled by the users to perform desired actions or moves.

Claims

1. A system for controlling virtual objects in a video game display in a manner responsive to human motions, the system comprising:

at least one hand-held motion sensing device generating motion signals in response to the human motions;

a processing unit, loaded with a set of motion recognizers created in advance by at least one trained user, receiving the motion signals from the hand-held motion sensing device, and configured to;

compute motion recognition signals from some of the motion signals in reference to the motion recognizers;

form a training set including some or all of the motion signals;

tune one or more of the motion recognizers in the set of motion recognizers with the training set to modify a motion recognition behavior of the one or more tuned motion recognizers; and

use the motion recognition signals to control one or more virtual objects in the video game display.

2. The system as recited in claim 1, further comprising:

the video game display showing a virtual interactive environment, wherein movements of at least one of the virtual objects in the video game display is responsive to one or more of the motion recognition signals.

3. The system as recited in claim 2, wherein the processing unit receives the motion recognizers in a portable storage medium, by downloading the motion recognizers via the Internet;

or extracting the motion recognizers embedded in a video game.

4. The system as recited in claim 3, wherein the processing unit receives another set of the motion recognizers when the another set of the motion recognizers becomes available and relevant to the virtual environment.

5. The system as recited in claim 1, further comprising;

tuning one or more of the motion recognizers repeatedly once a predefined motion recognition level is below a certain value.

6. The system as recited in claim 1, wherein the processing unit includes a module configured to:

receive the motion signals wirelessly from the hand-held motion-sensing device;

preprocess the motion signals by a filtering means; and

segment the motion signals adaptively according to corresponding magnitudes of the underlying motion signals.

7. The system as recited in claim 6, wherein the module is configured further to:

Calculate classification rates for the motion signals; and

tune one or more of the motion recognizers by determining which of the motion signals to add as prototypes.

8. A method for controlling virtual objects in a video game display in a manner responsive to human motions, the method comprising:

loading a set of the motion recognizers that are created in advance by at least one trained user;

receiving motion signals from a hand-held motion-sensitive device, where the hand-held motion-sensitive device is being manipulated in response to a virtual environment being displayed on a display screen; and

forming a training set including some or all of the motion signals;

tuning one or more of the motion recognizers in the set of motion recognizers with the training set to modify a motion recognition behavior of the one or more tuned motion recognizers;

computing motion recognition signals from the motion signals using the set of motion recognizers so as to control one or more of the virtual objects in the virtual environment.

9. The method as recited in claim 8, further comprising:

Showing the virtual interactive environment in the video game display, movements of at least one of the virtual objects in the video game display is responsive to one or more of the motion recognition signals.

10. The method as recited in claim 9, wherein the motion recognizers are received in a portable storage medium, downloaded via the Internet, or embedded in a video game.

11. The method as recited in claim 8, further comprising receiving another set of the motion recognizers when the another set of the motion recognizers becomes available and relevant to the virtual environment.

12. The method as recited in claim 11, further comprising:

tuning one or more of the motion recognizers repeatedly once a predefined motion recognition level is below a certain value.

13. The method as recited in claim 8, wherein said receiving motion signals from the hand-held motion-sensitive device comprises receiving the motion signals wirelessly from the hand-held motion-sensitive device.

14. The method as recited in claim 13, wherein said computing motion recognition signals from the motion signals comprises:

preprocessing the motion signals by a filtering means; and

segmenting the motion signals adaptively according to corresponding magnitudes of the underlying motion signals.

15. The method as recited in claim 14, further comprising:

Calculating classification rates for the motion signals; and

tuning one or more of the motion recognizers by determining which of the motion signals to add as prototypes.

16. The method as recited in claim 15, further comprising:

calculating a classification distance of each of the motion signals to prototypes in the motion recognizers;

labeling the each of the motion signals as undetermined if the classification distance matches none of the prototypes;

labeling the each of the motion signals as a labeled motion associated with one of the prototypes if the classification distance matches only one prototype;

or

labeling the each of the motion signals as a labeled motion associated with one of some of the prototypes if the classification distance matches some of the prototypes, where the one of some of the prototypes is determined by a smallest classification distance; and

adding one or more of the motion signals as prototypes to the one or more of the motion recognizers.

17. The method as recited in claim 16, wherein capacity for the new motion recognizers and substantially all other information needed to perform classification are created automatically and directly from the training set.

18. A method for controlling virtual objects in a video game display in a manner responsive to human motions, the method comprising:

loading a set of the motion recognizers that are created in advance by at least one trained user;

receiving motion signals from a hand-held motion-sensitive device, where an end user is manipulating the hand-held motion-sensitive device in response to a virtual environment being displayed on a display screen; and

forming a training set including some or all of the motion signals;

tuning one or more of the motion recognizers in the set of motion recognizers with the training set to modify a motion recognition behavior of the one or more tuned motion recognizers;

computing motion recognition signals for some or all of the motion signals using the set of motion recognizers to determine what type of translations and/or rotations shall be applied to one or more objects in the virtual environment.

19. The method as recited in claim 18, further comprising:

showing the virtual interactive environment on a video game display, movements of at least one of the virtual objects in the video game display is responsive to one or more of the motion recognition signals.

20. The method as recited in claim 19, wherein the motion recognizers are received in a portable storage medium, downloaded via the Internet, or embedded in a video game.

21. The method as recited in claim 18, further comprising receiving another set of the motion recognizers when the another set of the motion recognizers becomes available and relevant to the virtual environment.

22. The method as recited in claim 21, further comprising

tuning one or more of the motion recognizers repeatedly once a predefined motion recognition level is below a certain value.

23. The method as recited in claim 18, wherein said receiving motion signals from the hand-held motion-sensitive device comprises receiving the motion signals wirelessly from the hand-held motion sensitive device.

24. The method as recited in claim 23, wherein said computing motion recognition signals from the motion signals comprises:

preprocessing the motion signals by a filtering means; and

segmenting the motion signals adaptively according to corresponding magnitudes of the underlying motion signals.

25. The method as recited in claim 24, further comprising:

Calculating classification rates for the motion signals; and

tuning one or more of the motion recognizers by determining which of the motion signals to add as prototypes.

26. The method as recited in claim 25, further comprising:

calculating a classification distance of each of the motion signals to prototypes in the motion recognizers;

labeling the each of the motion signals as undetermined if the classification distance matches none of the prototypes;

labeling the each of the motion signals as a labeled motion associated with one of the prototypes if the classification distance matches only one prototype;

or

labeling the each of the motion signals as a labeled motion associated with one of some of the prototypes if the classification distance matches some of the prototypes, where the one of some of the prototypes is determined by a smallest classification distance; and

adding one or more of the motion signals as prototypes to the one or more of the motion recognizers.

27. The method as recited in claim 26, wherein capacity for the new motion recognizers and substantially all other information needed to perform classification are created automatically and directly from the training set.

28. A method for controlling virtual objects in a video game display in a manner responsive to human motions, the method comprising:

loading at least two motion recognizers in a processing unit, the two motion recognizers being hierarchically related such that both are instances of a same type of motion, each of the motion recognizers including an undefined class such that a misinterpretation of a given move is explicitly disallowed, each of the motion recognizers created in advance by at least one trained user;

receiving a set of motion signals from a hand-held motion-sensitive device, where the hand-held motion-sensitive device is manipulated in response to a virtual environment being displayed on a display screen;

forming a training set including some or all of the motion signals;

tuning one or more of the motion recognizers in the set of motion recognizers with the training set to modify a motion recognition behavior of the one or more tuned motion recognizers;

wherein the processing unit is configured to compute a motion recognition label for each of the set of motion signals in response to the motion recognizers, the motion recognition label is used to select a predetermined action that determines what type of translations and/or rotations to apply to one or more objects in the display.

29. The method as recited in claim 28, further comprising:

Showing the virtual interactive environment on the display screen, wherein movements of at least one of the virtual objects in the video game display is responsive to one or more of the motion recognition labels.

30. The method as recited in claim 29, wherein the motion recognizers are received in a portable storage medium, downloaded via the Internet, or embedded in a video game.

31. The method as recited in claim 28, further comprising receiving additional motion recognizers when the additional motion recognizers becomes available and relevant to the virtual environment.

32. The method as recited in claim 28, further comprising:

tuning one or more of the motion recognizers repeatedly once a predefined motion recognition level is below a certain value.

33. The method as recited in claim 28, wherein said receiving a set of motion signals from the hand-held motion-sensitive device comprises receiving the set of motion signals wirelessly from the hand-held motion sensitive device.

34. The method as recited in claim 33, further comprising:

Preprocessing the set of motion signals by a filtering means; and

Segmenting the set of motion signals adaptively according to corresponding magnitudes of the set of underlying motion signals.

35. The method as recited in claim 34, further comprising:

Calculating classification rates for the set of motion signals; and

tuning each of the motion recognizers by determining which of the set of motion signals to add as prototypes.

36. The method as recited in claim 35, further comprising:

calculating a classification distance of each of the set of motion signals to prototypes in the motion recognizers;

labeling the each of the set of motion signals as undetermined if the classification distance matches none of the prototypes;

labeling the each of the set of motion signals as a labeled motion associated with one of the prototypes if the classification distance matches only one prototype;

or

labeling the each of the set of motion signals as a labeled motion associated with one of some of the prototypes if the classification distance matches some of the prototypes, where the one of some of the prototypes is determined by a smallest classification distance; and

adding one or more of the set of motion signals as prototypes to the one or more of the motion recognizers.