Motion scoring method and apparatus

• US 10,086,283 B2
• Filed: 10/12/2012
• Issued: 10/02/2018
• Est. Priority Date: 10/14/2011
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment Europe Limited (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment Europe Limited (Sony Group Corp.)
• Inventors: Trewartha, Christopher, Derbyshire, Jamie William, Skilton, David John
• Primary Examiner(s): Liddle, Jay
• Application Number:
  US13/650,338
• Publication Number:
  US 20130095926A1
• Time in Patent Office: 2,181 Days
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/46 Computing the game score
• CPC Class Codes: A63F 13/814 Musical performances, e.g. by evaluating the player's ability to follow a notation
• CPC Class Codes: A63F 2300/1012 involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
• CPC Class Codes: A63F 2300/1031 using a wireless connection, e.g. Bluetooth, infrared connections
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/61 Score computation

Abstract

An entertainment system comprises at least one input for receiving data representative of a reference dance performance, a dance analyzer for generating data descriptive of an aspect of the reference dance, a key-point selector for selecting a plurality of key-points in the reference dance responsive to a property of the reference motion data, and an error tolerance calculator for generating error tolerance values at the plurality of key-points in the dance for use in scoring subsequent performances of the dance. The error tolerance values are responsive to a property of the reference motion data. An input may receive telemetry from one or more game controllers in communication with the entertainment system, in which the reference motion data comprises motion data generated by the dance analyzer from the telemetry received from one or more game controllers while the controller(s) was held during the reference performance of the dance.

Claims

1. An entertainment system, comprising:

at least one input operable to receive data representative of a reference dance performance of a dance during a performance recording mode, the performance recording mode occurring while a dancer performs the dance; and

one or more processors operatively coupled to the at least one input, the one or more processors implementing;

a dance analyser operable to generate, from analysis of the received data, reference motion data descriptive of an aspect of the reference dance performance;

a key-point selector operable to select a plurality of event-driven key-points in the reference dance performance, the selection of the plurality of event-driven key points being responsive to a property of the reference motion data, the property of the reference motion data being one or more of a positional, velocity, acceleration or jerk value change over time by more than a selected threshold amount;

an error tolerance calculator operable to generate error tolerance values at the plurality of selected event-driven key-points in the reference dance performance for use in scoring one or more subsequent dance performances, the error tolerance values being linearly or nonlinearly responsive to the property of the reference motion data for each selected event-driven key-point, and the error tolerance values being stored for use in the scoring of the one or more subsequent dance performances;

wherein the one or more processors are configured to;

compare the stored error tolerance values against motion data of the one or more subsequent dance performances to determine a degree of error for each of the subsequent dance performances, the motion data of the one or more subsequent dance performances being of a same type as the received data representative of the reference dance performance; and

when the determined degree of error for a given one of the subsequent dance performances exceeds an error tolerance threshold, generate a score indicative of the determined degree of error.

2. The entertainment system according to claim 1, in which:

one of the at least one inputs is operable to receive telemetry from one or more game controllers in communication with the entertainment system; and

in which

the reference motion data comprises motion data generated by the dance analyser from the telemetry received from the or each game controller whilst the or each game controller was held during the reference dance performance.

3. The entertainment system according to claim 1, in which:

the at least one input is operable to receive captured video images comprising the reference dance performance; and

in which

the reference motion data comprises motion data generated by the dance analyser from analysis of the captured video images.

4. The entertainment system according to claim 1, in which the property of the reference motion data is a ratio of jerk over velocity.

5. The entertainment system according to claim 1, in which the dance analyser is operable to generate motion data for one or more captured repetitions of the reference dance performance, and a property of the reference motion data at a predetermined point is the variance of the motion data.

6. The entertainment system according to claim 1, in which the reference motion data comprises a plurality of sets of motion data, and the error tolerance calculator is operable to generate an error tolerance value in response to only a subset of the motion data sets.

7. The entertainment system according to claim 1, in which the reference motion data comprises a plurality of sets of motion data, and the error tolerance calculator is operable to generate an error tolerance value in response to a plurality of differently weighted motion data sets.

8. The entertainment system according to claim 1, wherein the one or more processors are further configured to implement:

a scoring system operable to estimate an error value based upon a comparison of the reference motion data and player motion data generated responsive to a player'"'"'s attempt to perform the dance; and

in which

the scoring system is operable to rescale one of the reference motion data and player motion data to match the scale of the other of the reference motion data and player motion data prior to comparison.

9. The entertainment system according to claim 8, in which the scoring system is operable to calculate a score responsive to the estimated error value and responsive to the generated error tolerance value.

10. The entertainment system according to claim 1, comprising reference motion data for at least one pre-recorded reference dance, and operable to generate reference motion data for at least one additional reference dance performed by a player of a dance game.

11. The entertainment system according to claim 1, in which the error tolerance calculator is operable to generate the error tolerance values for one or more channels of the reference motion data, each channel corresponding to a given one of the at least one input.

12. The entertainment system according to claim 11, in which the error tolerance calculator is further operable to generate the error tolerance values having different levels of error tolerance for different ones of the one or more channels.

13. The entertainment system according to claim 1, in which the one or more processors are configured to determine an overall difficulty of the reference dance performance according to the error tolerance values.

14. The entertainment system according to claim 1, wherein the error tolerance calculator is operable to generate the error tolerance values responsive to the particular property of the reference motion data by generating a variable error tolerance responsive to one or more qualities of the dance.

15. The entertainment system according to claim 14, wherein the variable error tolerance is subject to at least one of an upper cap and a lower cap.

16. The entertainment system according to claim 1, wherein the key-point selector is further operable to select additional key-points points in the reference dance performance responsive to a time property of the reference dance performance.

17. A method of motion scoring by an entertainment device, comprising the steps of:

receiving data representative of a reference dance performance of a dance from at least one input during a performance recording mode, the performance recording mode occurring while a dancer performs the dance;

analysing, by one or more processors, the received data to generate reference motion data descriptive of an aspect of the reference dance performance;

selecting, by the one or more processors, a plurality of event-driven key-points in the reference dance performance, the selection of the plurality of event-driven key points being responsive to a property of the reference motion data, the property of the reference motion data being one or more of a positional, velocity, acceleration or jerk value change over time by more than a selected threshold amount;

generating, by the one or more processors, error tolerance values at the plurality of event-driven key-points in the reference dance performance for use in scoring one or more subsequent dance performances, the error tolerance values being linearly or nonlinearly responsive to the property of the reference motion data for each selected event-driven key-point;

comparing the stored error tolerance values against motion data of the one or more subsequent dance performances to determine a degree of error for each of the subsequent dance performances, the motion data of the one or more subsequent dance performances being of a same type as the received data representative of the reference dance performance; and

when the determined degree of error for a given one of the subsequent dance performances exceeds an error tolerance threshold, generating a score indicative of the determined degree of error.

18. The method of motion scoring according to claim 17, in which the data representative of the reference dance performance comprises telemetry from one or more game controllers in communication with an entertainment system and held by a user during the reference dance performance;

and the step of analysing the received data comprises;

analysing some or all of the telemetry to generate reference motion data descriptive of an aspect of the reference dance.

19. A non-transitory computer program product storing computer instructions thereon, the instructions, when executed by one or more processors, perform a method of motion scoring, the method comprising:

receiving data representative of a reference dance performance of a dance from at least one input during a performance recording mode, the performance recording mode occurring while a dancer performs the dance;

analysing the received data to generate reference motion data descriptive of an aspect of the reference dance performance;

selecting a plurality of event-driven key-points in the reference dance performance, the selection of the plurality of event-driven key points being responsive to a property of the reference motion data, the property of the reference motion data being one or more of a positional, velocity, acceleration or jerk value change over time by more than a selected threshold amount;

generating error tolerance values at the plurality of event-driven key-points in the reference dance performance for use in scoring one or more subsequent dance performances, the error tolerance values being linearly or nonlinearly responsive to the property of the reference motion data for each selected event-driven key-point;

comparing the stored error tolerance values against motion data of the one or more subsequent dance performances to determine a degree of error for each of the subsequent dance performances, the motion data of the one or more subsequent dance performances being of a same type as the received data representative of the reference dance performance; and

when the determined degree of error for a given one of the subsequent dance performances exceeds an error tolerance threshold, generating a score indicative of the determined degree of error.