User selection and navigation based on looped motions

• US 8,613,666 B2
• Filed: 08/31/2010
• Issued: 12/24/2013
• Est. Priority Date: 08/31/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Esaki, Chris, Yano, Keiichi
• Primary Examiner(s): McClellan, James S
• Application Number:
  US12/872,790
• Publication Number:
  US 20120052942A1
• Time in Patent Office: 1,211 Days
• Field of Search: 463 1- 5, 463 7- 9, 463 30- 34, 463 36- 43, 463 49- 57, 273/108.1, 273/127.R, 273/148.R, 273/148.B, 273/309, 273/317.1, 273/340, 273/343, 273/348, 273361-367, 273/461, 250/205, 250330-334, 250/338.1, 250/339.02, 250/339.06, 250/339.09, 250/339.11, 250/339.14, 250/341.8, 250/342, 250344-345, 250350-351, 250/353, 345 11- 13, 345 21- 23, 345 31- 34, 345/24, 345/419, 345467-469, 345/473, 345/539, 345543-544, 345/625, 345/636, 345/638, 345653-656, 345664-666, 345682-683, 345/686, 345949-950, 345/FOR.139, 345/FOR.153, 348/14.15, 348/39, 348/42, 348 47- 52, 348/115, 348/117, 348/121, 348135-137, 348/141, 348/211.2, 348/211.4, 3482117-2118, 348/211.14, 348/211.99, 348/576, 348588-589, 348/719, 348/721, 348/734, 348/E13.004, 348E13064-E13067, 37524015-24016, 375/240.25, 398106-108, 398111-112, 398/118, 398/123, 398127-131, 434 37- 38, 434 43- 44, 434/69, 434/118, 434/240, 434256-257, 717168-178
• US Class Current: 463/39
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/46 Computing the game score
• CPC Class Codes: A63F 13/533 for prompting the player, e.g. by displaying a game menu
• CPC Class Codes: A63F 13/5375 for graphically or textually suggesting an action, e.g. by displaying an arrow indicating a turn in a driving game
• CPC Class Codes: A63F 13/814 Musical performances, e.g. by evaluating the player's ability to follow a notation
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/305 for providing a graphical or textual hint to the player
• CPC Class Codes: A63F 2300/308 Details of the user interface
• CPC Class Codes: A63F 2300/6607 for animating game characters, e.g. skeleton kinematics
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

A system for performing user selection of an option is provided. A user interface displays one or more options. Each option is associated with a sequence of a looped motion. In response, a user will perform one of the displayed sequences of looped motion. User motion data is received from one or more sensors connected to the computing device. The user'"'"'s progress of performing the sequence of the looped motion is determined based on the user motion data matching one or more looped motion criteria associated with the performed looped motion. The user'"'"'s progress of performing the sequence of the looped motion is displayed to the user. A selection of the option associated with the performed looped motion is automatically triggered in response to determining that the user has completed the sequence of looped motion.

Claims

1. A method for performing user selection of an option, comprising:

displaying an option and a sequence of a looped motion associated with the option by a user interface of a computing device, the computing device including one or more processors and memory;

receiving user motion data from a sensor connected to the computing device;

determining if the user is performing the looped motion based on the received user motion data;

if the user is performing the looped motion, determining if the user motion data matches one or more looped motion criteria by determining if energy produced through a movement of a body part in the user motion data matches energy of a corresponding moving point in the looped motion, wherein the energy produced through the movement of the body part is derived as a function of a mass of the body part and a speed of the movement of the body part;

determining the user'"'"'s progress of performing the sequence of the looped motion based on the user motion data matching the one or more looped motion criteria;

displaying the user'"'"'s progress of performing the sequence of the looped motion;

determining that the user has completed the sequence of looped motion based on the user'"'"'s progress of performing the sequence of the looped motion; and

automatically triggering a selection of the option displayed in response to determining that the user has completed the sequence of looped motion and performing a function associated with the selection of the option.

2. The method of claim 1, wherein determining if the user motion data matches the one or more looped motion criteria further comprises:

determining if a movement of a body part in the user motion data is aligned with a musical beat sequence associated with the looped motion.

3. The method of claim 1, wherein determining if the user motion data matches the one or more looped motion criteria further comprises:

determining if a movement of a body part in the user motion data matches a match threshold value associated with the looped motion, wherein the match threshold value is a measure of correlation of the user motion data with the looped motion.

4. The method of claim 1, wherein:

the looped motion comprises a repeatable motion performed by the user, the repeatable motion comprising at least one of an exercise routine or a dance move in an application executing in the computing device.

5. The method of claim 1, wherein displaying the user'"'"'s progress of performing the sequence of the looped motion further comprises:

displaying a progression tracking bar to the user, wherein a filled fraction of the progression tracking bar indicates a percentage of completion of the sequence of the looped motion.

6. The method of claim 1, wherein:

the option depicts an on-screen representation of a character performing the sequence of the looped motion to the user.

7. The method of claim 1, wherein prompting a user to perform a sequence of a looped motion further comprises:

presenting the user with guided text instructing the user to perform the sequence of the looped motion displayed in the option.

8. The method of claim 1, wherein:

the displaying the option includes presenting the user with multiple skill-based options via the user interface in the computing device;

each skill-based option is associated with a different sequence of looped motion; and

the option is one of the skill-based options.

9. The method of claim 1, wherein:

the prompting the user comprises presenting the user with guided text instructing the user to perform the sequence of the looped motion displayed in the option;

the displaying the option comprises depicting an on-screen representation of a character performing the sequence of the looped motion;

the determining if the user is performing the looped motion comprises determining if the user motion data matches one or more pre-defined looped motions; and

the displaying the user'"'"'s progress of performing the sequence of the looped motion comprises displaying a progression tracking bar to the user, wherein a filled fraction of the progression tracking bar indicates a percentage of completion of the sequence of the looped motion.

10. One or more processor readable memory storage devices having processor readable code embodied on said one or more processor readable memory storage devices, the processor readable code for programming one or more processors to perform a method comprising:

presenting a user with multiple skill-based options via a user interface in a computing device, each skill-based option is associated with a different sequence of a looped motion;

displaying the different sequences of the looped motion, the different sequences of the looped motion each having different skill levels to performing the associated sequence of the looped motion;

receiving user motion data from a sensor connected to the computing device;

determining if the user is performing one of the looped motions depicted by the multiple skill-based options, based on the received user motion data;

if the user is performing one of the looped motions, determining if the user motion data matches one or more looped motion criteria for the performed loop motion by determining if energy produced through a movement of a body part in the user motion data matches energy of a corresponding moving point in the looped motion, wherein the energy produced through the movement of the body part is derived as a function of a mass of the body part and a speed of the movement of the body part;

determining user'"'"'s progress of performing the sequence of the looped motion based on the user motion data matching the one or more looped motion criteria;

displaying the user'"'"'s progress performing the sequence of the looped motion;

determining that the user has completed the sequence of looped motion based on the user'"'"'s progress of performing the sequence of the looped motion; and

automatically triggering a selection of one of the multiple skill-based options in response to determining that the user has completed the sequence of the looped motion.

11. One or more processor readable memory storage devices according to claim 10, wherein:

the multiple skill-based options comprise at least one of a beginner level, an intermediate level and an advanced level corresponding to performing the one or more looped motions.

12. One or more processor readable memory storage devices according to claim 10, wherein determining if the user motion data matches one or more looped motion criteria for the performed loop motion further comprises:

determining if a movement of a body part in the user motion data is aligned with a musical beat sequence associated with the looped motion.

13. One or more processor readable memory storage devices according to claim 10, wherein displaying the user'"'"'s progress of performing the sequence of the looped motion further comprises:

displaying a progression tracking bar to the user, wherein a filled fraction of the progression tracking bar indicates a percentage of completion of the sequence of the looped motion.

14. One or more processor readable memory storage devices according to claim 10, wherein determining if the user motion data matches one or more looped motion criteria for the performed loop motion further comprises:

determining if a movement of a body part in the user motion data matches a match threshold value associated with the looped motion, wherein the match threshold value is a measure of correlation of the user motion data with the looped motion.

15. An apparatus to perform user selection of an option, comprising:

a depth camera for capturing user motion data of a user in a field of view of the depth camera;

a computing device, including one or more processors and memory, communicatively coupled to the depth camera for receiving the user motion data of the user;

a display device communicatively coupled to and under the control of the computing device;

the computing device causing the display device to display an option and a sequence of a looped motion associated with the option;

the computing device determining if the user is performing the looped motion based on the received user motion data;

if the user is performing the looped motion, the computing device determining if the user motion data matches one or more looped motion criteria by the computing device determining if energy produced through a movement of a body part in the user motion data matches energy of a corresponding moving point in the looped motion, wherein the energy produced through the movement of the body part is derived as a function of a mass of the body part and a speed of the movement of the body part;

the computing device determining the user'"'"'s progress of performing the sequence of the looped motion based on the user motion data matching the one or more looped motion criteria;

the computing device causing the display to display the user'"'"'s progress of performing the sequence of the looped motion;

the computing device determining that the user has completed the sequence of looped motion based on the user'"'"'s progress of performing the sequence of the looped motion;

the computing device automatically triggering a selection of the option displayed in response to determining that the user has completed the sequence of looped motion; and

the computing device performing a function associated with the selection of the option.

16. The apparatus of claim 15, wherein:

the computing device causes the display to display the users'"'"' progress of performing the sequence of the looped motion.

17. The apparatus of claim 16, wherein:

the computing device causes the display to display a progression tracking bar, wherein a filled fraction of the progression tracking bar indicates a percentage of completion of the sequence of the looped motion by the user.

18. The apparatus of claim 15, wherein the computing device determining if the user motion data matches one or more looped motion criteria further comprises:

the computing device determining if a movement of a body part in the user motion data is aligned with a musical beat sequence associated with the looped motion.

19. The apparatus of claim 15, wherein the computing device determining if the user motion data matches one or more looped motion criteria further comprises:

determining if a movement of a body part in the user motion data matches a match threshold value associated with the looped motion, wherein the match threshold value is a measure of correlation of the user motion data with the looped motion.