Notation for gesture-based composition

• US 10,319,352 B2
• Filed: 04/28/2017
• Issued: 06/11/2019
• Est. Priority Date: 04/28/2017
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Kar, Swarnendu, Jain, Dipika, Lin, Yingxi A, Shah, Saurin
• Primary Examiner(s): Warren, David S
• Application Number:
  US15/582,120
• Publication Number:
  US 20180315406A1
• Time in Patent Office: 774 Days
• Field of Search: 84615, 84477 R, 844831, 844832
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06V 40/107 Static hand or arm
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: G10G 1/00 Means for the representation of music
• CPC Class Codes: G10H 1/0008 Associated control or indicating means
• CPC Class Codes: G10H 1/18 Selecting circuits
• CPC Class Codes: G10H 2220/201 for movement interpretation, i.e. capturing and recognizing a gesture or a specific kind of movement, e.g. to control a musical instrument
• CPC Class Codes: G10H 2220/321 Garment sensors, i.e. musical control means with trigger surfaces or joint angle sensors, worn as a garment by the player, e.g. bracelet, intelligent clothing vital parameter sensing G10H2220/371; wearable interfaces in general G06F3/00
• CPC Class Codes: G10H 2220/326 Control glove or other hand or palm-attached control device
• CPC Class Codes: G10H 2240/161 Memory and use thereof, in electrophonic musical instruments, e.g. memory map data structures G10H2240/005; memory cache G10H2230/031; libraries G10H2240/121; files G10H2240/011

Abstract

Various systems and methods for air gesture-based composition and instruction systems are described herein. A composition system for composing gesture-based performances may receive an indication of an air gesture performed by a user; reference a mapping of air gestures to air gesture notations to identify an air gesture notation corresponding to the air gesture; and store an indication of the air gesture notation in a memory of the computerized composition system. Another system used for instruction may present a plurality of air gesture notations in a musical arrangement; receive an indication of an air gesture performed by a user; reference a mapping of air gestures to air gesture notations to identify an air gesture notation corresponding to the air gesture; and guide the user through the musical arrangement by sequentially highlighting the air gesture notations in the musical arrangement based on the mapping of air gestures to air gesture notations.

Claims

1. A computerized composition system for composing gesture-based performances, the system comprising:

a processor subsystem; and

memory including instructions, which when executed by the processor subsystem, cause the processor subsystem to;

receive an indication of an air gesture performed by a user;

reference a mapping of air gestures to air gesture notations to identify an air gesture notation corresponding to the air gesture; and

store an indication of the air gesture notation in a memory of the computerized composition system,

wherein the air gesture notation comprises a circle with a shaded portion, the shaded portion indicating a directionality of the air gesture, or the air gesture comprises a non-equilateral triangle indicating a directionality of the air gesture.

2. The system of claim 1, wherein to receive the indication of the air gesture, the processor subsystem is to receive sensor data via a sensor interface circuit, from a sensor worn by the user.

3. The system of claim 2, wherein the sensor is integrated into a bracelet, a glove, an anklet, or a shoe.

4. The system of claim 2, wherein the indication of the air gesture is provided by a gesture detection processor.

5. The system of claim 1, wherein to receive the indication of the air gesture, the processor subsystem is to receive image data from a camera observing the user.

6. The system of claim 1, wherein the shaded portion indicates a palm tap air gesture.

7. The system of claim 1, wherein the non-equilateral triangle indicates a knife tap air gesture.

8. The system of claim 1, wherein the air gesture notation comprises a modifier that alters the air gesture notation.

9. A method for composing gesture-based performances, the method comprising:

receiving, at a computerized composition system, an indication of an air gesture performed by a user;

referencing a mapping of air gestures to air gesture notations to identify an air gesture notation corresponding to the air gesture; and

storing an indication of the air gesture notation in a memory of the computerized composition system,

wherein the air gesture notation comprises a modifier that alters the air gesture notation, and wherein the modifier comprises a hat symbol oriented over the air gesture notation, the hat symbol indicating that performance of a second air gesture is performed simultaneously with the air gesture.

10. The method of claim 9, wherein the hat symbol indicates that the user perform a foot gesture as the second air gesture.

11. The method of claim 9, wherein the hat symbol indicates that a musical note is modified from the air gesture notation without the hat symbol.

12. The method of claim 11, wherein the musical note is modified with a sharp.

13. The method of claim 9, wherein the air gesture notation indicates a handedness of the air gesture.

14. The method of claim 13, wherein the air gesture notation is in a first color for a right hand performance of the air gesture, and the air gesture notation is in a second color for a left hand performance of the air gesture, the first and second colors being different colors.

15. The method of claim 13, wherein the air gesture notation is presented on a plurality of staffs, the plurality of staffs including a first staff for right hand air gesture notations, and a second staff for left hand air gesture notations.

16. The method of claim 9, further comprising presenting a plurality of air gesture notations in a musical arrangement.

17. The method of claim 16, wherein presenting the plurality of air gesture notations comprises presenting the plurality of air gesture notations on a musical staff.

18. The method of claim 16, wherein presenting the plurality of air gesture notations comprises presenting the plurality of air gesture notations without a musical staff.

19. The method of claim 16, wherein presenting the plurality of air gesture notations comprises presenting a first set of the plurality of air gesture notations and a second set of the plurality of air gesture notations, the first set indicating musical arrangement and the second set indicating dance choreography, the first and second set of air gesture notations coordinated in time.

20. At least one non-transitory machine-readable medium including instructions for composing gesture-based performances, which when executed by a computerized composition system, cause the computerized composition system to:

receive an indication of an air gesture performed by a user;

reference a mapping of air gestures to air gesture notations to identify an air gesture notation corresponding to the air gesture; and

store an indication of the air gesture notation in a memory of the computerized composition system,

wherein the air gesture notation comprises a modifier that alters the air gesture notation, and wherein the modifier comprises a hat symbol oriented over the air gesture notation, the hat symbol indicating that performance of a second air gesture is performed simultaneously with the air gesture.

21. The at least one non-transitory machine-readable medium of claim 20, further comprising instructions to present a plurality of air gesture notations in a musical arrangement.