Method, apparatus and software for providing user feedback

• US 9,208,763 B2
• Filed: 05/02/2014
• Issued: 12/08/2015
• Est. Priority Date: 07/26/2013
• Status: Expired due to Fees

• Current Assignee: Sony Europe BV (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Europe Limited (Sony Group Corp.)
• Inventors: Avitabile, Antonio, Chen, Xiaoming
• Primary Examiner(s): Warren, David
• Application Number:
  US14/268,324
• Publication Number:
  US 20150027297A1
• Time in Patent Office: 585 Days
• Field of Search: 846/15
• US Class Current: 1/1
• CPC Class Codes: G01B 11/14 for measuring distance or clearance between spaced objects or spaced apertures G01B11/26 takes precedence; rangefinders G01C3/00
• CPC Class Codes: G01L 1/10 by measuring variations of frequency of stressed vibrating elements, e.g. of stressed strings using resistance strain gauges G01L1/22
• CPC Class Codes: G01L 5/16 for measuring several components of force
• CPC Class Codes: G09B 15/003 with indication of the keys or strings to be played on instruments
• CPC Class Codes: G09B 15/06 Devices for exercising or strengthening fingers or arms; Devices for holding fingers or arms in a proper position for playing for teaching typing G09B13/00; exercising apparatus for developing or strengthening the muscles for physical training A63B21/00, A63B23/00
• CPC Class Codes: G10H 1/18 Selecting circuits
• CPC Class Codes: G10H 1/368 displaying animated or moving pictures synchronized with the music or audio part

Abstract

A device for providing feedback to a user of a string instrument is described. The device comprises receiving circuitry operable in use to communicate with at least one finger sensor capable of detecting the force of a finger on a string, a camera operable in use to capture an image and a display operable in use to display data, wherein the receiving circuitry, camera and display are connected to processing circuitry which is operable to: detect, from the receiving circuitry, the force of a user'"'"'s finger on at least one string at a given time; detect, from the camera, the position of the user'"'"'s finger on the at least one string at the given time; compare the detected force of the user'"'"'s finger and the position of the user'"'"'s finger on the at least one string at the given time with a predetermined value of force and position at the given time; and generate, on the basis of the said comparison, feedback to display on the display.

Claims

1. A method of providing feedback to a user of a string instrument, comprising:

detecting a force of a user'"'"'s finger on at least one string at a given time;

detecting a position of the user'"'"'s finger on the at least one string at the given time;

comparing the detected force of the user'"'"'s finger and the position of the user'"'"'s finger on the at least one string at the given time with at least two predetermined values of force and position at the given time; and

generating feedback to display to the user on the basis of the said comparison.

2. The method according to claim 1, further comprising:

detecting a frequency of vibration on the user'"'"'s finger at the given time;

calculating a frequency spectrum of the vibration at the given time; and

determining a number of strings played by the user at the given time.

3. The method according to claim 1, further comprising:

receiving an image of the string instrument and

determining the position of the user'"'"'s finger on the string instrument from the received image.

4. The method according to claim 3, wherein the position of the user'"'"'s finger is determined by performing edge detection on the received image.

5. The method according to claim 3, further comprising:

performing image detection on the received image, wherein the position of the user'"'"'s finger is determined by detecting a colored finger sensor in the received image.

6. The method according to claim 1, further comprising:

determining a difference between the detected force and at least one of the predetermined values of force at the given time; and

determining an error based on the determined differences over a predetermined time.

7. The method according to claim 1, further comprising:

determining a difference between the detected force and at least one of the predetermined values of force at the given time; and

determining when the difference exceeds a threshold over a predetermined period of time.

16. Computer software comprising computer readable code which when loaded onto a computer configures the computer to perform the method according to claim 1.

17. A non-transitory storage medium configured to store the computer software of claim 16 thereon or therein.

8. A device for providing feedback to a user of a string instrument, comprising:

receiving circuitry configured to communicate with at least one finger sensor capable of detecting a force of a finger on a string, a camera configured to capture an image and a display configured to display data, wherein the receiving circuitry, camera and display are connected to processing circuitry which is configured to;

detect, from the receiving circuitry, the force of a user'"'"'s finger on at least one string at a given time;

detect, from the camera, a position of the user'"'"'s finger on the at least one string at the given time;

compare the detected force of the user'"'"'s finger and the position of the user'"'"'s finger on the at least one string at the given time with at least two predetermined values of force and position at the given time; and

generate, on the basis of the said comparison, feedback to display on the display.

9. The device according to claim 8, wherein the processing circuitry is further configured to

detect, from the receiving circuitry, a frequency of vibration on the user'"'"'s finger at the given time;

calculate a frequency spectrum of the vibration at the given time; and

determine a number of strings played by the user at the given time.

10. The device according to claim 8, wherein the processing circuitry is further configured to

receive an image of the string instrument from the camera;

and

determine the position of the user'"'"'s finger on the string instrument from the received image.

11. The device according to claim 10, wherein the position of the user'"'"'s finger is determined by performing edge detection on the received image.

12. The device according to claim 10, wherein the processing circuitry is further configured to perform image detection on the received image, and the position of the user'"'"'s finger is determined by detecting a colored finger sensor in the received image.

13. The device according to claim 8, wherein the processing circuitry is further configured to

determine a difference between the detected force and at least one of the predetermined values of force at the given time;

and

determine an error based on the determined differences over a predetermined time.

14. The device according to claim 8, wherein the processing circuitry is further configured to

determine a difference between the detected force and at least one of the predetermined values of force at the given time;

and

determine when the difference exceeds a threshold over a predetermined period of time.

15. A tablet device for providing feedback to a user of a string instrument, comprising:

receiving circuitry configured to communicate with at least one finger sensor capable of detecting a force of a finger on a string, a camera configured to operable in use to capture an image and a display configured to display data, wherein the receiving circuitry, camera and display are connected to processing circuitry which is configured to;

detect, from the receiving circuitry, the force of a user'"'"'s finger on at least one string at a given time;

detect, from the camera, a position of the user'"'"'s finger on the at least one string at the given time;

compare the detected force of the user'"'"'s finger and the position of the user'"'"'s finger on the at least one string at the given time with at least two predetermined values of force and position at the given time; and

generate, on the basis of the said comparison, feedback to display on the display.