METHOD, APPARATUS AND SOFTWARE FOR PROVIDING USER FEEDBACK

• US 20150027297A1
• Filed: 05/02/2014
• Published: 01/29/2015
• Est. Priority Date: 07/26/2013
• Status: Active Grant

• Current Assignee: Sony Europe (Amsterdam) BV (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Europe Limited (Sony Group Corp.)
• Inventors: Avitabile, Antonio, Chen, Xiaoming
• Granted Patent: US 9,208,763 B2
• Time in Patent Office: Days
• US Class Current: 84/470.R
• CPC Class Codes: G01B 11/14 for measuring distance or clearance between spaced objects or spaced apertures G01B11/26 takes precedence; rangefinders G01C3/00
• CPC Class Codes: G01L 1/10 by measuring variations of frequency of stressed vibrating elements, e.g. of stressed strings using resistance strain gauges G01L1/22
• CPC Class Codes: G01L 5/16 for measuring several components of force
• CPC Class Codes: G09B 15/003 with indication of the keys or strings to be played on instruments
• CPC Class Codes: G09B 15/06 Devices for exercising or strengthening fingers or arms; Devices for holding fingers or arms in a proper position for playing for teaching typing G09B13/00; exercising apparatus for developing or strengthening the muscles for physical training A63B21/00, A63B23/00
• CPC Class Codes: G10H 1/18 Selecting circuits
• CPC Class Codes: G10H 1/368 displaying animated or moving pictures synchronized with the music or audio part

Abstract

A device for providing feedback to a user of a string instrument is described. The device comprises receiving circuitry operable in use to communicate with at least one finger sensor capable of detecting the force of a finger on a string, a camera operable in use to capture an image and a display operable in use to display data, wherein the receiving circuitry, camera and display are connected to processing circuitry which is operable to: detect, from the receiving circuitry, the force of a user'"'"'s finger on at least one string at a given time; detect, from the camera, the position of the user'"'"'s finger on the at least one string at the given time; compare the detected force of the user'"'"'s finger and the position of the user'"'"'s finger on the at least one string at the given time with a predetermined value of force and position at the given time; and generate, on the basis of the said comparison, feedback to display on the display.

Claims

1. A method of providing feedback to a user of a string instrument, comprising the steps:

detecting the force of a user'"'"'s finger on at least one string at a given time;

detecting the position of the user'"'"'s finger on the at least one string at the given time;

comparing the detected force of the user'"'"'s finger and the position of the user'"'"'s finger on the at least one string at the given time with a predetermined value of force and position at the given time; and

generating feedback to display to the user on the basis of the said comparison.

2. A method according to claim 1 further comprising detecting the frequency of vibration on the user'"'"'s finger at the given time, calculating the frequency spectrum of the vibration at the given time and determining the number of strings played by the user at the given time.

3. A method according to claim 1 further comprising receiving an image of the string instrument and determining the position of the user'"'"'s finger on the string instrument from the received image.

4. A method according to claim 3, wherein the position of the user'"'"'s finger is determined by performing edge detection on the received image.

5. A method according to claim 3, comprising performing image detection on the received image, wherein the position of the user'"'"'s finger is determined by detecting a coloured finger sensor in the received image.

6. A method according to claim 1, comprising determining the difference between the detected force and the predetermined value of force at the given time and determining an error based on the determined differences over a predetermined time.

7. A method according to claim 1, comprising determining the difference between the detected force and the predetermined value of force at the given time, and determining when the difference exceeds a threshold over a predetermined period of time.

16. Computer software comprising computer readable code which when loaded onto a computer configures the computer to perform a method according to claim 1.

17. A storage medium configured to store the computer software of claim 16 thereon or therein.

8. A device for providing feedback to a user of a string instrument, comprising:

receiving circuitry operable in use to communicate with at least one finger sensor capable of detecting the force of a finger on a string, a camera operable in use to capture an image and a display operable in use to display data, wherein the receiving circuitry, camera and display are connected to processing circuitry which is operable to;

detect, from the receiving circuitry, the force of a user'"'"'s finger on at least one string at a given time;

detect, from the camera, the position of the user'"'"'s finger on the at least one string at the given time;

compare the detected force of the user'"'"'s finger and the position of the user'"'"'s finger on the at least one string at the given time with a predetermined value of force and position at the given time; and

generate, on the basis of the said comparison, feedback to display on the display.

9. A device according to claim 8 wherein the processing circuitry is further operable to detect, from the receiving circuitry, the frequency of vibration on the user'"'"'s finger at the given time, and to calculate the frequency spectrum of the vibration at the given time and determining the number of strings played by the user at the given time.

10. A device according to claim 8 wherein the processing circuitry is further operable in use to receive an image of the string instrument from the camera, and from the received image is operable to determine the position of the user'"'"'s finger on the string instrument.

11. A device according to claim 10, wherein the position of the user'"'"'s finger is determined by performing edge detection on the received image.

12. A device according to claim 10, wherein the processing circuitry is further operable in use to perform image detection on the received image, wherein the position of the user'"'"'s finger is determined by detecting a coloured finger sensor in the received image.

13. A device according to claim 8, wherein the processing circuitry is further operable in use to determine the difference between the detected force and the predetermined value of force at the given time and to determine an error based on the determined differences over a predetermined time.

14. A device according to claim 8, wherein the processing circuitry is further operable in use to determine the difference between the detected force and the predetermined value of force at the given time, and determine when the difference exceeds a threshold over a predetermined period of time.

15. A tablet device for providing feedback to a user of a string instrument, comprising:

receiving circuitry operable in use to communicate with at least one finger sensor capable of detecting the force of a finger on a string, a camera operable in use to capture an image and a display operable in use to display data, wherein the receiving circuitry, camera and display are connected to processing circuitry which is operable to;

detect, from the receiving circuitry, the force of a user'"'"'s finger on at least one string at a given time;

detect, from the camera, the position of the user'"'"'s finger on the at least one string at the given time;

compare the detected force of the user'"'"'s finger and the position of the user'"'"'s finger on the at least one string at the given time with a predetermined value of force and position at the given time; and

generate, on the basis of the said comparison, feedback to display on the display.