Accelerometer determined input velocity

• US 8,884,888 B2
• Filed: 08/30/2010
• Issued: 11/11/2014
• Est. Priority Date: 08/30/2010
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Chin, Robert Michael
• Primary Examiner(s): Chang, Kent
• Assistant Examiner(s): BRITTINGHAM, NATHANIEL P
• Application Number:
  US12/871,507
• Publication Number:
  US 20120050176A1
• Time in Patent Office: 1,534 Days
• Field of Search: 345/173
• US Class Current: 345/173
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/2145 the surface being also a display device, e.g. touch screens
• CPC Class Codes: A63F 13/426 involving on-screen location information, e.g. screen coordinates of an area at which the player is aiming with a light gun
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/44 involving timing of operations, e.g. performing an action within a time slot
• CPC Class Codes: A63F 13/54 involving acoustic signals, e.g. for simulating revolutions per minute [RPM] dependent engine sounds in a driving game or reverberation against a virtual wall
• CPC Class Codes: A63F 13/63 by the player, e.g. authoring using a level editor
• CPC Class Codes: A63F 13/814 Musical performances, e.g. by evaluating the player's ability to follow a notation
• CPC Class Codes: A63F 2300/105 using inertial sensors, e.g. accelerometers, gyroscopes
• CPC Class Codes: A63F 2300/1075 using a touch screen
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/6081 generating an output signal, e.g. under timing constraints, for spatialization
• CPC Class Codes: A63F 2300/8047 Music games
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G10H 1/02 Means for controlling the tone frequencies, e.g. attack or decay; Means for producing special musical effects, e.g. vibratos or glissandos
• CPC Class Codes: G10H 1/055 by switches with variable impedance elements
• CPC Class Codes: G10H 1/34 Switch arrangements, e.g. keyboards or mechanical switches specially adapted for electrophonic musical instruments
• CPC Class Codes: G10H 1/46 Volume control
• CPC Class Codes: G10H 2210/211 : Pitch vibrato, i.e. repetitive and smooth variation in pitch, e.g. as obtainable with a whammy bar or tremolo arm on a guitar non-repetitive smooth pitch variation, e.g. glissando G10H2220/221; repeatedly alternating stepwise in pitch between two notes G10H2210/191
• CPC Class Codes: G10H 2220/096 : using a touch screen touch screen note input, e.g. using a displayed keyboard G10H2220/241; personal digital assistant [PDA] G10H2230/015
• CPC Class Codes: G10H 2220/241 : on touchscreens, i.e. keys, frets, strings, tablature or staff displayed on a touchscreen display for note input purposes
• CPC Class Codes: G10H 2220/395 : Acceleration sensing or accelerometer use, e.g. 3D movement computation by integration of accelerometer data, angle sensing with respect to the vertical, i.e. gravity sensing. conductor baton movement sensing G10H2220/206, angle sensing without reference to gravity G10H2220/391; player body joint angle sensing G10H2220/321
• CPC Class Codes: G10H 2230/015 : PDA [personal digital assistant] or palmtop computing devices used for musical purposes, e.g. portable music players, tablet computers, e-readers or smart phones in which mobile telephony functions need not be used touch-screen interfaces G10H2220/096
• CPC Class Codes: G10H 2230/065 : Spint piano, i.e. mimicking acoustic musical instruments with piano, cembalo or spinet features, e.g. with piano-like keyboard; Electrophonic aspects of piano-like acoustic keyboard instruments; MIDI-like control therefor
• CPC Class Codes: G10H 2240/056 : MIDI or other note-oriented file format

Abstract

The disclosed technology relates to an electronic apparatus including a processor, a touch-sensitive display, and an accelerometer. An exemplary method includes receiving in the processor an acceleration value outputted by the accelerometer in response to a user touching a position on the touch-sensitive display with a given velocity. The method includes associating the acceleration value with a function, such as the output of a musical note, indicated by a touched location on the touch-sensitive display. The method includes modifying the function, such as audible volume of the musical note, in accordance with the acceleration value.

Claims

1. An electronic apparatus comprising:

an accelerometer configured to periodically output acceleration values that represent touch inputs on a touch-sensitive surface; and

a processor configured to perform operations comprising;

receiving a plurality of acceleration values outputted by the accelerometer;

modifying each acceleration value of the plurality of acceleration values according to a mathematical function to provide a plurality of modified acceleration values;

selecting a largest modified acceleration value of the plurality of modified acceleration values;

determining a ratio of the largest modified acceleration value and a constant that is greater than the largest modified acceleration value;

associating the determined ratio modified acceleration value with an audio function indicated by a touched location on a touch-sensitive display; and

modifying the audio function in accordance with the determined ratio.

2. The electronic apparatus as set forth in claim 1, wherein the audio function is the output of a musical note and modifying the audio function comprises modifying an audible volume of said musical note.

3. The electronic apparatus as set forth in claim 1, wherein the audio function is the output of a musical note and modifying the audio function comprises modifying a pitch of said musical note to create vibrato.

4. The electronic apparatus as set forth in claim 1, wherein the audio function is the output of a musical note and modifying the audio function comprises modifying an attack time of said musical note.

5. The electronic apparatus as set forth in claim 1, further comprising scaling each acceleration value of the plurality of acceleration values with a scaling factor when more than one touched location exists on the touch-sensitive display.

6. The electronic apparatus as set forth in claim 5, wherein said scaling factor is determined by a distance between touched locations on the touch-sensitive display.

7. The electronic apparatus as set forth in claim 6, wherein said scaling factor is adjusted in accordance with the position of said touched location on the touch-sensitive display.

8. The electronic apparatus as set forth in claim 1, wherein the selected modified acceleration value of the plurality of acceleration values is the largest modified acceleration value of the plurality of acceleration values, the method further comprising:

comparing each of the plurality of modified acceleration values with a minimum threshold value;

receiving additional modified acceleration values in response to determining that none of the plurality of modified acceleration values exceeds the threshold value; and

using the additional modified acceleration values to determine the largest modified acceleration value.

9. The electronic apparatus as set forth in claim 1, wherein output of said accelerometer is filtered prior to conversion to an acceleration value to compensate for gravitational influence on a response of said accelerometer to touching of said display.

31. The apparatus of claim 1, wherein the plurality of acceleration values are consecutively outputted by the accelerometer, and wherein modifying each acceleration value of the plurality of acceleration values according to a mathematical function comprises applying the mathematical function to each of the consecutively outputted plurality of acceleration values.

32. The apparatus of claim 1, wherein the mathematical function is at least one of a high-pass filtering function, a low-pass filtering function, or a slope determination function, and wherein the operations further comprise storing each modified acceleration value of the plurality of modified acceleration values.

33. The apparatus of claim 1, wherein modifying the audio function in accordance with the determined ratio comprises:

assigning the ratio to the audio function such that in response to a touch at the touched location on the touch-sensitive display, the audio function executes to output a maximum audio function value multiplied by the determined ratio.

34. The apparatus of claim 33, further comprising modifying the ratio by a scaling factor is determined by a distance between touched locations on the touch-sensitive display.

10. A method for modifying an audio function that is activated by touching a location on a touch-sensitive display, comprising:

receiving a plurality of values from an accelerometer coupled to a touch-sensitive display, the plurality of values representing touch inputs on the touch-sensitive display;

modifying each value of the plurality of values according to a mathematical function to provide a plurality of modified values;

selecting a largest modified value of the plurality of values to be associated with the audio function;

determining a ratio of the largest modified value and a constant that is greater than the largest modified acceleration value;

receiving coordinates of a location on the touch-sensitive display that has been touched by a user;

retrieving the audio function corresponding to the received coordinates;

associating the determined ratio with the retrieved audio function;

modifying the audio function in accordance with the modified value; and

executing the audio function modified in accordance with the determined ratio to an output device.

11. The method as set forth in claim 10, wherein the audio function is the playing of a musical note, and modifying the audio function comprises modifying an output volume of said note.

12. The method as set forth in claim 10, wherein the audio function is the playing of a musical note, and modifying the audio function comprises modifying a pitch of said musical note to create vibrato.

13. The method as set forth in claim 10, wherein the audio function is the playing of a musical note, and modifying the audio function comprises modifying an attack time of said musical note.

14. The method as set forth in claim 10, wherein said output device is a speaker.

15. The method as set forth in claim 10, wherein the plurality of values are received in a predetermined period of time with respect to receiving said coordinates, and wherein the method further comprises storing the plurality of values.

16. The method as set forth in claim 15, further comprising;

storing the plurality of modified values;

determining that none of the stored plurality of modified values exceed a threshold value; and

in response to determining that none of the stored plurality of modified values exceed the threshold value;

receiving additional values from the accelerometer,

modifying each of the additional values to provide modified additional values, and

using the modified additional values to determine the ratio.

17. The method as set forth in claim 10, wherein modifying each value comprises filtering each value to compensate for gravitational influence on a response of said accelerometer to touching of said display.

35. The method of claim 10, wherein the plurality of values are consecutively outputted by the accelerometer, and wherein modifying each value of the plurality of values according to a mathematical function comprises applying the mathematical function to each of the consecutively outputted plurality of values.

18. A system comprising:

a touch screen;

an accelerometer; and

a processor executing instructions to perform operations comprising;

receiving a plurality of acceleration values outputted by the accelerometer, the plurality of acceleration values representing touch input on the touch screen;

modifying each acceleration value of the plurality of acceleration values according to at least one of a high-frequency filtering function, a low-frequency filtering function, or a slope derivation function resulting in a plurality of modified acceleration values;

selecting a largest modified acceleration value of the plurality of modified acceleration values;

determining a ratio of the largest modified acceleration value and a constant that is greater than the largest modified acceleration value;

associating the determined ratio with an audio function indicated by a touched location on said touch-sensitive display; and

modifying the audio function in accordance with the determined ratio.

19. The system of claim 18, wherein the audio function is the output of a musical note and the operations further comprise modifying an audible volume of said musical note.

20. The system of claim 18, wherein the audio function is the output of a musical note and the operations further comprise modifying a pitch of said musical note to create vibrato.

21. The system of claim 18, wherein the audio function is the output of a musical note and the operations further comprise modifying an attack time of said musical note.

22. The system of claim 18, the operations further comprising scaling each acceleration value with a scaling factor when more than one touched location exists on said display.

23. The system of claim 22, wherein said scaling factor is determined by a distance between touched locations on said display.

36. The system of claim 18, wherein the plurality of acceleration values are consecutively outputted by the accelerometer, and wherein modifying each acceleration value of the plurality of acceleration values according to at least one of a high-frequency filtering function, a low-frequency filtering function, or a slope derivation function comprises applying the at least one of a high-frequency filtering function, a low-frequency filtering function, or a slope derivation function to each of the consecutively outputted plurality of acceleration values.

24. A non-transitory computer-readable storage medium storing instructions which, when executed by a computing device, cause the computing device to perform operations comprising:

receiving a plurality of acceleration values from an accelerometer, the plurality of acceleration values representing touch inputs on a touch-sensitive surface;

modifying each acceleration value of the plurality of acceleration values according to a mathematical function to provide a plurality of modified acceleration values;

selecting a largest modified acceleration value of the plurality of modified acceleration values;

determining a ratio of the largest modified acceleration value and a constant that is greater than the largest modified acceleration value;

associating the determined ratio with an audio function indicated by a touched location on the touch-sensitive display; and

modifying the audio function in accordance with the determined ratio.

25. The non-transitory computer-readable storage medium of claim 24, wherein the audio function is the output of a musical note and modifying the audio function comprises modifying an audible volume of said musical note.

26. The non-transitory computer-readable storage medium of claim 24, wherein the audio function is the output of a musical note and modifying the audio function comprises modifying a pitch of said musical note to create vibrato.

27. The non-transitory computer-readable storage medium of claim 24, wherein the audio function is the output of a musical note and modifying the audio function comprises modifying an attack time of said musical note.

28. The non-transitory computer-readable storage medium of claim 24, the operations further comprising scaling each acceleration value with a scaling factor when more than one touched location exists on said display.

29. The non-transitory computer-readable storage medium of claim 28, wherein said scaling factor is determined by a distance between touched locations on said display.

30. The non-transitory computer-readable storage medium of claim 29, wherein said scaling factor is adjusted in accordance with the position of said touched location on said display.

37. The non-transitory computer-readable storage medium of claim 24, wherein the plurality of acceleration values are consecutively outputted by the accelerometer, and wherein modifying each acceleration value of the plurality of acceleration values according to a mathematical function comprises applying the mathematical function to each of the consecutively outputted plurality of acceleration values.

38. The non-transitory computer-readable storage medium of claim 24, wherein modifying the audio function in accordance with the determined ratio comprises:

assigning the ratio to the audio function such that in response to a touch at the touched location on the touch-sensitive display, the audio function executes to output a maximum audio function value multiplied by the determined ratio.

39. The medium of claim 24, wherein modifying an acceleration value to provide a modified acceleration value comprises:

filtering, based on a previously determined modified acceleration value, low-frequency signals associated with the modified acceleration value;

filtering, based on the previously determined modified acceleration value, high-frequency signals associated with the modified acceleration value; and

determining a ratio of a difference between the modified acceleration value from which the high-frequency signals have been filtered and the modified acceleration value from which the low-frequency signals have been filtered, and a time period between two successive acceleration values outputs by the accelerometer.