Systems and methods for playing virtual music instrument through tracking of fingers with coded light
First Claim
1. A finger-tracking system comprising:
- a. a projector configured to project a temporal projector light signal, wherein the temporal projector light signal is encoded, for each pixel of the projector, with an information segment comprising the pixel coordinates of the each pixel of the projector;
b. a plurality of light sensors, each of the plurality of light sensors being attached to a finger of a user, wherein each of the plurality of light sensors is configured to detect the temporal projector light signal and generate a sensor signal; and
c. a processing unit operatively coupled to each of the plurality of light sensors and configured to receive the sensor signal from each of the plurality of light sensors, to determine a location information of each finger of the user and to issue a command based on the detected location of at least one finger of the user.
2 Assignments
0 Petitions
Accused Products
Abstract
Finger-tracking systems and methods for virtual instruments playback. The described system tracks the position of user'"'"'s ten fingers on a projection surface and can be used to play virtual instruments such as virtual piano, drums, and bells. The system tracks the movement of user'"'"'s ten fingers while keeping them free of encumbrance or excessive postural constraints. More specifically, in one or more embodiments, a coded light based projector is used to send out location signal onto a flat surface, and ten light sensors are mounted on user'"'"'s fingers to receive these signals and locate user'"'"'s fingers. Based on their locations and relative distance to a fixed point, a printed music instrument can be used for virtual instrument music playback. With the described tracking system, various embodiments of virtual music instruments may be implemented, including a system and method for virtual piano playing as well as virtual Chinese bell playing on printed keyboard and printed Chinese bell set.
-
Citations
20 Claims
-
1. A finger-tracking system comprising:
-
a. a projector configured to project a temporal projector light signal, wherein the temporal projector light signal is encoded, for each pixel of the projector, with an information segment comprising the pixel coordinates of the each pixel of the projector; b. a plurality of light sensors, each of the plurality of light sensors being attached to a finger of a user, wherein each of the plurality of light sensors is configured to detect the temporal projector light signal and generate a sensor signal; and c. a processing unit operatively coupled to each of the plurality of light sensors and configured to receive the sensor signal from each of the plurality of light sensors, to determine a location information of each finger of the user and to issue a command based on the detected location of at least one finger of the user. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15)
-
-
16. A method for tracking fingers of a user, the method comprising:
-
a. using a projector to project a temporal projector light signal, wherein the temporal projector light signal is encoded, for each pixel of the projector, with an information segment comprising the pixel coordinates of the each pixel of the projector; b. detecting the temporal projector light signal using a plurality of light sensors, each of the plurality of light sensors being attached to a finger of a user, wherein each of the plurality of light sensors is configured to detect the temporal projector light signal and generate a sensor signal; and c. using a processing unit operatively coupled to each of the plurality of light sensors to receive the sensor signal from each of the plurality of light sensors, to determine a location information of each finger of the user and to issue a command based on the detected location of at least one finger of the user. - View Dependent Claims (17, 18, 19)
-
-
20. A computer-readable medium embodying a set of instructions implementing a method for tracking fingers of a user, the method comprising:
-
a. using a projector to project a temporal projector light signal, wherein the temporal projector light signal is encoded, for each pixel of the projector, with an information segment comprising the pixel coordinates of the each pixel of the projector; b. detecting the temporal projector light signal using a plurality of light sensors, each of the plurality of light sensors being attached to a finger of a user, wherein each of the plurality of light sensors is configured to detect the temporal projector light signal and generate a sensor signal; and c. using a processing unit operatively coupled to each of the plurality of light sensors to receive the sensor signal from each of the plurality of light sensors, to determine a location information of each finger of the user and to issue a command based on the detected location of at least one finger of the user.
-
Specification