Method and apparatus for real-time gesture recognition
First Claim
1. A computer-implemented method of storing and recognizing an aspect of a subject within an image, the method including:
- a) building a background model by obtaining at least one frame of an image;
b) obtaining a data frame containing a subject;
c) removing background for said data frame based on said background model whereby the subject is isolated;
d) analyzing the data frame thereby determining particular coordinates of the subject at a particular time while the subject is moving;
e) adding the particular coordinates to a frame data set; and
f) examining the particular coordinates such that the particular coordinates are compared to positional data making up a plurality of recognizable aspects, wherein a recognizable aspect is made up of at least one dimension such that the positional data describes dimensions of the recognized aspect.
6 Assignments
0 Petitions
Accused Products
Abstract
A system and method are disclosed for providing a gesture recognition system for recognizing gestures made by a moving subject within an image and performing an operation based on the semantic meaning of the gesture. A subject, such as a human being, enters the viewing field of a camera connected to a computer and performs a gesture, such as flapping of the arms. The gesture is then examined by the system one image frame at a time. Positional data is derived from the input frames and compared to data representing gestures already known to the system. The comparisons are done in real-time and the system can be trained to better recognize known gestures or to recognize new gestures. A frame of the input image containing the subject is obtained after a background image model has been created. An input frame is used to derive a frame data set that contains particular coordinates of the subject at a given moment in time. This series of frame data sets is examined to determine whether it conveys a gesture that is known to the system. If the subject gesture is recognizable to the system, an operation based on the semantic meaning of the gesture can be performed by a computer.
-
Citations
22 Claims
-
1. A computer-implemented method of storing and recognizing an aspect of a subject within an image, the method including:
-
a) building a background model by obtaining at least one frame of an image;
b) obtaining a data frame containing a subject;
c) removing background for said data frame based on said background model whereby the subject is isolated;
d) analyzing the data frame thereby determining particular coordinates of the subject at a particular time while the subject is moving;
e) adding the particular coordinates to a frame data set; and
f) examining the particular coordinates such that the particular coordinates are compared to positional data making up a plurality of recognizable aspects, wherein a recognizable aspect is made up of at least one dimension such that the positional data describes dimensions of the recognized aspect. - View Dependent Claims (2, 3, 4, 5)
e) repeating a through d for a plurality of data frames; and
f) determining whether the plurality of the data frames when examined in a particular sequence, conveys a subject aspect by the subject that resembles a recognizable aspect, thereby causing an operation based on a predetermined meaning of the recognizable aspect be performed by a computer.
-
-
6. A computer program embodied on a computer-readable medium that stores and recognizes an aspect of a subject within an image, including:
-
a) a code segment that builds a background model by obtaining at least one frame of an image;
b) a code segment that obtains a data frame containing a subject;
c) a code segment that subtracts background for said data frame based on said background model;
d) a code segment that analyzes the data frame thereby determining particular coordinates of the subject at a particular time while the subject is moving;
e) a code segment that adds the particular coordinates to a frame data set; and
f) a code segment that examines the particular coordinates such that the particular coordinates are compared to positional data making up a plurality of recognizable aspects, wherein a recognizable aspect is made up of at least one dimension such that the positional data describes dimensions of the recognized aspect. - View Dependent Claims (7, 8, 9, 10)
e) a code segment that repeats a through d for a plurality of data frames; and
f) a code segment that determines whether the plurality of the data frames when examined in a particular sequence, conveys a subject aspect by the subject that resembles a recognizable aspect, thereby causing an operation based on a predetermined meaning of the recognizable aspect be performed by a computer.
-
-
11. A computer-implemented method of storing and recognizing aspect of a subject within an image, the method including:
-
building a background model by obtaining at least one frame of an image;
storing a plurality of samples of a subject;
subtracting background for said data frame based on said background model;
inputting a number of key points that fit in an aspect of the subject;
inputting a corresponding time value representing the time of the aspect of the subject to complete; and
wherein the key points and corresponding time value are adapted for being used to recognize aspect of the subject. - View Dependent Claims (12, 13, 14, 15, 16)
determining locations of key points in a model representative of the aspect of the subject.
-
-
13. A method as recited in claim 12 further including:
refining the model such that the plurality of samples of the aspect of the subject fit within the model.
-
14. A method as recited in claim 11 further including:
calculating a probability distribution for key points indicating the likelihood that a certain output will be observed.
-
15. A method as recited in claim 14 further including:
calculating a confusion matrix wherein the aspect of the subject is compared with previously stored recognizable aspects so that similarities between the new aspect to previously stored recognizable aspect can be determined.
-
16. A method as recited in claim 11 further including:
inputting a number of dimensions of the aspect of the subject.
-
17. A computer program embodied on a computer-readable medium that stores and recognizes an aspect of a subject within an image, including:
-
a code segment that builds a background model by obtaining at least one frame of an image;
a code segment that stores a plurality of samples of a subject;
a code segment that subtracts background for said data frame based on said background model;
a code segment that inputs a number of key points that fit in an aspect of the subject;
a code segment that inputs a corresponding time value representing the time of the aspect of the subject to complete; and
wherein the key points and corresponding time value are adapted for being used to recognize aspect of the subject. - View Dependent Claims (18, 19, 20, 21, 22)
a code segment that determines locations of key points in a model representative of the aspect of the subject.
-
-
19. A computer program embodied on a computer-readable medium as recited in claim 18 further including:
a code segment that refines the model such that the plurality of samples of the aspect of the subject fit within the model.
-
20. A computer program embodied on a computer-readable medium as recited in claim 17 further including:
a code segment that calculates a probability distribution for key points indicating the likelihood that a certain output will be observed.
-
21. A computer program embodied on a computer-readable medium as recited in claim 20 further including:
a code segment that calculates a confusion matrix wherein the aspect of the subject is compared with previously stored recognizable aspects so that similarities between the new aspect to previously stored recognizable aspect can be determined.
-
22. A computer program embodied on a computer-readable medium as recited in claim 17 further including:
a code segment that inputs a number of dimensions of the aspect of the subject.
Specification