Action recognition apparatus and apparatus for recognizing attitude of object

• US 20050084141A1
• Filed: 08/27/2004
• Published: 04/21/2005
• Est. Priority Date: 08/29/2003
• Status: Active Grant

• Current Assignee: Fuji Xerox Company Limited (Xerox Holdings Corp.)
• Original Assignee: Fuji Xerox Company Limited (Xerox Holdings Corp.)
• Inventors: Ikeda, Hitoshi, Maeda, Masahiro, Kato, Noriji
• Granted Patent: US 7,734,062 B2
• Time in Patent Office: Days
• US Class Current: 382/118
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching

Abstract

An action recognition apparatus acquires image data and outputs action information representing an action content of a human captured in the image data. The action recognition apparatus includes a face-portion specifying section, an acquisition section, a specifying section, and an information output section. The face-portion specifying section specifies a first image region including a face portion of the human from the image data. The acquisition section acquires skin-color information peculiar to the human from pixel values of the first image region. The specifying section specifies at least one second image region including a predetermined portion of a body of the human from the image data except for the first image region based on the skin-color information. The information output section outputs the action information representing the action content of the human based on position relation between the specified first image region and the specified second image region.

Claims

1. An action recognition apparatus that acquires image data generated by capturing an image including a face of a human and outputs action information representing an action content of the human captured in the image data, the action recognition apparatus comprising:

a face-portion specifying section that specifies a first image region including a face portion of the human from the image data;

an acquisition section that acquires skin-color information peculiar to the human from pixel values of the first image region;

a specifying section that specifies at least one second image region including a predetermined portion of a body of the human from the image data except for the first image region based on the skin-color information; and

an information output section that outputs the action information representing the action content of the human based on position relation between the specified first image region and the specified second image region.

2. The action recognition apparatus according to claim 1, further comprising:

a holding section that uses a model image obtained by projecting a three-dimensional model of a human body onto a virtual second-dimensional plane to hold plural pieces of action information representing action contents, each of which is associated with position information representing a position corresponding to the head portion in the model image and a position corresponding to the predetermined portion in the model image, wherein;

the information output section compares the plural pieces of position information held by the holding section with information representing a position of the specified first image region and a position of the specified second image region to obtain and output one of the plural pieces of action information, which is associated with the position information.

3. The action recognition apparatus according to claim 1, wherein:

the face-portion specifying section outputs face-direction information that specifies a three-dimensional direction of the face of the human from the image data; and

the information output section outputs the action information representing the action content of the human based on the position relation between the specified first image region and the specified second image region and the face-direction information.

4. The action recognition apparatus according to claim 3, further comprising:

a holding section that uses a model image obtained by projecting a three-dimensional model of a human body onto a virtual second-dimensional plane to hold plural pieces of action information representing action contents, each of which is associated with;

position information representing a position corresponding to the head portion in the model image and a position corresponding to the predetermined portion in the model image; and

face-direction information representing a direction of the face portion of the model image, wherein;

the face-portion specifying section outputs a face direction of the face portion in the specified first image region; and

the information output section compares the plural pieces of position information held by the holding section with information representing a position of the specified first image region and a position of the specified second image region and compares the plural pieces of face-direction information held by the holding section with the face-direction information output from the face-portion specifying section to obtain and output one of the plural pieces of action information, which is associated with the position information.

5. The action recognition apparatus according to claim 1, further comprising:

a conversion section that converts the image data into a gray-scale image data and outputs the gray-scale image data to the face-portion specifying section, wherein;

the face-portion specifying section specifies the first image region based on the gray-scale image data.

6. The action recognition apparatus according to claim 1, further comprising:

a conversion section that converts the image data into a hue image data and outputs the hue image data to the acquisition section, wherein;

the acquisition section acquires skin-color information peculiar to the human from the first image region in the hue image data.

7. The action recognition apparatus according to claim 1, wherein:

the face-portion specifying section stores a normal mage of a face in advance;

the face-portion specifying section normalizes the image data and judges based on the normalized image data and the stored normal image whether or not the image data includes the face portion of the human; and

when the face-portion specifying section judges that the image data includes the face portion of the human, the face-portion specifying section outputs the first image region including the face portion of the human.

8. The action recognition apparatus according to claim 1, wherein the predetermined portion of the body of the human includes a hand of the human.

9. An apparatus for recognizing attitude of an object, comprising:

a reception section that acquires image data generated by capturing at least a part of the object;

a first-portion specifying section that specifies a first image region including a predetermined characteristic portion from the image data;

an acquisition section that acquires attribute information concerning appearance of the object from pixel values of the first image region;

a specifying section that specifies at least one second image region including an image portion showing a part of the object from the image data except for the first image region based on the attribute information; and

an information output section that outputs information concerning attitude of the object based on position relation between the specified first image region and the specified second image region.

10. An action recognition method using a computer for acquiring image data generated by capturing an image including a face of a human and outputting action information representing an action content of the human captured in the image data, the action recognition method comprising:

specifying a first image region including a face portion of the human from the image data;

acquiring skin-color information peculiar to the human from pixel values of the first image region;

specifying at least one second image region including a predetermined portion of a body of the human from the image data except for the first image region based on the skin-color information; and

outputting the action information representing the action content of the human based on position relation between the specified first image region and the specified second image region.

11. An action recognition program that causes a computer to acquire image data generated by capturing an Image Including a face of a human and output action information representing an action content of the human captured in the image data, the action recognition program causing the computer to execute a process comprising:

specifying a first image region including a face portion of the human from the image data;

acquiring skin-color information peculiar to the human from pixel values of the first image region;

specifying at least one second image region including a predetermined portion of a body of the human from the image data except for the first image region based on the skin-color information; and

outputting the action information representing the action content of the human based on position relation between the specified first image region and the specified second image region.