Monitoring system, image capturing apparatus, analysis apparatus, and monitoring method

• US 8,982,208 B2
• Filed: 04/22/2010
• Issued: 03/17/2015
• Est. Priority Date: 05/21/2009
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Takeuchi, Hideto, Hokimoto, Akihiro
• Primary Examiner(s): Lazaro, David
• Assistant Examiner(s): Drabik, Sarah
• Application Number:
  US12/765,390
• Publication Number:
  US 20100295944A1
• Time in Patent Office: 1,790 Days
• Field of Search: 348/143, 382/103, 382115-119, 709201-205
• US Class Current: 348/143
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

Monitoring system is provided which includes an image capturing apparatus including a basic analysis section that performs analysis processing based on image data input from an image capturing section that captures an image of a subject and generates first metadata, and a first metadata output section that outputs the first metadata and second metadata different from the first metadata to a monitoring apparatus connected to a network via the network, and an analysis apparatus including an extended analysis section that performs analysis processing different from that of the basic analysis section based on the image data received from the image capturing apparatus and generates the second metadata, and a second metadata output section that outputs the second metadata to the image capturing apparatus.

Claims

1. A monitoring system comprising:

a plurality of image capturing apparatuses, each image capturing apparatus including

an image capturing circuit that captures an image illustrating a head and a face of a person,

an analysis circuit that analyzes the image captured by the image capturing circuit and generates first metadata, the first metadata including a circumscribed rectangle according to moving object detection processing on the image, and

a first metadata output circuit that outputs the first metadata and second metadata to a monitoring apparatus via a network, the second metadata being different from the first metadata; and

an analysis apparatus including

a plurality of extended analysis circuits that each analyzes an image captured by a different one of the image capturing apparatuses to generate the corresponding second metadata that specifies a head orientation of the head and positions of facial features of the face in at least one of the circumscribed rectangle and the analyzed image, the head orientation being information of a roll, a pitch or a yaw of the head within the analyzed image,

a metadata combining circuit that generates combined metadata by combining the second metadata generated by a first extended analysis circuit for a first image capturing apparatus of the plurality of image capturing apparatuses and the second metadata generated by a second extended analysis circuit for a second image capturing apparatus of the plurality of image capturing apparatuses, and

a second metadata output circuit that outputs the second metadata to each of the image capturing apparatuses and outputs the combined metadata to the one of the image capturing apparatuses.

2. The monitoring system according to claim 1, wherein the image capturing apparatus and the analysis apparatus exchange capability information according to a simple object access protocol (SOAP).

3. The monitoring system according to claim 1, wherein the image capturing apparatus and the analysis apparatus exchange capability information prior to generating at least one of the first metadata and the second metadata.

4. The monitoring system according to claim 1, wherein

each of the plurality of image capturing apparatuses captures a different view of the person, and each of the plurality of image capturing apparatuses generates additional metadata, that is different from the first and second metadata, based on the different views of the person.

5. The monitoring system according to claim 4, wherein

the different views of the person from each of the plurality of image capturing apparatuses are displayed concurrently in a camera image panel, and

the additional metadata from each of the plurality of image capturing devices are displayed in an alert list panel.

6. The monitoring system according to claim 4, wherein the additional metadata specifies a name of the person.

7. An image capturing apparatus comprising:

an image capturing circuit that captures an image illustrating a head and a face of a person;

an analysis circuit that analyzes the image captured by the image capturing circuit and generates first metadata, the first metadata including a circumscribed rectangle according to moving object detection processing performed on the image;

a first metadata output circuit that outputs the first metadata and second metadata to a monitoring apparatus via a network, wherein

the second metadata is generated in an extended analysis circuit in an analysis apparatus, the second metadata including information of a head orientation of the head and positions of facial features of the face in at least one of the circumscribed rectangle and the image, the head orientation being information of a roll, a pitch or a yaw of the head within the analyzed image; and

a receiving circuit that receives combined metadata from the analysis apparatus,

wherein the combined metadata is generated at the analysis apparatus by combining the generated second metadata with second metadata generated at the analysis apparatus for one or more other image capturing apparatuses,

wherein the second metadata for the one or more other image capturing apparatuses specifies a head orientation of the head of the person.

8. The image capturing apparatus according to claim 7, further comprising:

a metadata combining circuit that combines the first metadata and the second metadata to generate second combined metadata, wherein

the first metadata output section outputs the second combined metadata to the monitoring apparatus.

9. An analysis apparatus comprising:

an extended analysis circuit that

receives image data and first metadata from a plurality of image capturing apparatuses, the first metadata including a circumscribed rectangle according to moving object detection processing on the image data, and

analyzes an image captured by one of the image capturing apparatuses to generate second metadata that specifies a head orientation of the head and positions of facial features of the face in at least one of the circumscribed rectangle and the analyzed image, the head orientation being information of a roll, a pitch or a yaw of the head within the analyzed image;

a metadata combining circuit that generates combined metadata by combining the second metadata generated for the one image capturing apparatus and other second metadata generated for another image capturing apparatus of the plurality of image capturing apparatuses; and

a metadata output circuit that outputs the second metadata to each of the image capturing apparatuses and outputs the combined metadata to one of the image capturing apparatuses.

10. The analysis apparatus according to claim 9, wherein

the extended analysis circuit is one of a plurality of extended analysis circuits included in the analysis apparatus,

each extended analysis circuit of the plurality of extended analysis circuits analyzes an image captured by a different one of the image capturing apparatuses to generate corresponding second metadata, and

the metadata combining circuit combines the second metadata generated by a first extended analysis circuit of the plurality of extended analysis circuits with the second metadata generated by a second extended analysis circuit of the plurality of extended analysis circuits.

11. A monitoring method comprising:

analyzing, by an image capturing apparatus, an image captured by an image capturing circuit of the image capturing apparatus, the image illustrating a head and a face of a person, and generating first metadata;

generating, by the image capturing apparatus, first metadata, the first metadata including a circumscribed rectangle according to moving object detection processing on the image;

analyzing, by an analysis apparatus, the image captured by the image capturing circuit of the image capturing apparatus;

generating, by the analysis apparatus, second metadata that specifies a head orientation of the head and positions of facial features of the face in at least one of the circumscribed rectangle and the analyzed image, the head orientation being information of a roll, a pitch or a yaw of the head within the analyzed image;

generating combined metadata by combining the second metadata and other second metadata, the other second metadata generated for another image capturing apparatus;

outputting, by the analysis apparatus, the second metadata and the combined metadata to the image capturing apparatus;

outputting, by the image capturing apparatus, the first metadata and the second metadata to a monitoring apparatus via a network.

12. The monitoring method according to claim 11, further comprising:

tracking the person according to the images of a plurality of image capturing apparatuses;

generating in real-time, by at least one of the plurality of image capturing apparatuses, additional metadata that is different from the first and second metadata, based on the tracking; and

displaying views of the tracked person obtained by the plurality of image capturing apparatuses with the additional metadata.

13. The monitoring method according to claim 12, wherein the additional metadata specifies license plate information corresponding to the person.