Recognition data transmission device

• US 10,699,541 B2
• Filed: 12/08/2014
• Issued: 06/30/2020
• Est. Priority Date: 12/17/2013
• Status: Active Grant

• Current Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Shimura, Tomoya
• Primary Examiner(s): Le, Vu
• Assistant Examiner(s): Mangialaschi, Tracy
• Application Number:
  US15/105,452
• Publication Number:
  US 20160335861A1
• Time in Patent Office: 2,031 Days
• Field of Search: None
• CPC Class Codes: G08B 13/19613 Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion
• CPC Class Codes: G08B 13/19663 Surveillance related processing done local to the camera
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

A feature of a part or the entirety of a body of a person is detected from a captured video; the person in the video is specified from the detected feature; user information indicating physical features of the person is detected from the feature related to the specified person; motion information including a motion or a gesture with a body and hands of a user is detected from the user information and the captured video; a behavior including the motion of the person is recognized from the motion information and the user information; recognized behavior information is divided for each behavior of the person; the divided behavior information is generated as block data; and the generated block data is transmitted to an outside for each block.

Claims

1. A recognition data transmission device, comprising:

an image capturing unit;

a person or animal identifier that detects a feature of a person or an animal from a video captured by the image capturing unit and specifies the person or the animal in the video from the detected feature;

a behavior recognition unit that recognizes a behavior of the person or the animal, and produces recognition information including information identifying the person or the animal and the behavior of the person or the animal, the behavior of the person or the animal being classified as either an urgent behavior or a non-urgent behavior;

a data conversion unit that divides the recognition information output by the behavior recognition unit, for each behavior of the person or the animal on a time axis, and generates at least one block of data for a behavior of the person or the animal, each said block of data storing at least one of the recognition information associated with the behavior or the video captured by the video capturing unit corresponding to the recognition information; and

a transmission control unit that performs control for transmitting the block data, which is output by the data conversion unit, to outside of the transmission device, for each block,

wherein the transmission control unit

performs control for transmitting the block data, which is output by the data conversion unit, to an outside as recognition data for each block including the recognition information,

recognizes, with a time point at which the behavior is finished as a reference, that detection of the behavior is finished, and transmits the block data in order of completion of detection, and performs control such that when the behavior of the person or the animal is considered non-urgent behavior, no image data is transmitted as part of the recognition information and, when the behavior of the person or the animal is considered urgent behavior, image data of the behavior considered urgent is transmitted.

2. The recognition data transmission device according to claim 1, wherein

the transmission control unit sets a higher priority to the recognition information in order of an earlier starting time of a behavior in a case where the priority is the same and the time point at which the behavior is finished is also the same.

3. The recognition data transmission device according to claim 1, wherein

in a case where urgent recognition information identifying behavior considered urgent is detected among the recognition information output by the behavior recognition unit, urgent information indicating urgency is stored in the block data generated for each behavior, and

the transmission control unit transmits the block data having the urgent information before other block data.

4. The recognition data transmission device according to claim 3, wherein

in a case where the urgent recognition information is detected, the behavior recognition unit outputs a video captured by the image capturing unit, which corresponds to the recognition information, and

the transmission control unit transmits the block data storing the video corresponding to the urgent recognition information before other block data excluding block data in which the urgent recognition information is stored.

5. The recognition data transmission device according to claim 4, wherein

the data conversion unit stores information which allows the urgent recognition information and the corresponding video to refer to each other in at least one of the block data storing the recognition information and the block data storing the video.

6. The recognition data transmission device according to claim 1, wherein

the transmission control unit transmits, to an outside, the block data storing only the recognition information output by the behavior recognition unit.

7. The recognition data transmission device according to claim 1 wherein the block data includes video and is transmitted with the behavior information when behavior that is considered urgent is detected.

8. The recognition data transmission device according to claim 1, wherein

the transmission control unit detects the recognition information, in which plural behaviors having overlapping behavior times from start to finish of the behavior are detected, and

when it is possible to predict that a behavior is abnormal among the detected recognition information, the transmission control unit recognizes that the behavior is abnormal at a stage of predicting abnormality before the abnormal behavior is finished, and sets a higher priority to the recognition information indicating the recognized abnormal behavior.

9. A recognition data recording device, including:

an image capturing unit;

a person or animal detection unit that detects a feature of (a part or an entirety of a body of) a person or an animal from a video captured by the image capturing unit and specifies the person or the animal in the video from the detected feature;

a behavior recognition unit that recognizes a behavior of the person or the animal and produces recognition information including information identifying the person or the animal and the behavior of the person or the animal, from the feature related to the person or the animal, which is detected by the person or the animal detection unit;

a data conversion unit that divides the recognition information, output by the behavior recognition unit, for each behavior of the person or the animal on a time axis, and generates divided block data; and

a recording control unit that causes a recording unit to record the block data, output by the data conversion unit, as recognition data, wherein the recording control unit records, with a time point at which the behavior is finished as a reference, that detection of the behavior recognized by the behavior unit is finished, and performs control such that when the behavior of the person or the animal is not considered urgent behavior, block data that stores the video corresponding to the recognition information among the pieces of the block data is not recorded and where the behavior of the person or the animal is considered urgent behavior, image data of the behavior considered urgent is recorded.

10. The recognition data recording device according to claim 9 wherein said recording control unit records behavior information representing the behavior of the person or the animal, and

wherein the block data includes video and is recorded with the behavior information when behavior that is considered urgent is detected.

11. A method of storing recognition data, including:

detecting a feature of a person or an animal from a video captured by at least one image capturing unit and specifying the person or the animal in the video from the detected feature;

recognizing a behavior or movement of the person or the animal specified in said detecting, which includes information related to the person or the animal and a motion of the person or the animal, from the feature and outputting the information related to the person or the animal and their or its behavior as recognition information;

dividing the recognition information, which is output from the recognizing, for each behavior of the person or the animal on a time axis, and generating divided block data; and

transmitting the block data, output as recognition data for each block,

wherein the recognizing recognizes, with a time point at which the behavior is finished as a reference, that detection of the behavior is finished,

wherein said transmitting

transmits the recognition information representing the behavior of the person or the animal in order of completion of detection, and

performs control such that when the behavior of the person or the animal is not considered urgent behavior, block data that stores the video corresponding to the recognition information among the pieces of the block data is not transmitted and where the behavior of the person or the animal is considered urgent behavior, image data of the behavior considered urgent is transmitted.

12. The method of storing according to claim 11 wherein the block data includes video and is transmitted with the behavior information when behavior considered to be urgent is detected.

13. A method of recording recognition data, including:

detecting a feature of (a part or an entirety of a body of) a person or an animal from a video captured by at least one image capturing unit and specifying the person or the animal in the video from the detected feature;

recognizing a behavior of the person or the animal specified in said detecting, which includes information related to the person or the animal and a motion of the person or the animal, from the feature related to the person or the animal, and outputting the information related to the person or the animal and their or its behavior as recognition information;

dividing the recognition information, which is output from the recognizing, for each behavior of the person or the animal on a time axis, and generating divided block data; and

recording the block data, as recognition data, and

performing control such that when the behavior of the person or the animal is not considered urgent behavior, block data that stores the video corresponding to the recognition information among the pieces of the block data is not recorded and where the behavior of the person or the animal is considered urgent behavior, image data of the behavior considered urgent is recorded.

14. The recognition data transmission device according to claim 13 wherein the block data includes video and is transmitted with the behavior information when behavior considered to be urgent is detected.