Method and apparatus for detecting abnormal situation, and non-transitory computer-readable recording medium

• US 10,438,069 B2
• Filed: 03/03/2016
• Issued: 10/08/2019
• Est. Priority Date: 03/09/2015
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Fan, Shengyin, Cheng, Zhongwei, Wang, Xin, Wang, Qian, Qiao, Gang
• Primary Examiner(s): Patel, Jayanti K
• Assistant Examiner(s): Habib, Irfan
• Application Number:
  US15/059,787
• Publication Number:
  US 20160267328A1
• Time in Patent Office: 1,314 Days
• CPC Class Codes: G06F 18/23213 with fixed number of clusters, e.g. K-means clustering
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 2201/12 Acquisition of 3D measurements of objects

Abstract

A method and an apparatus for detecting an abnormal situation are disclosed. The method includes detecting whether a first target exists in an obtained image; recognizing whether the first target holds an object, when the first target exists in the image; obtaining motion information of the object, when the first target holds the object; and determining, based on the motion information of the object, whether the abnormal situation exists.

Claims

1. A method for detecting an abnormal situation, the method comprising:

detecting whether a first target exists in an obtained image;

recognizing whether the first target holds an object, when the first target exists in the image;

obtaining motion information of the object, when the first target holds the object;

determining, based on the motion information of the object, whether the abnormal situation exists;

obtaining motion information of the first target, when the first target holds the object; and

determining, based on the motion information of the first target, whether the abnormal situation exists, wherein obtaining the motion information of the object and the motion information of the first target includes

generating a depth image of the obtained image,

projecting the first target and the object into a three-dimensional space using the depth image to obtain three-dimensional motion information of the first target and the object in the three-dimensional space, and

projecting the three-dimensional motion information of the first target and the object into a bird'"'"'s-eve view to obtain bird'"'"'s-eye motion information of the first target and the object, wherein determining whether the abnormal situation exists based on the motion information of the object includes

determining, based on the bird'"'"'s-eve motion information of the object, whether the abnormal situation exists, and

wherein determining whether the abnormal situation exists based on the motion information of the first target includes

determining, based on the bird'"'"'s-eve motion information of the first target, whether the abnormal situation exists.

2. The method for detecting an abnormal situation according to claim 1,

wherein the motion information of the object includes at least one of a motion speed of the object, a motion direction of the object, and a method for using the object by the first target.

3. The method for detecting an abnormal situation according to claim 1,

wherein obtaining the motion information of the object includes

obtaining two-dimensional motion information of the object,

generating a depth image of the obtained image, and

projecting the two-dimensional motion information of the object into a three-dimensional space using the depth image to generate three-dimensional motion information of the object, and

wherein determining whether the abnormal situation exists based on the motion information of the object includes

determining, based on the three-dimensional motion information of the object, whether the abnormal situation exists.

4. The method for detecting an abnormal situation according to claim 1, the method further comprising:

determining, in the obtained image, based on the motion information of the first target, whether the first target moves toward a second target; and

determining, based on a distance between the first target and the second target, whether the abnormal situation exists, when the first target moves toward the second target.

10. The method for detecting an abnormal situation according to claim 1, wherein the first target is a person and the object is an implement held by the person.

5. An apparatus for detecting an abnormal situation, the apparatus comprising:

a camera configured to detect whether a first target exists in an obtained image;

a memory storing computer-readable instructions; and

one or more processors configured to execute the computer-readable instructions such that the one or more processors are configured to perform operations including,

recognizing whether the first target holds an object, when the first target exists in the image;

obtaining motion information of the object, when the first target holds the object;

determining, based on the motion information of the object, whether the abnormal situation exists;

obtaining motion information of the first target, when the first target holds the object; and

determining, based on the motion information of the first target, whether the abnormal situation exists, wherein obtaining the motion information of the object and the motion information of the first target includes

generating a depth image of the obtained image,

projecting the first target and the object into a three-dimensional space using the depth image to obtain three-dimensional motion information of the first target and the object in the three-dimensional space, and

projecting the three-dimensional motion information of the first target and the object into a bird'"'"'s-eve view to obtain bird'"'"'s-eye motion information of the first target and the object, wherein determining whether the abnormal situation exists based on the motion information of the object includes

determining, based on the bird'"'"'s-eye motion information of the object, whether the abnormal situation exists, and

wherein determining whether the abnormal situation exists based on the motion information of the first target includes

determining, based on the bird'"'"'s-eve motion information of the first target, whether the abnormal situation exists.

6. The apparatus for detecting an abnormal situation according to claim 5,

wherein the one or more processors are configured to

obtain two-dimensional motion information of the object,

generate a depth image of the obtained image, and

project the two-dimensional motion information of the object into a three-dimensional space using the depth image to generate three-dimensional motion information of the object, and

wherein the one or more processors determine, based on the three-dimensional motion information of the object, whether the abnormal situation exists.

7. The apparatus for detecting an abnormal situation according to claim 5, wherein the one or more processors are further configured to

obtain motion information of the first target, when the first target holds the object, and

determine, based on the motion information of the first target, whether the abnormal situation exists.

8. The apparatus for detecting an abnormal situation according to claim 7, wherein the one or more processors are further configured to

determine, in the obtained image, based on the motion information of the first target, whether the first target moves toward a second target,

and determine, based on a distance between the first target and the second target, whether the abnormal situation exists, when the first target moves toward the second target.

9. A non-transitory computer-readable recording medium having stored therein a program for causing a computer to execute a process for detecting an abnormal situation, the process comprising:

detecting whether a first target exists in an obtained image;

recognizing whether the first target holds an object, when the first target exists in the image;

obtaining motion information of the object, when the first target holds the object;

determining, based on the motion information of the object, whether the abnormal situation exists;

obtaining motion information of the first target, when the first target holds the object; and

determining, based on the motion information of the first target, whether the abnormal situation exists, wherein obtaining the motion information of the object and the motion information of the first target includes

generating a depth image of the obtained image,

projecting the first target and the object into a three-dimensional space using the depth image to obtain three-dimensional motion information of the first target and the object in the three-dimensional space, and

projecting the three-dimensional motion information of the first target and the object into a bird'"'"'s-eye view to obtain bird'"'"'s-eve motion information of the first target and the object, wherein determining whether the abnormal situation exists based on the motion information of the object includes

determining, based on the bird'"'"'s-eye motion information of the object, whether the abnormal situation exists, and

wherein determining whether the abnormal situation exists based on the motion information of the first target includes

determining, based on the bird'"'"'s-eve motion information of the first target, whether the abnormal situation exists.