Surveillance system with intelligent robotic surveillance device

• US 10,671,050 B2
• Filed: 05/04/2018
• Issued: 06/02/2020
• Est. Priority Date: 05/11/2017
• Status: Active Grant

• Current Assignee: Turing Video
• Original Assignee: Turing Video, Inc.
• Inventors: Wang, Fang, Cao, Song
• Primary Examiner(s): Dang, Philip P.
• Application Number:
  US15/971,437
• Publication Number:
  US 20180329392A1
• Time in Patent Office: 760 Days
• CPC Class Codes: G05B 19/406 characterised by monitoring or safety G05B19/19 takes precedence
• CPC Class Codes: G05B 2219/40002 Camera, robot follows direction movement of operator head, helmet, headstick
• CPC Class Codes: G05B 2219/45103 Security, surveillance applications
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/44 Event detection
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: Y10S 901/47 Optical

Abstract

A surveillance system may comprise one or more computing devices and one or more robotic surveillance devices. The one or more computing devices may be configured to obtain video data captured by one or more cameras. The one or more computing devices may analyze the video data to determine whether there is any trigger event. In response to determining that there is a trigger event, the one or more computing device may determine an optimal robotic surveillance device among the one or more robotic surveillance devices based on the trigger event and provide an instruction to the optimal robotic surveillance device. The optimal robotic surveillance device may be configured to perform a responding action in response to receiving the instruction.

Claims

1. A surveillance system, comprising one or more computing devices and one or more robotic surveillance devices, wherein:

the one or more computing devices are configured to;

obtain video data captured by one or more cameras;

analyze the video data to determine whether there is any trigger event;

when there is a trigger event, validate whether or not the trigger event is reliable;

in response to determining that the trigger event is reliable, determine an optimal robotic surveillance device among the one or more robotic surveillance devices based on the trigger event;

determine, based on user devices with which one or more security persons are equipped, and locations of the one or more security persons;

determine, based on the locations of the one or more security persons, whether there is a security person close to the optimal robotic surveillance device;

in response to determining that there is the security person close to the optimal robotic surveillance device, provide an instruction to the security person and the optimal robotic surveillance device to cooperate to perform a responding action; and

in response to determining that there is no security person close to the optimal robotic surveillance device, provide an instruction to the optimal robotic surveillance device to perform a responding action autonomously,

wherein the optimal robotic surveillance device is further configured to predict a future location of a target indicated by the trigger event, wherein the target includes a person, and the optimal robotic surveillance device is further configured to track the target.

2. The surveillance system according to claim 1, wherein the trigger event includes a person identity, an activity, a combination of a person identity and an activity, or a non-human object.

3. The surveillance system according to claim 1, wherein determining the optimal robotic surveillance device comprises:

determining a desired location based on the trigger event; and

determining, among the one or more robotic surveillance devices, a robotic surveillance device that is closest to the desired location as the optimal robotic surveillance device.

4. The surveillance system according to claim 1, wherein the optimal robotic surveillance devices is further configured to, in response to receiving the instruction, plan a traveling path to a desired location indicated by the trigger event.

5. The surveillance system according to claim 1, wherein the responding action comprises one or more of a video or picture recording, a voice alarming, sending a high intensity light, a physical interference, carrying a person, or interacting with a person.

6. The surveillance system according to claim 1, wherein the one or more robotic surveillance devices are selected from one or more of following devices:

a motorized device, a drone, or a platform for transportation.

7. A surveillance method, comprising:

obtaining video data;

analyzing the video data to determine whether there is any trigger event;

when there is a trigger event, validating whether or not the trigger event is reliable;

in response to determining that the trigger event is reliable, determining an optimal robotic surveillance device among one or more robotic surveillance devices based on the trigger event;

determining, based on user devices with which one or more security persons are equipped, and locations of the one or more security persons;

determining, based on the locations of the one or more security persons, whether there is a security person close to the optimal robotic surveillance device;

in response to determining that there is the security person close to the optimal robotic surveillance device, providing an instruction to the security person and the optimal robotic surveillance device to cooperate to perform a responding action;

in response to determining that there is no security person close to the optimal robotic surveillance device, providing an instruction to the optimal robotic surveillance device to perform a responding action autonomously;

predicting a future location of a target indicated by the trigger event by the optimal robotic surveillance device, wherein the target includes a person; and

tracking the target by the optimal robotic surveillance device.

8. The surveillance method according to claim 7, wherein the trigger event includes a person identity, an activity, a combination of a person identity and an activity, or a non-human object.

9. The surveillance method according to claim 7, wherein determining the optimal robotic surveillance device comprises:

determining a desired location based on the trigger event; and

determining, among the one or more robotic surveillance devices, a robotic surveillance device that is closest to the desired location as the optimal robotic surveillance device.

10. The surveillance method according to claim 7, further comprising:

in response to receiving the instruction, planning a traveling path to a desired location indicated by the trigger event by the optimal robotic surveillance device.

11. The surveillance method according to claim 7, wherein the responding action comprises one or more of a video or picture recording, a voice alarming, sending a high intensity light, a physical interference, carrying a person, or interacting with a person.

12. The surveillance method according to claim 7, wherein the one or more robotic surveillance devices are selected from one or more of following devices:

a motorized device, a drone, or a platform for transportation.

13. A robotic surveillance device, the robotic device comprising:

one or more processors; and

a memory storing instructions that, when executed by the one or more processors, cause the robotic surveillance device to perform;

obtaining video data captured by one or more cameras;

analyzing the video data to determine whether there is any trigger event;

when there is a trigger event, validating whether or not the trigger event is reliable;

in response to determining that the trigger event is reliable, determining an optimal robotic surveillance device among one or more robotic surveillance devices based on the trigger event;

determining, based on user devices with which one or more security persons are equipped, and locations of the one or more security persons;

determining, based on the locations of the one or more security persons, whether there is a security person close to the optimal robotic surveillance device;

in response to determining that there is the security person close to the optimal robotic surveillance device, providing an instruction to the security person and the optimal robotic surveillance device to cooperate to perform a responding action;

in response to determining that there is no security person close to the optimal robotic surveillance device, providing an instruction to the optimal robotic surveillance device to perform a responding action autonomously;

predicting a future location of a target indicated by the trigger event by the optimal robotic surveillance device, wherein the target includes a person; and

tracking the target by the optimal robotic surveillance device.