OBJECT TRACKING SYSTEM, METHOD AND SMART NODE USING ACTIVE CAMERA HANDOFF

• US 20100157064A1
• Filed: 08/20/2009
• Published: 06/24/2010
• Est. Priority Date: 12/18/2008
• Status: Active Grant

• Current Assignee: Industrial Technology Research Institute
• Original Assignee: Industrial Technology Research Institute
• Inventors: Kuo, Jen-Yau, Cheng, Sung-Ping, Chuang, Jen-Hui, Jang, Lih-Guong
• Granted Patent: US 8,218,011 B2
• Time in Patent Office: Days
• US Class Current: 348/169
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08B 13/19645 Multiple cameras, each having view on one of a plurality of scenes, e.g. multiple cameras for multi-room surveillance or for tracking an object by view hand-over
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

If an active smart node detects that an object leaves a center region of a FOV for a boundary region, the active smart node predicts a possible path of the object. When the object gets out of the FOV, the active smart node predicts the object appears in a FOV of another smart node according to the possible path and a spatial relation between cameras. The active smart node notifies another smart node to become a semi-active smart node which determines an image characteristic similarity between the object and a new object and returns to the active smart node if a condition is satisfied. The active smart node compares the returned characteristic similarity, an object discovery time at the semi-active smart node, and a distance between the active smart node and the semi-active smart node to calculate possibility.

Claims

1. A object tracking method using a camera handoff mechanism, the object tracking method applied in an intelligent distributed object tracking system comprising a control center, a plurality of smart nodes and a plurality of cameras, the smart nodes managing the cameras, the object tracking method comprising steps of:

detecting, by a first smart node of the smart nodes, a position of a object to judge whether the object leaves a center region of a first field of view (FOV) of the first smart node for a boundary region when the first smart node is tracking the object;

predicting, by the first smart node, a possible path of the object when the object enters the boundary region of the first FOV of the first smart node;

predicting, by the first smart node, that the object will appear in a second FOV of a second smart node of the smart nodes according to the possible path of the object and a spatial relation between the cameras when the object gets out of the first FOV of the first smart node;

notifying, by the first smart node, the second smart node to detect a first newly entered object inside the second FOV;

determining, by the second smart node, a first image characteristic similarity between the object and the first newly entered object, and returning the first image characteristic similarity to the first smart node;

deciding, by the first smart node, whether to make handoff transfer to the second smart node according to the first image characteristic similarity returned by the second smart node; and

if yes, making handoff transfer, by the first smart node, to the second smart node so that the second smart node takes over tracking on the object.

2. The object tracking method according to claim 1, wherein the image characteristic similarity comprises at least one of a size, a moving speed, a moving direction, a moving track, a color characteristic of the object.

3. The object tracking method according to claim 1, wherein the step of predicting, by the first smart node, that the object will appear in the second FOV of the second smart node of the smart nodes according to the possible path of the object and the spatial relation between the cameras when the object gets out of the first FOV of the first smart node comprises:

judging by the first smart node, whether the possible path of the object is covered by the second FOV of the second smart node to predict whether the object appears in the second FOV of the second smart node.

4. The object tracking method according to claim 1, wherein in the step of notifying by the first smart node, the second smart node to detect a first newly entered object inside the second FOV, the first smart node notifies to the second smart node contents comprising:

a serial number of the object, a serial number of the first smart node, a serial number of a handoff control program of the first smart node, an image characteristic of the object, a predicted appearance time of the object, and a handoff timing limit control parameter.

5. The object tracking method according to claim 1, wherein the step of deciding, by the first smart node, whether to make handoff transfer to the second smart node according to the first image characteristic similarity returned by the second smart node comprises:

judging, by the first smart node, that the second smart node has successfully tracked the object if the first image characteristic similarity returned by the second smart node is higher than a threshold value; and

judging, by the first smart node, that the second smart node has not yet tracked the object, and executing an object loss retrieving operation by the first smart node if the first image characteristic similarity returned by the second smart node is lower than the threshold value.

6. The object tracking method according to claim 5, wherein the step of executing, by the first smart node, the object loss retrieving operation comprises:

notifying, by the first smart node, at least one third smart node of the smart nodes to detect a second newly entered object inside a third FOV thereof;

determining, by the third smart node, a second image characteristic similarity between the object and the second newly entered object, and returning the second image characteristic similarity to the first smart node;

judging, by the first smart node, that the third smart node has successfully tracked the object if the second image characteristic similarity returned by the third smart node is higher than the threshold value, and notifying, by the first smart node, the third smart node for handoff transfer; and

judging, by the first smart node, that the third smart node has not yet tracked the object if the second image characteristic similarity returned by the third smart node is lower than the threshold value, and notifying, by the first smart node, the control center to re-designate a track on the object.

7. The object tracking method according to claim 1, wherein deciding, by the first smart node, whether to make handoff transfer to the second smart node comprises:

deciding, by the first smart node, according to the first image of the characteristics of similarity, an object discovery time, and the distance between first smart node and second smart node returned from the second smart node, whether or not to handoff to the second smart node;

or deciding, by the first smart node, whether to make handoff transfer to the second smart node according to a lookup table.

8. The object tracking method according to claim 1, wherein the step of making handoff transfer, by the first smart node, to the second smart node so that the second smart node takes over tracking on the object further comprises:

transmitting, by the first smart node, to the second smart node messages comprise a serial number of the object, an image characteristic of the object and a monitor metadata of the object, when the first smart node makes handoff transfer to the second smart node.

9. The object tracking method according to claim 8, wherein the monitor metadata of the object comprises at least one of a threat level and a handoff timing limit parameter.

10. The object tracking method according to claim 1, wherein when the first smart node makes handoff transfer to the second smart node:

the second smart node again judges whether the object is still inside the second FOV;

if the second smart node judges that the object is still inside the second FOV, the second smart node replies to the first smart node to make handoff transfer; and

if the second smart node judges that the object is not inside the second FOV, the second smart node replies to the first smart node and the first smart node performs a object loss retrieving operation.

11. An intelligent distributed object tracking system using a camera handoff mechanism, the system comprising:

a control center;

a plurality of smart nodes communicating with the control center; and

a plurality of cameras controlled by the smart nodes, wherein;

when a first smart node of the smart nodes is tracking an object, the first smart node detects a position of the object to judge whether the object leaves a center region of a first FOV of the first smart node for a boundary region of the first FOV to predict a possible path of the object;

when the object gets out of the first FOV of the first smart node, the first smart node predicts that the object appears inside a second FOV of a second smart node of the smart nodes according to the possible path of the object and a spatial relation between the cameras;

the first smart node notifies the second smart node to detect a first newly entered object inside the second FOV;

the second smart node determines a first image characteristic similarity between the object and the first newly entered object, and returns to the first smart node;

the first smart node decides whether to make handoff transfer to the second smart node according to the first image characteristic similarity, an object discovery time and the distance between the smart node of these data returned by the second smart node; and

if the first smart node decides to make handoff transfer to the second smart node, the second smart node takes over tracking on the object.

12. The system according to claim 11, wherein the first smart node predicts the possible path of the object when the object enters into the boundary region of the first FOV of the first smart node.

13. The system according to claim 11, wherein each of the smart nodes comprises:

an image processing module for obtaining an image characteristic of the object;

an object tracking module, coupled to the image processing module, for designating a serial number to the object, receiving an object tracking enablement command from the control center, periodically returning a label information and a position information of the object under track to the control center, and performing FOV boundary detection and image characteristic comparison;

an alert detection module, coupled to the object tracking module, for enabling a detection function according to a monitor metadata of the object, wherein when an object triggering alert is the object tracked by the first smart node, the alert detection module outputs alertness, the alert detection module decides an alert level according to the monitor metadata of the object, and the alert detection module transmits an alert message to the control center; and

a handoff operation control module, coupled to the alert detection module and the object tracking module, for making handoff transfer and enabling a dedicated handoff control program for the object, wherein a state of the handoff control program relates to a handoff operation performance condition, and the handoff operation control module transmits the monitor metadata of the object to the alert detection module.

14. The system according to claim 13, wherein the handoff operation control module comprises:

an object position tracking unit for judging whether the object is inside a center region of a FOV or inside a boundary region of the FOV or whether the object is outside the FOV according to an object position obtained by the image processing module;

a moving path predicting unit for calculating the possible path of the object according to movement of the object inside the FOV and judging that the possible path of the object is covered by the second FOV of the second smart node;

a semi-active smart node prediction unit for deciding to notify the second smart node to prepare for a handoff transfer according to a predicted result of the moving path predicting unit;

a new object similarity comparing unit for generating an image characteristic similarity;

a handoff transfer smart node predicting unit for predicting that the first smart node makes handoff transfer to the second smart node according to the image characteristic similarity, an object discovery time and the distance between the smart node;

an object loss retrieving unit for performing an object loss retrieving operation;

a handoff process message addressing and communicating unit for encoding and decoding a message appeared in a handoff operation;

a handoff control program managing unit for controlling each handoff control program in the smart node and a state thereof;

a module interacting interface communicating with other modules in the smart node; and

a handoff timing limit control unit for providing timing limit management and control for handoff.

15. The system according to claim 14, wherein when the object loss retrieving operation is performed,

the first smart node further notifies at least one third smart node of the smart nodes to detect a second newly entered object inside its third FOV;

the third smart node determines a second image characteristic similarity between the object and the second newly entered object and returns to the first smart node;

if the second image characteristic similarity returned by the third smart node is higher than a threshold value, the first smart node judges that the third smart node has successfully tracked the object, and the first smart node notifies the third smart node to prepare for handoff transfer; and

if the second image characteristic similarity returned by the third smart node is lower than the threshold value, the first smart node judges that the third smart node has not yet tracked the object, and the first smart node notifies the control center to re-designate tracking on the object.

16. The system according to claim 11, wherein when the first smart node makes handoff transfer to the second smart node, the first smart node transmits to the second smart node messages comprising a serial number of the object, an image characteristic of the object and a monitor metadata of the object.

17. The system according to claim 16, wherein the monitor metadata of the object comprises at least one of a threat level and a handoff timing limit parameter.

18. The system according to claim 11, wherein when the first smart node makes handoff transfer to the second smart node:

the second smart node again judges whether the object is still inside the second FOV;

if the second smart node judges that the object is still inside the second FOV, the second smart node returns to the first smart node for handoff transfer; and

if the second smart node judges that the object is not inside the second FOV, the second smart node returns to the first smart node, and the first smart node performs an object loss retrieving operation.

19. A smart node using a camera handoff mechanism, applied in an intelligent distributed object tracking system for tracking at least one object, the intelligent distributed object tracking system comprising a control center and a plurality of cameras, the smart node comprising:

an image processing module for obtaining an image characteristic of the object;

an object tracking module, coupled to the image processing module, for designating a serial number to the object, receiving an object tracking enabling command, periodically returning a label information and a position information of the object under track to the control center, and performing FOV boundary detection and image characteristic comparison;

an alert detection module, coupled to the object tracking module, for enabling a detection function according to a monitor metadata of the object, wherein when an object triggering alert is the object tracked by the smart node, the alert detection module outputs alertness, the alert detection module decides an alert level according to the monitor metadata of the object, and the alert detection module transmits an alert message to the control center; and

a handoff operation control module, coupled to the alert detection module and the object tracking module, for making handoff transfer and enabling a dedicated handoff control program for the object, wherein a state of the handoff control program relates to a handoff operation performing condition, and the handoff operation control module transmits the monitor metadata of the object to the alert detection module.

20. The smart node according to claim 19, wherein the handoff operation control module comprises:

an object position tracking unit for judging whether the object is inside a center region of a FOV or inside a boundary region of the FOV or whether the object is outside the FOV according to an object position obtained by the image processing module;

a moving path predicting unit for calculating a possible path of the object according to movement of the object inside the FOV and judging that the possible path of the object is covered by a FOV of other smart nodes in the intelligent distributed object tracking system;

a semi-active smart node prediction unit for deciding whether to let other smart nodes in the intelligent distributed object tracking system to prepare for handoff transfer according to a predicted result of the moving path predicting unit;

a new object similarity comparing unit for generating an image characteristic similarity;

a handoff transfer smart node predicting unit for calculating whether to make handoff transfer according to the image characteristic similarity, an object discovery time and the distance between the smart node;

an object loss retrieving unit for performing an object loss retrieving operation;

a handoff process message addressing and communicating unit for encoding and decoding a message appeared in a handoff operation;

a handoff control program managing unit for controlling each handoff control program in the smart node and a state thereof;

a module interacting interface communicating with other modules in the smart node; and

a handoff timing limit control unit for providing timing limit management and control for handoff.

21. The smart node according to claim 19, wherein when the object loss retrieving operation is performed:

the smart node further notifies other smart nodes in the intelligent distributed object tracking system to detect a newly entered object inside its FOV;

other smart nodes determine an image characteristic similarity between the object and the newly entered object and return to the smart node;

if the image characteristic similarity returned by other smart nodes is higher than a threshold value, the smart node judges that other smart nodes have successfully tracked the object, and the smart node notifies other smart nodes to prepare for handoff transfer; and

if the image characteristic similarity returned by other smart nodes is lower than the threshold value, the smart node judges that other smart nodes have not yet tracked the object, and the smart node notifies the control center to re-designate tracking on the object.

22. The smart node according to claim 19, wherein when the smart node makes handoff transfer to other smart nodes, the smart node transmits to other smart nodes messages comprising a serial number of the object, an image characteristic of the object and a monitor metadata of the object.

23. The smart node according to claim 22, wherein the monitor metadata of the object comprises at least one of a threat level and a handoff timing limit parameter.

24. The smart node according to claim 19, wherein when the smart node makes handoff to other smart nodes:

other smart nodes again judge whether the object is still inside their FOV;

if other smart nodes judge that the object is still inside their FOV, other smart nodes return to the smart node for handoff transfer; and

if other smart nodes judge that the object is not inside their FOV, other smart nodes return to the smart node, and the smart node performs an object loss retrieving operation.