ON-CAMERA SUMMARISATION OF OBJECT RELATIONSHIPS

• US 20090219391A1
• Filed: 02/17/2009
• Published: 09/03/2009
• Est. Priority Date: 02/28/2008
• Status: Abandoned Application

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Patrick, Lachlan James, McLeish, David Grant
• Application Number:
  US12/372,273
• Publication Number:
  US 20090219391A1
• Time in Patent Office: Days
• US Class Current: 348/143
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters

Abstract

Disclosed herein is a method of communicating video object relationships. The method comprises the steps of: detecting one or more objects in a first video frame (303); detecting one or more objects in a second video frame; determining a first set of temporal object relationships between said objects detected in said second video frame and said objects detected in said first video frame (304); detecting one or more objects in a third video frame; determining a second set of temporal object relationships between said objects detected in said third video frame and said objects detected in said first video frame, based on said first set of temporal object relationships; and transmitting said second set of temporal object relationships to a receiver (309).

Claims

1. A method of communicating video object relationships, comprising the steps of:

(a) detecting one or more objects in a first video frame;

(b) detecting one or more objects in a second video frame;

(c) determining a first set of temporal object relationships between said objects detected in said second video frame and said objects detected in said first video frame;

(d) detecting one or more objects in a third video frame;

(e) determining a second set of temporal object relationships between said objects detected in said third video frame and said objects detected in said first video frame, based on said first set of temporal object relationships; and

(f) transmitting said second set of temporal object relationships to a receiver.

2. The method according to claim 1, comprising the further steps of:

(g) detecting one or more objects in an intermediate frame between said first and said second video frames;

(h) determining an extra set of temporal object relationships between said objects from said intermediate video frame and said objects detected in said first video frame; and

wherein said first set of temporal object relationships are determined in step (c), based on said objects detected in said intermediate video frame and said extra set of temporal object relationships.

3. The method according to claim 1, wherein said determination in step (e) comprises the further steps of:

(i) determining one or more self and parent relationships between said objects detected in said second video frame to one or more of said objects detected in said first video frame to determine said first set of temporal object relationships;

(j) determining object self relationships between said objects in said third video frame and one or more said objects detected in said first video frame,

(k) deducing object self relationships between said objects detected in said second video frame and said objects detected in said third video frame using said self relationships from steps (i) and (j); and

(l) establishing a set of object parent relationships for each said object detected in said third video frame, such that the parents of each said object are determined by combining the said self relationships deduced in step (k) with the said parent relationships from step (i).

4. The method according to claim 3, wherein:

the first video frame, and data describing objects detected in said first video frame, are transmitted to the receiver; and

the third video frame, and data describing objects detected in said third video frame, are transmitted to the receiver with the second set of temporal object relationships.

5. The method according to claim 3, wherein:

there is a plurality of receivers;

each said receiver is associated with a receiver set;

each said receiver set has an associated transmission criterion; and

the said second set of temporal object relationships is transmitted to all said receivers within a receiver set only whenever said transmission criterion associated with said receiver set is evaluated as true.

6. The method according to claim 5, wherein the said transmission criterion evaluates to true if a time threshold has elapsed since the previous transmission to said receiver set.

7. The method according to claim 5, wherein the said transmission criterion evaluates to true if the said first set of object relationships contains at least one of a split relationship and a merge relationship.

8. The method according to claim 5, wherein the said transmission criterion evaluates to true if:

(m) the said first set of object relationships contains a parent relationship; and

(n) the said second set of object relationships contains a parent relationship deduced from the information in step (m).

9. The method according to claim 5, wherein the said transmission criterion evaluates to true if at least one of said objects detected in said third video frame overlaps a specified region within the frame.

10. The method according to claim 5, wherein the said transmission criterion includes at least one of the status of the camera, object detection, and communication subsystems.

11. The method according to claim 10, wherein the said status involves a low memory situation.

12. The method according to claim 10, wherein the said status involves at least one of a change to a direction of the camera and a change to a setting of a lens.

13. The method according to claim 6, wherein there are two or more of said receiver sets, each of which has a said transmission criterion which uses a different time interval from each other.

14. The method according to claim 1, wherein information is also transmitted concerning any said objects detected in said second video frame which are not related by any self relationships to any said objects in said first and third video frames.

15. The method according to claim 5, wherein if the said transmission criterion evaluates to true then transmission of video frames of a predefined resolution is initiated to receivers associated with said receiver set.

16. A system comprising:

a source of video frames;

an object detection subsystem for receiving video frames from said camera and detecting objects in said video frames, said object detection subsystem including a tracking means for determining relationship information between detected objects in a current video frame and objects detected in at least one preceding video frame; and

a communication subsystem for transmitting said relationship information to a receiver.

17. The system according to claim 16, wherein said source of video frames is a video camera.

18. The system according to claim 16, wherein said receiver is associated with a receiver set, said receiver set having an associated temporal criterion for determining when relationship information is transmitted to each receiver in said receiver set.