Automatically determining field of view overlap among multiple cameras

• US 9,165,375 B2
• Filed: 11/04/2014
• Issued: 10/20/2015
• Est. Priority Date: 03/28/2013
• Status: Expired due to Fees

• Current Assignee: Kyndryl Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Datta, Ankur, Feris, Rogerio S., Pankanti, Sharathchandra U., Zhai, Yun
• Primary Examiner(s): LE, BRIAN Q
• Application Number:
  US14/532,347
• Publication Number:
  US 20150055830A1
• Time in Patent Office: 350 Days
• Field of Search: 382/103, 382/154, 382/190, 382/194, 382/199, 382/225, 382/284, 382/285, 382/294
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/292 Multi-camera tracking

Abstract

Field of view overlap among multiple cameras are automatically determined as a function of the temporal overlap of object tracks determined within their fields-of-view. Object tracks with the highest similarity value are assigned into pairs, and portions of the assigned object track pairs having a temporally overlapping period of time are determined. Scene entry points are determined from object locations on the tracks at a beginning of the temporally overlapping period of time, and scene exit points from object locations at an ending of the temporally overlapping period of time. Boundary lines for the overlapping fields-of-view portions within the corresponding camera fields-of-view are defined as a function of the determined entry and exit points in their respective fields-of-view.

Claims

1. A method for automatically determining field of view overlap among multiple cameras as a function of the temporal overlap of object tracks, the method comprising:

determining values of similarity for potential pairings of each of a plurality of object tracks of movements of objects over time through a field-of-view of a first camera with each of a plurality of object tracks of movements of objects over time through a field-of-view of a second camera;

assigning into pairs each of the object tracks within the first camera field-of-view to ones each of the object tracks within the second camera field-of-view that each have a highest determined similarity value of the similarity values determined for their respective potential pairings;

determining that the assigned object track pairs correspond to overlapping portions of the fields-of-view of the first and second paired cameras if portions of each of the assigned object track pairs have a temporally overlapping period of time;

determining scene entry points on each of the tracks of the assigned object track pairs corresponding to the overlapping fields-of-view portions within their fields-of-view as locations of the object on the each of said tracks at a beginning of the temporally overlapping period of time;

determining scene exit points on each of tracks of the assigned object track pairs corresponding to the overlapping fields-of-view portions within their fields-of-view as locations of the object on the each of said tracks at an ending of the temporally overlapping period of time; and

defining boundary lines for the overlapping fields-of-view portions within their corresponding first and second camera fields-of-view as a function of the determined entry and exit points in their respective fields-of-view.

2. The method of claim 1, wherein the step of defining the boundary lines comprises determining linear boundary lines through the fields of view that comprise subsets of the determined entry and exit points in the respective fields-of-view.

3. The method of claim 2, wherein the step of defining the boundary lines comprises determining linear solutions using robust regression through the subsets of the determined entry and exit points in the respective fields-of-view.

4. The method of claim 2, wherein the step of assigning into the pairs each of the object tracks that have the highest determined similarity values for their respective potential pairings comprises:

building a similarity matrix for different respective values of similarity comprising a plurality of locations that are each defined as pairings of one each of the object tracks in the first camera view to one each of the object tracks in the second camera view;

populating each of the matrix locations with similarity values of their respective track pairings;

discarding matrix locations populated with similarity values that have similarity values below a requisite minimum threshold value; and

determining the potential pairings as the object tracks corresponding to the populated matrix locations having the highest similarity values.

5. The method of claim 2, further comprising:

readjusting a first feature weighting used to match appearances of objects represented by respective tracks of the assigned object tracks downward relative to a weighting of a second feature used to match the appearances of objects in response to;

the first feature indicating a first match of objects between the first and second camera fields-of-view that is located outside of the overlapping fields-of-view areas defined by the boundary lines; and

the second feature indicating a second match of objects between the first and second camera fields-of-view that is located within the overlapping fields-of-view areas defined by the boundary lines.

6. The method of claim 2, further comprising:

in response to a determined change in a relative difference in motion or size of a matched appearance of an object represented by respective tracks of one of the assigned object track pairs corresponding to the overlapping fields-of-view portions over time, determining a displacement distance between before and after locations of the object in one of the fields of views of the one assigned object track pair; and

updating a location of at least one of the boundary lines within the fields of view of the one assigned object track pair by the determined displacement distance.

7. The method of claim 1, further comprising:

integrating computer-readable program code into a computer system comprising a processing unit, a computer readable memory and a computer readable tangible storage medium; and

wherein the computer readable program code is embodied on the computer readable tangible storage medium and comprises instructions for execution by the processing unit via the computer readable memory to cause the processing unit to perform the steps of determining the value of similarity for the potential pairings of the object tracks, assigning into pairs the object tracks having the highest determined similarity values, determining that the assigned object track pairs correspond to overlapping portions of the fields-of-view if portions of each have the temporally overlapping period of time, determining the scene entry points as the locations of the object on the tracks at the beginning of the temporally overlapping period of time, determining the scene exit points as the locations of the objects on the tracks at the ending of the temporally overlapping period of time, and defining the boundary lines as the function of the determined entry and exit points in their respective fields-of-view.

8. The method of claim 7, wherein the processing unit executes the program instructions stored on the tangible computer-readable storage medium via the computer readable memory and thereby defines the boundary lines by determining linear boundary lines through the fields of views that comprise subsets of the determined entry and exit points in the respective fields-of-view.

9. A system, comprising:

a processing unit in communication with a computer readable memory and a computer-readable storage medium;

wherein the processing unit executes program instructions stored on the computer-readable storage medium via the computer readable memory and thereby;

determines values of similarity for potential pairings of each of a plurality of object tracks of movements of objects over time through a field-of-view of a first camera with each of a plurality of object tracks of movements of objects over time through a field-of-view of a second camera;

assigns into pairs each of the object tracks within the first camera field-of-view to ones each of the object tracks within the second camera field-of-view that each have a highest determined similarity value of the similarity values determined for their respective potential pairings;

determines that the assigned object track pairs correspond to overlapping portions of the fields-of-view of the first and second paired cameras if portions of each of the assigned object track pairs have a temporally overlapping period of time;

determines scene entry points on each of the tracks of the assigned object track pairs corresponding to the overlapping fields-of-view portions within their fields-of-view as locations of the object on the each of said tracks at a beginning of the temporally overlapping period of time;

determines scene exit points on each of tracks of the assigned object track pairs corresponding to the overlapping fields-of-view portions within their fields-of-view as locations of the object on the each of said tracks at an ending of the temporally overlapping period of time; and

defines boundary lines for the overlapping fields-of-view portions within their corresponding first and second camera fields-of-view as a function of the determined entry and exit points in their respective fields-of-view.

10. The system of claim 9, wherein the processing unit executes the program instructions stored on the computer-readable storage medium via the computer readable memory and thereby defines the boundary lines by determining linear boundary lines through the fields of view that comprise subsets of the determined entry and exit points in the respective fields-of-view.

11. The system of claim 10, wherein the processing unit executes the program instructions stored on the computer-readable storage medium via the computer readable memory and thereby defines the boundary lines by determining linear solutions using robust regression through the subsets of the determined entry and exit points in the respective fields-of-view.

12. The system of claim 9, wherein the processing unit executes the program instructions stored on the computer-readable storage medium via the computer readable memory and thereby assigns into the pairs each of the object tracks that have the highest determined similarity values for their respective potential pairings by:

building a similarity matrix for different respective values of similarity comprising a plurality of locations that are each defined as pairings of one each of the object tracks in the first camera view to one each of the object tracks in the second camera view;

populating each of the matrix locations with similarity values of their respective track pairings;

discarding matrix locations populated with similarity values that have similarity values below a requisite minimum threshold value; and

determining the potential pairings as the object tracks corresponding to the populated matrix locations having the highest similarity values.

13. The system of claim 9, wherein the processing unit executes the program instructions stored on the computer-readable storage medium via the computer readable memory and thereby further:

readjusts a first feature weighting used to match appearances of objects represented by respective tracks of the assigned object tracks downward relative to a weighting of a second feature used to match the appearances of objects in response to;

the first feature indicating a first match of objects between the first and second camera fields-of-view that is located outside of the overlapping fields-of-view areas defined by the boundary lines; and

the second feature indicating a second match of objects between the first and second camera fields-of-view that is located within the overlapping fields-of-view areas defined by the boundary lines.

14. The system of claim 9, wherein the processing unit executes the program instructions stored on the computer-readable storage medium via the computer readable memory and thereby further:

in response to a determined change in a relative difference in motion or size of a matched appearance of an object represented by respective tracks of one of the assigned object track pairs corresponding to the overlapping fields-of-view portions over time, determines a displacement distance between before and after locations of the object in one of the fields of view of the one assigned object track pair; and

updates a location of at least one of the boundary lines within the fields of views of the one assigned object track pair by the determined displacement distance.

15. A computer program product for automatically determining field of view overlap among multiple cameras as a function of the temporal overlap of object tracks, the computer program product comprising:

a non-transitory computer readable hardware storage device having computer readable program code embodied therewith, the computer readable program code comprising instructions for execution by a computer processing unit that cause the computer processing unit to;

determine values of similarity for potential pairings of each of a plurality of object tracks of movements of objects over time through a field-of-view of a first camera with each of a plurality of object tracks of movements of objects over time through a field-of-view of a second camera;

assign into pairs each of the object tracks within the first camera field-of-view to ones each of the object tracks within the second camera field-of-view that each have a highest determined similarity value of the similarity values determined for their respective potential pairings;

determine that the assigned object track pairs correspond to overlapping portions of the fields-of-view of the first and second paired cameras if portions of each of the assigned object track pairs have a temporally overlapping period of time;

determine scene entry points on each of the tracks of the assigned object track pairs corresponding to the overlapping fields-of-view portions within their fields-of-view as locations of the object on the each of said tracks at a beginning of the temporally overlapping period of time;

determine scene exit points on each of tracks of the assigned object track pairs corresponding to the overlapping fields-of-view portions within their fields-of-view as locations of the object on the each of said tracks at an ending of the temporally overlapping period of time; and

define boundary lines for the overlapping fields-of-view portions within their corresponding first and second camera fields-of-view as a function of the determined entry and exit points in their respective fields-of-view.

16. The computer program product of claim 15, wherein the computer readable program code instructions for execution by the computer processing unit further cause the computer processing unit to define the boundary lines by determining linear boundary lines through the fields of view that comprise subsets of the determined entry and exit points in the respective fields-of-view.

17. The computer program product of claim 16, wherein the computer readable program code instructions for execution by the computer processing unit further cause the computer processing unit to define the boundary lines by determining linear solutions using robust regression through the subsets of the determined entry and exit points in the respective fields-of-view.

18. The computer program product of claim 16, wherein the computer readable program code instructions for execution by the computer processing unit further cause the computer processing unit to assign into the pairs each of the object tracks that have the highest determined similarity values for their respective potential pairings by:

building a similarity matrix for different respective values of similarity comprising a plurality of locations that are each defined as pairings of one each of the object tracks in the first camera view to one each of the object tracks in the second camera view;

populating each of the matrix locations with similarity values of their respective track pairings;

discarding matrix locations populated with similarity values that have similarity values below a requisite minimum threshold value; and

determining the potential pairings as the object tracks corresponding to the populated matrix locations having the highest similarity values.

19. The computer program product of claim 16, wherein the computer readable program code instructions for execution by the computer processing unit further cause the computer processing unit to:

readjust a first feature weighting used to match appearances of objects represented by respective tracks of the assigned object tracks downward relative to a weighting of a second feature used to match the appearances of objects in response to;

the first feature indicating a first match of objects between the first and second camera fields-of-view that is located outside of the overlapping fields-of-view areas defined by the boundary lines; and

the second feature indicating a second match of objects between the first and second camera fields-of-view that is located within the overlapping fields-of-view areas defined by the boundary lines.

20. The computer program product of claim 16, wherein the computer readable program code instructions for execution by the computer processing unit further cause the computer processing unit to:

in response to a determined change in a relative difference in motion or size of a matched appearance of an object represented by respective tracks of one of the assigned object track pairs corresponding to the overlapping fields-of-view portions over time, determine a displacement distance between before and after locations of the object in one of the fields of view of the one assigned object track pair; and

update a location of at least one of the boundary lines within the fields of views of the one assigned object track pair by the determined displacement distance.