Generalized multi-sensor planning and systems

• US 8,184,157 B2
• Filed: 12/11/2006
• Issued: 05/22/2012
• Est. Priority Date: 12/16/2005
• Status: Active Grant

• Current Assignee: Siemens Corp. (Siemens AG)
• Original Assignee: Siemens Corp. (Siemens AG)
• Inventors: Mittal, Anurag
• Primary Examiner(s): Salad, Abdullahi
• Assistant Examiner(s): Tran, Jimmy H
• Application Number:
  US11/636,955
• Publication Number:
  US 20080007720A1
• Time in Patent Office: 1,989 Days
• Field of Search: 348/143, 348152-155, 348/159, 348169-172, 382/103
• US Class Current: 348/143
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A method is disclosed for the analysis of scenarios where there are dynamically occurring objects capable of occluding each others. Application of the method in vision systems is also disclosed. Methods for incorporating visibility constraints for occluding scenarios are provided in a multi-camera setting. Other static constraints such as image resolution and field-of-view, and algorithmic requirements such as stereo reconstruction, face detection and background appearance are also addressed. A generic framework for sensor planning is also provided.

Claims

1. A method for determining an optimal sensor configuration in a vision system, comprising:

a processor determining a capture quality function of an object at a plurality of locations for each of a plurality of sensor configurations, wherein the capture quality function depends non-uniformly upon an orientation of the object;

for each of the plurality of sensor configurations, the processor using the capture quality function determined at the plurality of locations to determine a cost function;

the processor determining an optimal sensor configuration based on the cost function determined for each of the plurality of sensor configurations; and

wherein

the cost function is provided by

2. The method as claimed in claim 1, wherein the sensors are cameras.

3. The method as claimed in claim 1, wherein the capture quality function depends on probabilities associated with an event set of objects.

4. The method as claimed in claim 1, wherein an optimal sensor configuration is determined by applying a Simulated Annealing algorithm on the cost function.

5. The method as claimed in claim 1, wherein objects are dynamic and may occlude each other.

6. The method as claimed in claim 1, wherein a sensor configuration is constrained by one or more of a set of constraints.

7. The method as claimed in claim 6, wherein the set of constraints comprises:

image resolution;

stereo matching;

face recognition;

person detection; and

field of view.

8. The method as claimed in claim 1, further comprising application in a surveillance system.

9. A vision system, comprising a plurality of sensors and a processor enabled to determine a location for each of the plurality of sensors by performing the steps:

(a) determining a capture quality function of an object at a plurality of locations for each of a plurality of sensor configurations, wherein the capture quality function depends non-uniformly upon an orientation of the object;

(b) for each of the plurality of sensor configurations, using the capture quality function determined at the plurality of locations to determine a cost function, wherein the cost function is provided by

10. The vision system as claimed in claim 9, wherein the sensors are cameras.

11. The system as claimed in claim 9, wherein the capture quality function depends on probabilities associated with an event set of objects.

12. The vision system as claimed in claim 9, wherein an optimal sensor configuration is determined by applying a Simulated Annealing algorithm on the cost function.

13. The vision system as claimed in claim 9, wherein objects are dynamic and may occlude each other.

14. The vision system as claimed in claim 9, wherein a sensor configuration is constrained by one or more of a set of constraints.

15. The vision system as claimed in claim 9, wherein the set of constraints comprises:

image resolution;

stereo matching;

face recognition;

person detection; and

field of view.

16. The vision system as claimed in claim 9, further comprising application in a surveillance system.

17. The vision system as claimed in claim 15, further comprising a surveillance system in one or more settings including:

a museum;

a shopping mall;

a subway station;

or a parking lot.