Visual content-aware automatic camera adjustment

• US 8,681,232 B2
• Filed: 08/26/2011
• Issued: 03/25/2014
• Est. Priority Date: 08/26/2011
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Fan, Quanfu, Gabbur, Prasad, Miyazawa, Sachiko, Pankanti, Sharathchandra, Trinh, Hoang
• Primary Examiner(s): Pasiewicz, Daniel M
• Application Number:
  US13/218,845
• Publication Number:
  US 20130050517A1
• Time in Patent Office: 942 Days
• Field of Search: 348/77, 348/143, 348/150, 348/207.11, 348/211.4, 348/211.6, 348/222.1, 382/100, 382/103, 705/7.27, 705/7.28
• US Class Current: 348/211.4
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image

Abstract

Visual content in images captured from a scene by a camera in each of a plurality of different pose settings are analyzed to determine predicted occurrences of a transaction associated with the visual content in each pose, which are compared with actual transaction occurrence data to generate performance values for each pose as a function difference between the predicted and actual transactions. Optimized poses are chosen having the best performance value, wherein a camera controller may place the camera in the optimum pose for use in monitoring the scene and generating the primitives of interest associated with the transactions.

Claims

1. A method for adjusting camera settings as a function of associated video analytics data output, the method comprising:

analyzing by a processing unit visual content captured from a scene by a camera in an initial pose setting to determine a predicted occurrence of a transaction associated with the visual content;

comparing by the processing unit the predicted transaction occurrence to data of actual transactions associated with the visual content to generate a performance value for the initial pose as a function of a difference between the predicted transaction and an actual transaction in the data of actual occurrences;

iteratively repeating, for each of at least one more different pose positions, the analyzing by the processing unit visual content captured from each scene by the camera in each of the at least one more pose settings to determine predicted occurrences of transactions associated with the visual content, and the comparing by the processing unit the each predicted transaction occurrences to the data of actual transactions associated with the visual content to generate performance values for each of the at least one more poses as the function of differences between the each predicted transactions and actual transactions in the data; and

choosing by the processing unit an optimized pose from the initial and at least one more different camera poses that has the best performance value, wherein a camera controller may place the camera in the optimum pose for use in monitoring the scene and generating the primitives of interest associated with the transactions.

2. The method of claim 1, further comprising:

selecting the initial pose as a sample drawn randomly from a proposal probability distribution over a state space of all possible camera poses; and

iteratively updating the proposal distribution as a function of each of the determined performance values of each of different samples for each of the different at least one more poses until reaching a stationary distribution approximately converging to the best performance value of the optimum pose.

3. The method of claim 2, wherein each of the state space samples comprise four-dimensional vector values S, Tx, Ty, and Alpha;

and

wherein each of the performance values are objective function distances from transaction occurrences predicted by video analytics for each pose to the actual occurrences in the actual transaction data.

4. The method of claim 3, wherein the step of iteratively updating the proposal distribution further comprises iteratively:

drawing samples from a current proposal distribution;

updating importance weights up to a normalizing constant using the objective function;

creating a new set of samples using their updated importance weights;

setting the updated importance weights of the drawn samples to uniform; and

updating the proposal distribution.

5. The method of claim 4, wherein the pose settings each comprise panning angle and zoom factor values, the method further comprising:

detecting feature points in an image acquired from the camera in a first pose;

comparing feature points in an example image to the feature points detected in the image acquired from the camera in the first pose via an image registration technique;

finding transformation values for at least one of the panning angle and zoom factor values of the pose settings of the first pose as a function of the comparing the feature points to yield an image by the camera that is close to the example image; and

applying the found transformation values to the first pose setting to move the camera into the initial position.

6. The method of claim 5, wherein the steps of finding the transformation values for the first pose and the applying the found transformation values to move the camera into the initial position further comprise at least one of the steps of:

determining a scaling transformation value and adjusting a zoom setting of the camera first pose into another, different zoom setting in the initial position as a function of the determined scaling transformation value;

determining a panning translation transformation value and panning the camera from the first pose into a different panning position in the initial position as a function of the determined panning translation transformation value; and

determining a rotation transformation value and rotating the camera from the first pose into a different rotation position in the initial position as a function of the determined rotation transformation value.

7. A method of providing a service for classifying events through string pattern recognition, the method comprising:

providing a scene analyzer that evaluates a camera pose by comparing transaction occurrences predicted through analysis of scene images acquired by the camera while in the pose to actual transaction data to generate a performance value for the pose as a function of a difference between the expected and predicted transaction data occurrences;

providing a camera pose selector that chooses an optimum pose from a plurality of poses iteratively analyzed by the scene analyzer that has a best performance value;

providing a camera controller that moves the camera into the optimum pose for use in monitoring the scene and generating primitive images of interest associated with the transactions.

8. The method of claim 7, wherein the camera pose selector further selects the initial pose as a sample drawn randomly from a proposal probability distribution over a state space of all possible camera poses;

and

the scene analyzer further iteratively updates the proposal distribution as a function of each of the determined performance values of each of different samples for each of the different at least one more poses until reaching a stationary distribution approximately converging to the best performance value of the optimum pose.

9. The method of claim 8, wherein each of the performance values are objective function distances from transaction occurrences predicted by video analytics for each pose to the actual occurrences in the actual transaction data.

10. The method of claim 9, wherein the scene analyzer further updates the proposal distribution by iteratively:

drawing samples from a current proposal distribution;

updating importance weights up to a normalizing constant using the objective function;

creating a new set of samples using their updated importance weights;

setting the updated importance weights of the drawn samples to uniform; and

updating the proposal distribution.

11. The method of claim 10, wherein the pose settings each comprise panning angle and zoom factor values, and wherein the scene analyzer further:

detects feature points in an image acquired from the camera in a first pose;

compares feature points in an example image to the feature points detected in the image acquired from the camera in the first pose via an image registration technique;

finds transformation values for at least one of the panning angle and zoom factor values of the pose settings of the first pose as a function of the comparing the feature points to yield an image by the camera that is close to the example image; and

applies the found transformation values to the first pose setting to move the camera into the initial position.

12. The method of claim 11, wherein the steps of finding the transformation values for the first pose and the applying the found transformation values to move the camera into the initial position further comprise at least one of the steps of:

the scene analyzer determining a scaling transformation value and the camera pose selector adjusting a zoom setting of the camera first pose into another, different zoom setting in the initial position as a function of the determined scaling transformation value;

the scene analyzer determining a panning translation transformation value and the camera pose selector panning the camera from the first pose into a different panning position in the initial position as a function of the determined panning translation transformation value; and

the scene analyzer determining a rotation transformation value and the camera pose selector rotating the camera from the first pose into a different rotation position in the initial position as a function of the determined rotation transformation value.

13. A system, comprising:

a processing unit;

a computer readable memory; and

a computer-readable storage medium;

wherein the processing unit, when executing program instructions stored on the computer-readable storage medium via the computer readable memory;

evaluates a camera pose by comparing transaction occurrences predicted through analysis of scene images acquired by the camera while in the pose to actual transaction data to generate a performance value for the pose as a function of a difference between the expected and predicted transaction data occurrences;

chooses an optimum pose from a plurality of iteratively analyzed poses that has a best performance value; and

moves the camera into the optimum pose for use in monitoring the scene and generating primitive images of interest associated with the transactions.

14. The system of claim 13, wherein the processing unit, when executing the program instructions stored on the computer-readable storage medium via the computer readable memory, further:

selects the initial pose as a sample drawn randomly from a proposal probability distribution over a state space of all possible camera poses; and

iteratively updates the proposal distribution as a function of each of the determined performance values of each of different samples for each of the different at least one more poses until reaching a stationary distribution approximately converging to the best performance value of the optimum pose.

15. The system of claim 14, wherein each of the performance values are objective function distances from transaction occurrences predicted by video analytics for each pose to the actual occurrences in the actual transaction data.

16. The system of claim 15, wherein the processing unit, when executing the program instructions stored on the computer-readable storage medium via the computer readable memory, further updates the proposal distribution by iteratively:

drawing samples from a current proposal distribution;

updating importance weights up to a normalizing constant using the objective function;

creating a new set of samples using their updated importance weights;

setting the updated importance weights of the drawn samples to uniform; and

updating the proposal distribution.

17. The system of claim 16, wherein the pose settings each comprise panning angle and zoom factor values, and wherein the processing unit, when executing the program instructions stored on the computer-readable storage medium via the computer readable memory, further:

detects feature points in an image acquired from the camera in a first pose;

compares feature points in an example image to the feature points detected in the image acquired from the camera in the first pose via an image registration technique;

finds transformation values for at least one of the panning angle and zoom factor values of the pose settings of the first pose as a function of the comparing the feature points to yield an image by the camera that is close to the example image; and

applies the found transformation values to the first pose setting to move the camera into the initial position.

18. The system of claim 17, wherein the processing unit, when executing the program instructions stored on the computer-readable storage medium via the computer readable memory, further finds the transformation values for the first pose and applies the found transformation values to move the camera into the initial position by at least one of:

determining a scaling transformation value and adjusting a zoom setting of the camera first pose into another, different zoom setting in the initial position as a function of the determined scaling transformation value;

determining a panning translation transformation value and panning the camera from the first pose into a different panning position in the initial position as a function of the determined panning translation transformation value; and

determining a rotation transformation value and rotating the camera from the first pose into a different rotation position in the initial position as a function of the determined rotation transformation value.

19. An article of manufacture, comprising:

a tangible computer readable storage hardware device having computer readable program code embodied therewith, the computer readable program code comprising instructions that, when executed by a computer processor, cause the computer processor to;

evaluate a camera pose by comparing transaction occurrences predicted through analysis of scene images acquired by the camera while in the pose to actual transaction data to generate a performance value for the pose as a function of a difference between the expected and predicted transaction data occurrences;

choose an optimum pose from a plurality of iteratively analyzed poses that has a best performance value; and

move the camera into the optimum pose for use in monitoring the scene and generating primitive images of interest associated with the transactions.

20. The article of manufacture of claim 19, wherein the computer readable program code instructions, when executed by the computer processor, further cause the computer processor to:

select the initial pose as a sample drawn randomly from a proposal probability distribution over a state space of all possible camera poses; and

iteratively update the proposal distribution as a function of each of the determined performance values of each of different samples for each of the different at least one more poses until reaching a stationary distribution approximately converging to the best performance value of the optimum pose.

21. The article of manufacture of claim 20, wherein each of the performance values are objective function distances from transaction occurrences predicted by video analytics for each pose to the actual occurrences in the actual transaction data.

22. The article of manufacture of claim 21, wherein the computer readable program code instructions, when executed by the computer processor, further cause the computer processor to update the proposal distribution by iteratively:

drawing samples from a current proposal distribution;

updating importance weights up to a normalizing constant using the objective function;

creating a new set of samples using their updated importance weights;

setting the updated importance weights of the drawn samples to uniform; and

updating the proposal distribution.

23. The article of manufacture of claim 22, wherein the pose settings each comprise panning angle and zoom factor values, and wherein the computer readable program code instructions, when executed by the computer processor, further cause the computer processor to:

detect feature points in an image acquired from the camera in a first pose;

compare feature points in an example image to the feature points detected in the image acquired from the camera in the first pose via an image registration technique;

find transformation values for at least one of the panning angle and zoom factor values of the pose settings of the first pose as a function of the comparing the feature points to yield an image by the camera that is close to the example image; and

apply the found transformation values to the first pose setting to move the camera into the initial position.

24. The article of manufacture of claim 23, wherein the computer readable program code instructions, when executed by the computer processor, further cause the computer processor to find the transformation values for the first pose and apply the found transformation values to move the camera into the initial position by at least one of:

determining a scaling transformation value and adjusting a zoom setting of the camera first pose into another, different zoom setting in the initial position as a function of the determined scaling transformation value;

determining a panning translation transformation value and panning the camera from the first pose into a different panning position in the initial position as a function of the determined panning translation transformation value; and

determining a rotation transformation value and rotating the camera from the first pose into a different rotation position in the initial position as a function of the determined rotation transformation value.