Moving-object tracking control apparatus, moving-object tracking system, moving-object tracking control method, and program

• US 7,783,076 B2
• Filed: 07/06/2006
• Issued: 08/24/2010
• Est. Priority Date: 07/15/2005
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Tsunashima, Nobuhiro
• Primary Examiner(s): Tucker; Wesley
• Assistant Examiner(s): Roz; Mark
• Application Number:
  US11/428,968
• Publication Number:
  US 20070014432A1
• Time in Patent Office: 1,510 Days
• Field of Search: 382/103, 382/107, 375/240.16, 348/416.1
• US Class Current: 382/103
• CPC Class Codes: G01S 3/7864 T.V. type tracking systems
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

A moving-object region is specified on an image captured by an imaging apparatus, and a plurality of calculation points are arranged on the specified region. A group of local motion vectors is determined for the calculation points, and a moving-object motion vector is obtained based on the determined local motion vectors. A group of calculation points whose local motion vectors are the same as or similar to the moving-object motion vector is continuously used to calculate the local motion vectors, and is updated by moving the position of the group of calculation points along the moving-object motion vector, and a calculation point whose local motion vector is different from the moving-object motion vector is rearranged by placing a new calculation point at a position near any of the continuously used calculation points. A movable mechanism including the imaging apparatus is controlled in accordance with the moving-object motion vector.

Claims

1. A moving-object tracking control apparatus comprising:

a moving-object specifying unit that specifies a moving-object region on an image captured by an imaging apparatus;

a calculation point arranging unit that arranges a plurality of calculation points on the specified moving-object region;

a local motion vector determining unit that determines a group of local motion vectors for the calculation points;

a moving-object motion vector obtaining unit that obtains a moving-object motion vector based on the group of local motion vectors;

a calculation point updating unit that updates the calculation points including a group of calculation points whose determined local motion vectors are the same as or similar to the moving-object motion vector and calculation points whose determined local motion vectors are different from the moving-object motion vector,

the group of calculation points being continuously used to calculate the local motion vectors and being updated by moving the position of each of the group of calculation points along the moving-object motion vector,

each of the calculation points whose determined local motion vectors are different from the moving-object motion vector being rearranged by placing a new calculation point at a position in the vicinity of any of the continuously used calculation points; and

an imaging direction driving control unit that controls a movable mechanism including the imaging apparatus in accordance with the moving-object motion vector.

2. The moving-object tracking control apparatus according to claim 1, wherein the calculation point arranging unit arranges the calculation points randomly throughout the moving-object region.

3. The moving-object tracking control apparatus according to claim 1, wherein the calculation point arranging unit arranges the calculation points uniformly on the moving-object region.

4. The moving-object tracking control apparatus according to claim 1, wherein the calculation point arranging unit arranges the calculation points in the vicinity of a feature point in the moving-object region.

5. The moving-object tracking control apparatus according to claim 1, wherein the moving-object motion vector obtaining unit determines the local motion vector with the highest frequency of occurrence in the group of local motion vectors as the moving-object motion vector.

6. The moving-object tracking control apparatus according to claim 1, wherein the moving-object motion vector obtaining unit determines an arithmetic approximation determined with respect to the group of local motion vectors-as the moving-object motion vector.

7. The moving-object tracking control apparatus according to claim 1, wherein the calculation point updating unit randomly places the new calculation points at position in the vicinity of an arbitrary calculation point of the continuously used calculation points.

8. The moving-object tracking control apparatus according to claim 1, wherein the calculation point updating unit places the new calculation points according to a probability distribution centered at an arbitrary calculation point of the continuously used calculation points.

9. The moving-object tracking control apparatus according to claim 1, wherein the calculation point updating unit places the new calculation points at the centroids of all of the continuously used calculation points.

10. The moving-object tracking control apparatus according to claim 1, wherein the calculation point updating unit refers to a continuation coefficient corresponding to each of the calculation points to determine whether each of the calculation points are to be continuously used or rearranged, and continuously uses or rearranges each of the calculation points based on the value of the continuation coefficient.

11. The moving-object tracking control apparatus according to claim 10, wherein:

the continuation coefficient is set to an initial value when a new calculation point is placed or a calculation point is rearranged;

a first update value is added to the continuation coefficient when the local motion vector corresponding to the calculation point is the same as or similar to the moving-object motion vector; and

a second update value is subtracted from the continuation coefficient when the local motion vector corresponding to the calculation point is different from the moving-object motion vector.

12. The moving-object tracking control apparatus according to claim 1, further comprising a zoom control unit that controls a zoom lens of the imaging apparatus depending on the size of the moving-object region.

13. A moving-object tracking control apparatus comprising:

a moving-object specifying unit that specifies a moving-object region on an image captured by an imaging apparatus or a moving-object region on an image read from a server;

a calculation point arranging unit that arranges a plurality of calculation points on the specified moving-object region;

a local motion vector determining unit that determines a group of local motion vectors for the calculation points;

a moving-object motion vector obtaining unit that obtains a moving-object motion vector based on the group of local motion vectors;

a calculation point updating unit that updates the calculation points including a group of calculation points whose determined local motion vectors are the same as or similar to the moving-object motion vector and calculation points whose determined local motion vectors are different from the moving-object motion vector,

the group of calculation points being continuously used to calculate the local motion vectors and being updated by moving the position of each of the group of calculation points along the moving-object motion vector,

each of the calculation points whose determined local motion vectors are different from the moving-object motion vector being rearranged by placing a new calculation point at a position in the vicinity of any of the continuously used calculation points; and

an extracted-region moving control unit that controls so as to move a region extracted from the image in accordance with the moving-object motion vector.

14. A moving-object tracking system comprising:

an imaging apparatus installed in a movable mechanism;

a moving-object specifying unit that specifies a moving-object region on an image captured by the imaging apparatus;

a calculation point arranging unit that arranges a plurality of calculation points on the specified moving-object region;

a local motion vector determining unit that determines a group of local motion vectors for the calculation points;

a moving-object motion vector obtaining unit that obtains a moving-object motion vector based on the group of local motion vectors;

a calculation point updating unit that updates the calculation points including a group of calculation points whose determined local motion vectors are the same as or similar to the moving-object motion vector and calculation points whose determined local motion vectors are different from the moving-object motion vector,

the group of calculation points being continuously used to calculate the local motion vectors and being updated by moving the position of each of the group of calculation points along the moving-object motion vector,

each of the calculation points whose determined local motion vectors are different from the moving-object motion vector being rearranged by placing a new calculation point at a position in the vicinity of any of the continuously used calculation points; and

an imaging direction driving control unit that controls the movable mechanism in accordance with the moving-object motion vector.

15. A moving-object tracking system comprising:

an imaging apparatus;

a moving-object specifying unit that specifies a moving-object region on an image captured by the imaging apparatus;

a calculation point arranging unit that arranges a plurality of calculation points on the specified moving-object region;

a local motion vector determining unit that determines a group of local motion vectors for the calculation points;

a moving-object motion vector obtaining unit that obtains a moving-object motion vector based on the group of local motion vectors;

a calculation point updating unit that updates the calculation points including a group of calculation points whose determined local motion vectors are the same as or similar to the moving-object motion vector and calculation points whose determined local motion vectors are different from the moving-object motion vector,

the group of calculation points being continuously used to calculate the local motion vectors and being updated by moving the position of each of the group of calculation points along the moving-object motion vector,

each of the calculation points whose determined local motion vectors are different from the moving-object motion vector being rearranged by placing a new calculation point at a position in the vicinity of any of the continuously used calculation points; and

an extracted-region moving control unit that controls so as to move a region extracted from the image in accordance with the moving-object motion vector.

16. A moving-object tracking system comprising:

a server that stores image data;

a moving-object specifying unit that specifies a moving-object region on an image read from the server;

a calculation point arranging unit that arranges a plurality of calculation points on the specified moving-object region;

a local motion vector determining unit that determines a group of local motion vectors for the calculation points;

a moving-object motion vector obtaining unit that obtains a moving-object motion vector based on the group of local motion vectors;

a calculation point updating unit that updates the calculation points including a group of calculation points whose determined local motion vectors are the same as or similar to the moving-object motion vector and calculation points whose determined local motion vectors are different from the moving-object motion vector,

the group of calculation points being continuously used to calculate the local motion vectors and being updated by moving the position of each of the group of calculation points along the moving-object motion vector,

each of the calculation points whose determined local motion vectors are different from the moving-object motion vector being rearranged by placing a new calculation point at a position in the vicinity of any of the continuously used calculation points; and

an extracted-region moving control unit that controls so as to move a region extracted from the image in accordance with the moving-object motion vector.

17. A moving-object tracking control method comprising the steps of:

specifying a moving-object region on an image captured by an imaging apparatus;

arranging a plurality of calculation points on the specified moving-object region;

determining a group of local motion vectors for the calculation points;

obtaining a moving-object motion vector based on the group of local motion vectors;

updating the calculation points including a group of calculation points whose determined local motion vectors are the same as or similar to the moving-object motion vector and calculation points whose determined local motion vectors are different from the moving-object motion vector,

the group of calculation points being continuously used to calculate the local motion vectors and being updated by moving the position of each of the group of calculation points along the moving-object motion vector,

each of the calculation points whose determined local motion vectors are different from the moving-object motion vector being rearranged by placing a new calculation point at a position in the vicinity of any of the continuously used calculation points; and

controlling a movable mechanism including the imaging apparatus in accordance with the moving-object motion vector.

18. A moving-object tracking control method comprising the steps of:

specifying a moving-object region on an image captured by an imaging apparatus or a moving-object region on an image read from a server;

arranging a plurality of calculation points on the specified moving-object region by a programmed processor;

determining a group of local motion vectors for the calculation points;

obtaining a moving-object motion vector based on the group of local motion vectors;

updating the calculation points including a group of calculation points whose determined local motion vectors are the same as or similar to the moving-object motion vector and calculation points whose determined local motion vectors are different from the moving-object motion vector,

the group of calculation points being continuously used to calculate the local motion vectors and being updated by moving the position of each of the group of calculation points along the moving-object motion vector,

each of the calculation points whose determined local motion vectors are different from the moving-object motion vector being rearranged by placing a new calculation point at a position in the vicinity of any of the continuously used calculation points; and

controlling so as to move a region extracted from the image in accordance with the moving-object motion vector.

19. A computer readable medium storing a program that, when executed by a processor, causing a computer to execute the steps of:

specifying a moving-object region on an image captured by an imaging apparatus;

arranging a plurality of calculation points on the specified moving-object region;

determining a group of local motion vectors for the calculation points;

obtaining a moving-object motion vector based on the group of local motion vectors;

updating the calculation points including a group of calculation points whose determined local motion vectors are the same as or similar to the moving-object motion vector and calculation points whose determined local motion vectors are different from the moving-object motion vector,

the group of calculation points being continuously used to calculate the local motion vectors and being updated by moving the position of each of the group of calculation points along the moving-object motion vector,

each of the calculation points whose determined local motion vectors are different from the moving-object motion vector being rearranged by placing a new calculation point at a position in the vicinity of any of the continuously used calculation points; and

controlling a movable mechanism including the imaging apparatus in accordance with the moving-object motion vector.

20. A computer readable medium storing a program that, when executed by a processor, causing a computer to execute the steps of:

specifying a moving-object region on an image captured by an imaging apparatus or a moving-object region on an image read from a server;

arranging a plurality of calculation points on the specified moving-object region;

determining a group of local motion vectors for the calculation points;

obtaining a moving-object motion vector based on the group of local motion vectors;

updating the calculation points including a group of calculation points whose determined local motion vectors are the same as or similar to the moving-object motion vector and calculation points whose determined local motion vectors are different from the moving-object motion vector,

the group of calculation points being continuously used to calculate the local motion vectors and being updated by moving the position of each of the group of calculation points along the moving-object motion vector,

each of the calculation points whose determined local motion vectors are different from the moving-object motion vector being rearranged by placing a new calculation point at a position in the vicinity of any of the continuously used calculation points; and

controlling so as to move a region extracted from the image in accordance with the moving-object motion vector.

21. A moving-object tracking control apparatus comprising:

a moving-object specifying unit that specifies a moving-object region on an image captured by an imaging apparatus;

a calculation point arranging unit that arranges a plurality of calculation points on the specified moving-object region;

a local motion vector determining unit that determines a group of local motion vectors for the calculation points;

a moving-object motion vector obtaining unit that obtains a moving-object motion vector based on the group of local motion vectors; and

a calculation point updating processing unit that updates the calculation points including a group of calculation points whose determined local motion vectors are the same as or similar to the moving-object motion vector and calculation points whose determined local motion vectors are different from the moving-object motion vector,

the group of calculation points being continuously used to calculate the local motion vectors and being updated by moving the position of each of the group of calculation points along the moving-object motion vector,

each of the calculation points whose determined local motion vectors are different from the moving-object motion vector being rearranged by placing a new calculation point at a position in the vicinity of any of the continuously used calculation points the obtained moving-object motion vector provides tracking control for a movable mechanism.

22. A moving-object tracking control apparatus comprising:

a moving-object specifying unit that specifies a moving-object region on an image captured by an imaging apparatus or a moving-object region on an image read from a server;

a calculation point arranging unit that arranges a plurality of calculation points on the specified moving-object region;

a local motion vector determining unit that determines a group of local motion vectors for the calculation points;

a moving-object motion vector obtaining unit that obtains a moving-object motion vector based on the group of local motion vectors; and

a calculation point updating processing unit that updates the calculation points including a group of calculation points whose determined local motion vectors are the same as or similar to the moving-object motion vector and calculation points whose determined local motion vectors are different from the moving-object motion vector,

the group of calculation points being continuously used to calculate the local motion vectors and being updated by moving the position of each of the group of calculation points along the moving-object motion vector,

each of the calculation points whose determined local motion vectors are different from the moving-object motion vector being rearranged by placing a new calculation point at a position in the vicinity of any of the continuously used calculation points the obtained moving-object motion vector provides tracking control for a movable mechanism.