Object trace device, object method, and object trace program

• US 20050254686A1
• Filed: 08/27/2003
• Published: 11/17/2005
• Est. Priority Date: 08/30/2002
• Status: Abandoned Application

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Koizumi, Hirokazu
• Application Number:
  US10/525,874
• Publication Number:
  US 20050254686A1
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 10/62 relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
• CPC Class Codes: G06V 10/7515 Shifting the patterns to accommodate for positional errors
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

Object-zone extraction section 12 receives image information through image input terminal 11 and extracts object zones from the received image information. Characteristic-quantity generation section 14 generates the characteristic quantities of objects and the characteristic quantities of object zones. Characteristic-quantity synthesis section 15 synthesizes the characteristic quantities of a plurality of objects to generate a synthesized characteristic quantity. Correspondence establishment section 17 calculates the degrees of similarity between the characteristic quantities of object zones and synthesized characteristic quantities, and finds an optimum correspondence between the objects and object zone, on the basis of calculated degrees of similarity.

Claims

1. An object-tracking device for tracking an object based on image information, comprising:

a characteristic-quantity synthesizing means adapted to synthesize characteristic quantities of objects representative of characteristic quantities of respective objects included in said image information for generating synthesized characteristic quantities; and

a correspondence-establishing means for establishing correspondences between object zones and objects on the basis of degrees of similarity between characteristic quantities of said object zones and said synthesized characteristic quantities, wherein said object zones refer to the zones that are extracted from said image information and include the objects of interest.

2. An object-tracking device according to claim 1, wherein said characteristic-quantity synthesizing means is adapted to synthesize characteristic quantities for each of all required combinations of a plurality of objects to generate said synthesized characteristic quantities, and

said correspondence-establishing means establishes correspondences between objects and object zones through comparing each of said synthesized characteristic quantities generated by said characteristic-quantity synthesizing means and zone characteristic quantities representative of the characteristic quantities of object zones.

3. An object-tracking device according to claim 2, provided with:

an object-zone extracting means for extracting said object zones from said image information and providing the object-zone information that includes the image information about said object zones,

a state-of-tracking deciding means for deciding the states of tracking of individual objects or object zones, wherein said state-of-tracking means relative positions of each object with respect to other objects, and

a characteristic-quantity generating means for generating said zone characteristic quantities and object characteristic quantities through the use of said image information, said object-zone information and the decision results effected by said state-of-tracking deciding means,

wherein said characteristic-quantity synthesizing means generates synthesized characteristic quantities through the use of said object characteristic quantities and the decision results effected by said state-of-tracking deciding means.

4. An object-tracking device according to claim 3, wherein said state-of-tracking deciding means decides the states of tracking of respective objects or object zones based on the object-zone information and the correspondence information that has been determined that indicates the corresponding relationship between the object zones and objects prior to the present to provide the first zone-correspondence information that indicates the corresponding relationship among the object zones and objects and said states of tracking,

said characteristic-quantity generating means generates zone characteristic quantities and object characteristic quantities based on the current image information, said object-zone information, said first zone-correspondence information and said correspondence information that has been determined,

said characteristic-quantity synthesizing means generates synthesized characteristic quantities that serve as candidates to be placed in the corresponding relationship to individual object zones based on said object characteristic quantities and said first zone-correspondence information to provide synthesized characteristic-quantity information, wherein said synthesized characteristic-quantity information is the information that includes synthesized characteristic quantities and the corresponding relationship between the synthesized characteristic quantities and objects used for the generation of said synthesized characteristic quantities, and

said correspondence-establishing means includes a correspondence-determining means that associates objects and object zones to place in the corresponding relationship based on said first zone-correspondence information, zone characteristic-quantity information that is the information indicative of said zone characteristic quantities and said synthesized characteristic-quantity information to provide said correspondence information that has been determined in the present time

5. An object-tracking device according to claim 3, wherein said state of tracking includes at least one of or a combination of:

a stand-alone state in which only a single object resides in an object zone;

a crossover state in which a plurality of objects correspond to a single object zone; and

a state of parting that is a transient state in which a single object zone is parted into a plurality of object zones.

6. An object-tracking device according to claim 4, wherein said state of tracking includes at least one of or a combination of:

a stand-alone state in which only a single object resides in an object zone;

a crossover state in which a plurality of objects correspond to a single object zone; and

a state of parting that is a transient state in which a single object zone is parted into a plurality of object zones.

7. An object-tracking device according to claim 4, wherein said characteristic-quantity generating means

generates zone characteristic quantities, each including at least one of or one of combinations of a color histogram, area, image template and color histogram normalized with respect to said area, of the object zone, and

finds an object zone corresponding to the object of interest from the first zone-correspondence information and provides at least one or one of combinations of a color histogram, area, image template and color histogram normalized with respect to said area of the object zone as an object characteristic quantity.

8. An object-tracking device according to claim 4, wherein said state-of-tracking deciding means includes an object-zone storing means for storing the object-zone information,

an object-tracking means for tracking an object based on said object-zone information, the correspondence information that has been determined and the object-zone information prior to the present that is provided from said object-zone storing means and further providing a second zone-correspondence information that indicates the correspondences between objects and object zones, and

a state-deciding means for deciding the states of tracking of objects based on said second zone-correspondence information, said object-zone information and said object-zone information prior to the present and providing said first zone-correspondence information.

9. An object-tracking device according to claim 8, wherein said state-deciding means, based on at least one of or one of the combinations of the correspondences between objects and object zones, distances between object zones and continued periods of separation of said object zones, obtained from said second zone-correspondence information and object-zone information, groups the objects that have a common region in their corresponding object zones to sort the objects and corresponding object zones into one class, and sorts the object, which differs in the corresponding object zone from any other objects, and the object zone corresponding thereto into one class to sort the objects and object zones into a plurality of classes, and decides the state of tracking on the basis of the sorted classes.

10. An object-tracking device according to claim 9, wherein said state of tracking includes the state of parting that is a transient state through which an object zone parts into a plurality of object zones,

said state-deciding means decides that, if two or more object zones are included in a sorted class, then the class meets the condition of being in a state of parting, and that, if a class meets the condition of being in a state of parting, the states of tracking of the objects and object zones included in the class are the state of parting.

11. An object-tracking device according to claim 10, wherein if the sorted class meets the condition of being in the state of parting and if the sorted class meets at least one of or one of the combinations of the conditions that two or more objects are included in said class, that each of the distances between the object zones included in said class exceeds a predetermined threshold and that continued periods of separation of the object zones included in said class exceed a predetermined threshold, said state-deciding means decides that the states of tracking of the objects and object zones included in the class are said state of parting.

12. An object-tracking device according to claim 10, wherein said state of tracking includes a state of parting and a stand-alone state in which a single object resides in an object zone, and

if the sorted class includes only one object and if the states of tracking of the object and the object zone included in said class are not the state of parting, then said state-deciding means decides that the states of tracking of the object and the object zone included in said class are the stand-alone state.

13. An object-tracking device according to claim 11, wherein said state of tracking includes a state of parting and a stand-alone state in which a single object resides in an object zone, and

if the sorted class includes only one object and if the states of tracking of the object and the object zone included in said class are not the state of parting, then said state-deciding means decides that the states of tracking of the object and the object zone included in said class are the stand-alone state.

14. An object-tracking device according to claim 10, wherein said state of tracking includes a state of parting and also a crossover state in which a plurality of objects are in corresponding relationship to a single object zone, and

if a sorted class includes two or more objects and if the states of tracking of the objects and the object zones included in said class are not the state of parting, said state-deciding means decides that the states of tracking of the objects and the object zones included in said class are the crossover state.

15. An object-tracking device according to claim 11, wherein said state of tracking includes a state of parting and also a crossover state in which a plurality of objects are in correspondence relation to a single object zone, and if a sorted class includes two or more objects and if the states of tracking of the objects and the object zones included in said class are not the state of parting, said state-deciding means decides that the states of tracking of the objects and the object zones included in said class are the crossover state.

16. An object-tracking device according to claim 4, wherein said characteristic-quantity generating means includes:

a characteristic-quantity extracting means for extracting zone characteristic quantities from the image information, object-zone information and the first zone-correspondence information and providing the zone characteristic-quantity information that is the information indicative of said zone characteristic quantities;

characteristic-quantity storing means for storing object characteristic quantities and selecting the stored object characteristic quantities to supply the selected object characteristic quantities, as required, and characteristic-quantity updating means for updating said object characteristic quantities stored in said characteristic-quantity storing means based on said zone characteristic-quantity information, said first zone-correspondence information or correspondence information that has been determined and the object characteristic quantities generated prior to the present.

17. An object-tracking device according to claim 16, wherein

said state of tracking includes the state of parting that is a transient state through which an object zone parts into a plurality of object zones, and

said characteristic-quantity extracting means includes, in zone characteristic-quantity information, the information indicating that there is no need for establishing correspondences to objects for the object zones that represent the states other than the state of parting while in their tracking states, and

said correspondence-determining means excludes, from the establishment of the corresponding relationship, the object zones indicated in said zone characteristic-quantity information as there is no need to establish corresponding relationship to objects.

18. An object-tracking device according to claim 16, wherein said state of tracking includes a stand-alone state in which a single object resides in an object zone, and

said characteristic-quantity updating means

decides whether or not the state of tracking of an object is the stand-alone state on the basis of the first zone-correspondence information or the correspondence information that has been determined, and

if the state of tracking of the object is the state other than the stand-alone state, does not update the object characteristic quantities stored in said characteristic-quantity storing means.

19. An object-tracking device according to claim 4, wherein said characteristic-quantity synthesizing means

determines all possible combinations of objects and object zones based on the object characteristic quantities generated by said characteristic-quantity generating means and the first zone-correspondence information, and

synthesizes object characteristic quantities only for the determined combinations of objects and object zones to generate synthesized characteristic quantities.

20. An object-tracking device according to claim 4, wherein characteristic-quantity synthesizing means calculates the synthesis ratios that are coefficients for adjusting the ratios at which the object characteristic quantities are synthesized, and generates synthesized characteristic quantities on the basis of said synthesis ratios and object characteristic quantities.

21. An object-tracking device according to claim 4, wherein said characteristic-quantity synthesizing means receives zone characteristic quantities as well as object characteristic quantities from the characteristic-quantity generating means, calculates synthesized characteristic quantities depending on desired synthesis ratios on the basis of the received zone characteristic-quantity information and object characteristic quantities, and provides the synthesized characteristic quantity for the synthesis ratio that yields the highest of all degrees of similarity between the calculated synthesized characteristic quantities and the zone characteristic quantities.

22. An object-tracking device according to claim 4, wherein said state of tracking includes a state of parting that is a transient state through which an object zone parts into a plurality of object zones, and

said characteristic-quantity synthesizing means generates synthesized characteristic quantities only for the object zones that are indicated as having the state of parting as their states of tracking.

23. An object-tracking device according to claim 4, wherein said object characteristic quantity includes an area of an object, and

said characteristic-quantity synthesizing means calculates the synthesis ratios that are coefficients for adjusting the ratios at which the object characteristic quantities are synthesized on the basis of the areas of objects included in said object characteristic quantities and generates synthesized characteristic quantities from said synthesis ratios and said object characteristic quantities.

24. An object-tracking device according to claim 23, wherein said characteristic-quantity synthesizing means limits the synthesis ratios within a predetermined range on the basis of the variations in the areas of objects.

25. An object-tracking device according to claim 4, wherein said characteristic-quantity synthesizing means receives zone characteristic quantities together with object characteristic quantities from the characteristic-quantity generating means, calculates synthesized characteristic quantities within the range of the variations in the areas of objects based on the received zone characteristic quantities and object characteristic quantities, and provides the synthesized characteristic quantities that have the highest degrees of similarity to the zone characteristic quantities of the object zones of interest.

26. An object-tracking device according to claim 4, wherein said object characteristic quantity includes an image template representative of the shape and color of an object, and

said characteristic-quantity synthesizing means decides the back-to-belly relation of each of the objects from the image templates and zone characteristic quantities and obtains the synthesized characteristic quantities by synthesizing the image templates based on the respective decided back-to-belly relations of said objects.

27. An object-tracking device according to claim 4, wherein said correspondence-determining means is provided with

a correspondence-calculating means for calculating the combination of objects and object zones which have the highest similarity from all the possible combinations of the objects and object zones that are possibly associated in corresponding relationship based on said synthesized characteristic-quantity information, said zone characteristic-quantity information and said first zone-correspondence information, selecting the calculated combination of objects and object zones as an optimum combination and generating the optimum-correspondence information that indicates the optimum corresponding relationship between objects and object zones, and

a correspondence-deciding means for determining the corresponding relationship between objects and object zones on the basis of said first zone-correspondence information and said optimum-correspondence information and providing the correspondence information that has been determined that is the information that includes the corresponding relationship that has been decided between objects and object zones.

28. An object-tracking device according to claim 27, wherein said correspondence-calculating means calculates a total degree of similarity for each of all the possible combinations of objects and object zones, said total degree of similarity being a sum of the degrees of similarity between the characteristic quantities of object zones and synthesized characteristic quantities within each combination, and decides the combination that has the highest similarity based on the combination having the highest, total degree of similarity, of said all the possible combinations.

29. An object-tracking device according to claim 27, wherein said first zone-correspondence information includes the information about an at-rest/in-motion state that indicates whether an object zone is at rest or in motion, and

said correspondence-calculating means excludes the combination of the object and object zone that is indicated as being at rest in said information about an at-rest/in-motion state from said all possible combinations.

30. An object-tracking device according to claim 27, wherein if the degrees of combined similarity that can be obtained from the degrees of similarity of the sets of the objects and object zones, said sets of the objects and object zones making up the combinations decided to be ranked as the highest similarity, are equal to or lower than a predetermined threshold, then said correspondence-calculating means selects the combinations of the degrees of combined similarity within said predetermined threshold, from the combinations of the degrees of combined similarity ranked as the highest similarity of all possible combinations of objects and object zones, includes the corresponding relationship of objects and object zones common to the selected combinations, into the optimum-correspondence information as optimum correspondences, and further, for the objects and object zones having the corresponding relationship that are not included in said corresponding relationship of the object and object zone common to said selected combinations, includes the information indicating that there are no optimum correspondence between the objects and object zones, into the optimum-correspondence information,

for the objects not indicated as having no optimum corresponding relationship to any object zones in said optimum-correspondence information, said correspondence-deciding means provides the information indicating the corresponding relationship of objects and object zones included in said optimum-correspondence information as the correspondence information that has been determined, and

for the objects indicated as having no optimum corresponding relationship to any object zones in said optimum-correspondence information, said correspondence-deciding means provides the information indicating the corresponding relationship of objects and object zones included in said first zone-correspondence information as the correspondence information that has been determined.

31. An object-tracking device according to claim 28, wherein if the degrees of combined similarity that can be obtained from the degrees of similarity of the sets of the objects and object zones, said sets of the objects and object zones making up the combinations decided to be ranked as the highest similarity, are equal to or lower than a predetermined threshold, then said correspondence-calculating means selects the combinations of the degrees of combined similarity within said predetermined threshold, from the combinations of the degrees of combined similarity ranked as the highest similarity of all possible combinations of objects and object zones, includes the corresponding relationship of objects and object zones common to the selected combinations, into the optimum-correspondence information as optimum correspondences, and further, for the objects and object zones having the corresponding relationship that are not included in said corresponding relationship of the object and object zone common to said selected combinations, includes the information indicating that there are no optimum correspondence between the objects and object zones, into the optimum-correspondence information,

for the objects not indicated as having no optimum corresponding relationship to any object zones in said optimum-correspondence information, said correspondence-deciding means provides the information indicating the corresponding relationship of objects and object zones included in said optimum-correspondence information as the correspondence information that has been determined, and

for the objects indicated as having no optimum corresponding relationship to any object zones in said optimum-correspondence information, said correspondence-deciding means provides the information indicating the corresponding relationship of objects and object zones included in said first zone-correspondence information as the correspondence information that has been determined.

32. An object-tracking device according to claim 27, wherein said state of tracking includes a state of parting that is a transient state through which an object zone parts into a plurality of object zones, and

said correspondence-deciding means determines the corresponding relationship between objects and object zones to be indicated in the optimum-correspondence information only for the object zones that exhibit a state of parting as their states of tracking.

33. An object-tracking device according to claim 27, wherein said state of tracking includes a state of parting that is a transient state through which an object zone parts into a plurality of object zones, and

said correspondence-deciding means provides the correspondences between objects and object zones included in the first zone-correspondence information as the correspondence information that has been determined only for the object zones that exhibit states other than the state of parting while in their tracking states.

34. An object-tracking method for tracking an object based on image information, comprising steps of:

synthesizing object characteristic quantities, which represent characteristic quantities of respective objects included in said image information, to generate a synthesized characteristic quantity, and

establishing corresponding relationship between object or objects and object zone on the basis of the degree of similarity between said synthesized characteristic quantity and characteristic quantity of said object zone, wherein said object zone is a region extracted from said image information and including said object or objects.

35. An object-tracking method according to claim 34, including steps of

synthesizing characteristic quantities for each of all required combination of a plurality of objects to generate a synthesized characteristic quantity, and

establishing corresponding relationship between object or objects and object zone through comparison of the generated synthesized characteristic quantity and zone characteristic quantity that represents the characteristic quantity of said object zone.

36. An object-tracking method according to claim 35, including steps of

extracting an object zone from said image information and providing the object-zone information that includes image information about said object zone,

deciding a state of tracking representative of a relative position with respect to another object for every object or object zone,

generating zone characteristic quantities, object characteristic quantities through the use of said image information, said object-zone information, and the decision results, and

generating synthesized characteristic quantities through the use of said object characteristic quantities and said decision results.

37. An object-tracking method according to claim 36, including steps of

deciding the state of tracking of every object or every object zone based on said object-zone information and the correspondence information that has been determined that indicates the corresponding relationship between object zones and objects prior to the present and providing the first zone-correspondence information indicative of the corresponding relationship between the objects and object zones and the states of tracking,

generating zone characteristic quantities and object characteristic quantities based on the present image information, said object-zone information, said first zone-correspondence information and said correspondence information that has been determined,

generating a synthesized characteristic quantity that functions as a candidate to be placed in corresponding relationship to each object zone on the basis of said object characteristic quantities and said first zone-correspondence information and providing the synthesized characteristic-quantity information, which is the information that includes said synthesized characteristic quantities and the corresponding relationship between synthesized characteristic quantities and objects used for generating the synthesized characteristic quantities of interest, and

establishing correspondences between objects and object zones based on said first zone-correspondence information, zone characteristic-quantity information, which is the information indicative of said zone characteristic quantities, and said synthesized characteristic-quantity information, and providing said correspondence information that has been determined at present.

38. An object-tracking method according to claim 36, wherein said state of tracking includes at least one of or one of combinations of a stand-alone state, in which only a single object is present in an object zone, a crossover state, in which a plurality of objects are present in a single object zone, and a state of parting that is a transient state through which an object zone parts into a plurality of object zones.

39. An object-tracking method according to claim 37, wherein said state of tracking includes at least one of, or one of combinations of a stand-alone state, in which only a single object is present in an object zone, a crossover state, in which a plurality of objects are present in a single object, and a state of parting that is a transient state through which an object zone parts into a plurality of object zones.

40. An object-tracking method according to claim 37, including steps of

generating, as a zone characteristic quantity, at least one of, or one of the combinations of the color histograms, areas, image templates and color histograms normalized with respect to respective areas, of object zones, and

seeking the object zones corresponding to said objects from said first zone-correspondence information, and generating, as an object characteristic quantity, at least one of, or one of the combinations of the color histograms, areas, image templates and color histograms normalized with respect to respective areas of said object zones.

41. An object-tracking method according to claim 37, including steps of

storing said object-zone information,

tracking an object on the basis of said object-zone information, the correspondence information that has been determined and the object-zone information prior to the present and providing a second zone-correspondence information indicative of the correspondence between the object and object zone, and

deciding the state of tracking an object on the basis of said second zone-correspondence information, said object-zone information and said object-zone information prior to the present and providing said first zone-correspondence information.

42. An object-tracking method according to claim 41, including steps of

sorting objects and object zones into a plurality of classes by:

grouping the objects based on those objects which have a common region in the corresponding object zone, to enroll said objects and the corresponding object zones in one class; and

for the objects that correspond to the object zones that differ from the object zones corresponding to any other objects, enrolling the objects and the corresponding object zone in one class, based on at least one of, or one of combinations of the corresponding relationship between objects and object zones, the distances between object zones, and the duration period for on-parting of object zones calculated from said second zone-correspondence information and said object-zone information; and

deciding the state of tracking based on the classified class.

43. An object-tracking method according to claim 42, wherein said state of tracking includes a state of parting that is a transient state through which a single object zone parts into a plurality of object zones, and

said object-tracking method includes steps of

deciding that a classified class meets the condition of being in the state of parting if the class includes two or more object zones, and

if a class meets the condition of being in the state of parting, deciding the state of parting based on the states of tracking the objects and object zones included in the class of interest.

44. An object-tracking method according to claim 43, including a step of:

if the classified class meets the condition of being in the state of parting and further meets at least one of, or one combination of the conditions that said class includes two or more objects, that the distances between the object zones included in said class exceed a predetermined threshold, and that the continued periods of separation of the object zones included in said class exceed a predetermined threshold, deciding the state of parting based on the states of tracking the objects and object zones included in said class.

45. An object-tracking method according to claim 43, wherein said state of tracking includes the state of parting and a stand-alone state in which a single object is present in an object zone, and said object-tracking method includes a step of

deciding that the states of tracking of the object and object zone are the stand-alone state if the classified class includes a single object and also neither of the states of tracking of the object and the object zone included in the class is the state of parting.

46. An object-tracking method according to claim 44, wherein said state of tracking includes the state of parting and a stand-alone state in which a single object is present in an object zone, and said object-tracking method includes a step of

deciding that the states of tracking of the object and object zone are the stand-alone state if the classified class includes a single object and also neither of the states of tracking of the object and the object zone included in the class is the state of parting.

47. An object-tracking method according to claim 43, wherein said state of tracking includes the state of parting and the crossover state in which a plurality of objects correspond to a single object zone, and said object-tracking method

includes a step of deciding that the states of tracking of the objects and object zone included in the classified class are the crossover state if the class includes two or more objects and neither of the states of tracking of the objects and the object zone included in the class is the state of parting.

48. An object-tracking method according to claim 44, wherein said state of tracking includes the state of parting and the crossover state in which a plurality of objects correspond to a single object zone, and said object-tracking method includes a step of

deciding that the states of tracking of the objects and object zone included in the classified class are the crossover state if the class includes two or more objects and neither of the states of tracking of the objects and the object zone included in the class is the state of parting.

49. An object-tracking method according to claim 37, including steps of

extracting zone characteristic quantities from said image information, said object-zone information and said first zone-correspondence information and providing the zone characteristic-quantity information, which is the information indicative of said zone characteristic quantities,

storing said object characteristic quantities and selecting the stored object characteristic quantities to be provided as required, and

updating said stored object characteristic quantities on the basis of said zone characteristic-quantity information, said first zone-correspondence information or the correspondence information that has been determined and the object characteristic quantities generated prior to the present.

50. An object-tracking method according to claim 49, wherein said state of tracking includes the state of parting that is a transient state through which an object zone parts into a plurality of object zones, and said object-tracking method includes steps of

including, into the zone characteristic-quantity information, the information which indicates that there is no need to establish corresponding relationship to any objects, for the object zones that are indicated as having states other than the state of parting while in their tracking states, and

excluding the object zones, in which the zone-correspondence information indicates that there is no need of establishing the corresponding relationship to any objects, from the establishment of corresponding relationship.

51. An object-tracking method according to claim 49, wherein said state of tracking includes a stand-alone state in which a single object is present in an object zone, and said object-tracking method includes steps of

deciding whether or not the state of tracking is the stand-alone state based on said first zone-correspondence information or correspondence information that has been determined, and

skipping an update of the stored object characteristic quantity if the state of tracking of an object is any of the states other than the stand-alone state.

52. An object-tracking method according to claim 37, including steps of

determining all possible combinations of objects and object zones on the basis of said object characteristic quantities and said first zone-correspondence information, and

synthesizing object characteristic quantities to generate synthesized characteristic quantities only for the determined combinations of objects and object zones.

53. An object-tracking method according to claim 37, including steps of

finding synthesis ratios, which are the coefficients for adjusting the ratios of said object characteristic quantities to be synthesized, and

generating synthesized characteristic quantities on the basis of said synthesis ratios and object characteristic quantities.

54. An object-tracking method according to claim 37, including steps of

receiving zone characteristic quantities together with object characteristic quantities, calculating synthesized characteristic quantities for arbitrary synthesis ratios based on the received zone characteristic-quantity information and the object characteristic quantities, and providing the synthesized characteristic quantity corresponding to the highest degree of similarity between the zone characteristic quantity and the calculated synthesized characteristic quantity.

55. An object-tracking method according to claim 37, wherein said state of tracking includes a state of parting, said state of parting being a transient state through which an object zone parts into a plurality of object zones, and said object-tracking method includes a step of

generating synthesized characteristic quantities only for the object zones indicated as being in the state of parting while in their tracking states.

56. An object-tracking method according to claim 37, wherein said object characteristic quantity includes an area of an object, and said object-tracking method includes steps of

calculating a synthesis ratio, which is a coefficient for adjusting the ratios of the object characteristic quantities to be synthesized, on the basis of the areas of objects and generating a synthesized characteristic quantity from the calculated synthesis ratio and object characteristic quantities.

57. An object-tracking method according to claim 56, wherein said synthesis ratio is restricted within a range predetermined on the basis of the variations of the object areas.

58. An object-tracking method according to claim 37, including steps of

receiving said zone characteristic quantities together with said object characteristic quantities, generating synthesized characteristic quantities within the range of variations in the object areas based of the received zone characteristic quantities and the object characteristic quantities, and providing the synthesized characteristic quantity which has the highest degree of similarity to the zone characteristic quantity of the object zone of interest.

59. An object-tracking method according to claim 37, wherein said object characteristic quantity includes an image template, which describes a shape and/or color of an object, and said object-tracking method includes steps of

deciding the back-to-belly relations of the objects on the basis of the image templates and zone characteristic quantities, and synthesizing the image templates based on said decided back-to-belly relations of the objects to obtain a synthesized characteristic quantity.

60. An object-tracking method according to claim 37, including steps of

calculating, based on said synthesized characteristic-quantity information, said zone characteristic-quantity information and said first zone-correspondence information, the combination of objects and object zones which has the highest degree of similarity from all the possible combinations of objects and object zones that can be associated in corresponding relationship, selecting the calculated combination of objects and object zones as the objects and object zone of optimum correspondence, and generating the optimum-correspondence information that indicates the optimum correspondence relation of the objects and object zone, and

determining corresponding relationship between objects and object zones based on said first zone-correspondence information and said optimum-correspondence information and providing the correspondence information that has been determined, which is the information inclusive of the determined corresponding relationship between objects and object zones.

61. An object-tracking method according to claim 60, including steps of

calculating a total degree of similarity for all possible combinations of objects and object zones, wherein said total degree of similarity is a sum of the degrees of similarity between characteristic quantities of object zones and synthesized characteristic quantities in each of the combinations, and deciding that the combination of the highest total degree of similarity of all said possible combinations is the combination of the highest similarity.

62. An object-tracking method according to claim 60, wherein said first zone-correspondence information includes the information about an at-rest/in-motion state that indicates whether an object zone is at rest or in motion, and said object-tracking method includes a step of excluding, from all the possible combinations of objects and object zones, the combination of the object and object zone indicated as being in the at-rest state by said information about an at-rest/in-motion state.

63. An object-tracking method according to claim 60, including steps of:

if the degrees of combined similarity, obtained from the degrees of similarity of the sets of objects and an object zones that make up the combination decided as a combination of the highest similarity, is equal to or lower than a predetermined threshold, then selecting the combinations of the degrees of combined similarity within a predetermined threshold from the combinations of the degrees of combined similarity ranked as the highest similarity, of all the possible combinations of objects and object zones, including the corresponding relationship between objects and object zones common to the selected combinations into the optimum-correspondence information as optimum correspondences, and further including the information that indicates absence of optimum correspondence between any object and object zone in the optimum-correspondence information, for the object and object zone in the correspondence relation that is not included in the corresponding relationship of the objects and object zones common to said selected combinations;

for the objects not indicated in said optimum-correspondence information that are absent from the optimum corresponding relationship to object zones, providing the information indicating the corresponding relationship between the objects of interest and object zones included in said optimum-correspondence information, as the correspondence information that has been determined; and

for the objects indicated in said optimum-correspondence information that are absent from the optimum corresponding relationship to object zones, providing the information indicating the corresponding relationship between the objects of interest and object zones included in the first zone-correspondence information, as the correspondence information that has been determined.

64. An object-tracking method according to claim 61, including steps of:

if the degrees of combined similarity, obtained from the degrees of similarity of the sets of objects and object zones that make up the combination that is decided as the combination of the highest similarity, is equal to or lower than a predetermined threshold, then selecting the combinations of the degrees of combined similarity within a predetermined threshold from the combinations of the degrees of combined similarity ranked as having the highest similarity, from all the possible combinations of objects and object zones, including the corresponding relationship between objects and object zones common to the selected combinations in the optimum-correspondence information as optimum correspondences, and further including the information that indicates absence of an optimum correspondence between any object and object zone in the optimum-correspondence information, for the object and object zone in the correspondence relation that is not included in the corresponding relationship of the objects and object zones common to said selected combinations;

for the objects not indicated in said optimum-correspondence information that are absent from the optimum corresponding relationship to object zones, providing information indicating the corresponding relationship between the objects of interest and object zones included in said optimum-correspondence information, as the correspondence information that has been determined; and

for the objects indicated in said optimum-correspondence information that are absent from the optimum corresponding relationship to object zones, providing the information indicating the corresponding relationship between the objects of interest and object zones included in the first zone-correspondence information, as the correspondence information that has been determined.

65. An object-tracking method according to claim 60, wherein said state of tracking includes a state of parting, said state of parting being a transient state through which an object zone parts into a plurality of object zones, and said object-tracking method includes a step of

determining the corresponding relationship between objects and object zones to be identical with those indicated in the optimum-correspondence information only for the object zones that are indicated as having the state of parting while in their tracking states.

66. An object-tracking method according to claim 60, wherein said state of tracking includes a state of parting, said state of parting being a transient state through which an object zone parts into a plurality of object zones, and said object-tracking method includes a step of

providing the correspondences between objects and object zones included in the first zone-correspondence information as the correspondence information that has been determined only for the object zones indicated as having the state other than the state of parting while in their tracking states.

67. An object-tracking program for tracking an object based on image information, said program operating a computer to execute processes of

receiving image information,

synthesizing object characteristic quantities that represent characteristic quantities of respective objects included in said received image information and generating a synthesized characteristic quantity, and

establishing a correspondence between said objects and an object zone based on the degree of similarity between a characteristic quantity of said object zone and said synthesized characteristic quantity, said object zone being a region that is extracted from said image information and also includes said objects.

68. An object-tracking program for establishing correspondences between objects and object zones included in received image information, said program operating a computer to execute processes of

receiving image information,

extracting object zones from said received image information and providing the object-zone information inclusive of image information about said object zones,

deciding the state of tracking with each object or object zone on the basis of said object-zone information and the correspondence information that has been determined indicating corresponding relationship of the objects and object zones prior to the present and providing first zone-correspondence information, which indicates the corresponding relationship of said object zones and objects and the states of tracking,

generating the zone characteristic quantities, which represent the characteristic quantities of the object zones, and the object characteristic quantities, which represent the characteristic quantities of the objects through the use of said image information, said object-zone information and said first zone-correspondence information,

synthesizing characteristic quantities for all required combinations of a plurality of objects to generate each of synthesized characteristic quantities based on said object characteristic quantities and said first zone-correspondence information, and providing synthesized characteristic-quantity information, which is the information that includes said synthesized characteristic quantities and the corresponding relationship between the objects used for generating synthesized characteristic quantities and the synthesized characteristic quantities, and

associating said objects and object zones in corresponding relationship based on said first zone-correspondence information, said zone characteristic-quantity information and said synthesized characteristic-quantity information, and providing said correspondence information that has been determined for the present.

69. An object-tracking program for establishing correspondences between objects and object zones included in received image information, said program operating a computer to execute processes of

receiving image information,

deciding the state of tracking each object or each object zone on the basis of said object-zone information and the correspondence information that has been determined indicating the corresponding relationship of the objects and object zones prior to the present, and providing first zone-correspondence information, which indicates the corresponding relationship of said object zones and objects and the states of tracking,

generating the zone characteristic quantities, which represent the characteristic quantities of the object zones, and the object characteristic quantities, which represent the characteristic quantities of the objects through the use of said image information, said object-zone information and said first zone-correspondence information, and

while taking each of the objects as a target, deciding the correspondences between said objects and object zones through designating the correspondences between objects and object zones included in said first zone-correspondence information as said correspondences between said objects and object zones, for the objects that are decided to be in states other than a state of parting while in their tracking states, said state of parting being a transient state through which an object zone parts into a plurality of object zones;

for the objects that are decided to be in the state of parting while in their tracking states, synthesizing characteristic quantities for all required combinations of a plurality of objects on the basis of said object characteristic quantities and said first zone-correspondence information to generate respective synthesized characteristic quantities; and

comparing each of synthesized characteristic quantities with a zone characteristic quantity to associate the objects corresponding to the synthesized characteristic quantity, which has the highest degree of similarity to an object zone, with the object zone of interest to be placed in corresponding relationship.