Method and apparatus for processing pictures of mobile object

• US 20020196341A1
• Filed: 04/12/2002
• Published: 12/26/2002
• Est. Priority Date: 06/21/2001
• Status: Active Grant

• Current Assignee: Socionext Inc., The Foundation for the Promotion of Industrial Science
• Original Assignee: Fujitsu Limited
• Inventors: Ikeuchi, Katsushi, Kamijo, Shunsuke, Sakauchi, Masao
• Granted Patent: US 7,298,394 B2
• Time in Patent Office: Days
• US Class Current: 348/149
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/231 using full search
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G08G 1/04 using optical or ultrasonic detectors
• CPC Class Codes: G08G 1/164 Centralised systems, e.g. external to vehicles

Abstract

A feature amount of an inter-mobile unit relative movement are detected as an observation amount by an observation amount detecting section 26, time series of the observation amounts are stored as an observation series into a storage section 27 to calculate a similarity of the observation series to a predetermined collision observation series by a classification section 28. A determination section 29 determines to be a collision accident if, in a case where the similarity is larger than a predetermined value, a mobile unit associated with the similarity is at rest in a stoppage prohibition area set in a storage section 30 and another mobile unit is moving, and to be a mobile unit failure if collision determination conditions except for the similarity are met. By consisting of not only a first scalar obtained by quantizing a relative velocity vector between mobile units but also a second scalar obtained by quantizing a relative position vector between mobile units as the observation amount, a relative movement between mobile units is classified in more detail. A mobile unit is tracked in units of block by a mobile unit tracking section 25 to discriminate overlapped mobile units in pictures.

Claims

1. A mobile unit anomaly detection method of processing time series pictures to detect an anomaly of a mobile unit in a picture, comprising the steps of:

(a) detecting a feature amount of a relative movement of a second mobile unit with respect to a first mobile unit as an observation amount to store a time series of said observation amounts as an observation series;

(b) calculating a similarity of said observation series to a predetermined collision observation series; and

(c) when said similarity is larger than a predetermined value, determining whether or not a collision accident has happened on the basis of whether or not said first or second mobile unit is at rest in a predetermined stoppage prohibition area.

2. The mobile unit anomaly detection method according to claim 1, wherein in said step (c), when said similarity is larger than said predetermined value, it is determined to be said collision accident when said first or second mobile unit is at rest in said predetermined stoppage prohibition area and another mobile unit is moving.

3. The mobile unit anomaly detection method according to claim 1, further comprising the step of;

(d) when said similarity is smaller than said predetermined value, determining whether or not said first or second mobile unit at rest is in failure on the basis of whether said first or second mobile unit is at rest in said predetermined stoppage prohibition area.

4. The mobile unit anomaly detection method according to claim 3, wherein in said step (d), when said similarity is smaller than said predetermined value, it is determined that said first or second mobile unit at rest is in failure when said first or second mobile unit is at rest in said predetermined stoppage prohibition area and another mobile unit is moving.

5. A mobile unit anomaly detection apparatus comprising:

a picture storage device storing time series pictures; and

a picture processing device processing said time series pictures to detect an anomaly of a mobile unit in at least one of said pictures;

wherein said picture processing device comprises;

an observation amount detecting section detecting a feature amount of a relative movement of a second mobile unit with respect to a first mobile unit as an observation amount;

an observation series storage section storing a time series of said observation amounts as an observation series;

a stoppage prohibition area storage section storing a predetermined stoppage prohibition area;

a similarity calculation section calculating a similarity of said observation series to a predetermined collision observation series; and

a determination section, when said similarity is larger than a predetermined value, determining whether or not a collision accident has happened on the basis of whether or not said first or second mobile unit is at rest in said predetermined stoppage prohibition area, and outputs a result of the determination.

6. The mobile unit anomaly detection apparatus according to claim 5, wherein said determination section, when said similarity is larger than said predetermined value, determines to be said collision accident when said first or second mobile unit is at rest in said predetermined stoppage prohibition area and another mobile unit is moving.

7. The mobile unit anomaly detection apparatus according to claim 5, wherein said determination section, when said similarity is smaller than said predetermined value, determines whether or not said first or second mobile unit at rest is in failure on the basis of whether said first or second mobile unit is at rest in said predetermined stoppage prohibition area, and outputs a result of the determination.

8. The mobile unit anomaly detection apparatus according to claim 7, wherein said determination section, when said similarity is smaller than said predetermined value, determines that said first or second mobile unit at rest is in failure when said first or second mobile unit is at rest in said predetermined stoppage prohibition area and another mobile unit is moving.

9. An inter-mobile unit movement classification method of processing time series pictures to classify a movement between mobile units in at least one of said pictures, comprising the steps of:

(a) detecting a feature amount of a relative movement of a second mobile unit with respect to a first mobile unit as an observation amount to store a time series of said observation amounts as an observation series;

(b) calculating a similarity of said observation series to a reference series; and

(c) classifying a movement of said second object with respect to said first mobile unit according to a value of said similarity;

wherein said observation amounts each include;

a first scalar obtained by quantizing an amount associated with both a relative velocity V of said second mobile unit with respect to said first mobile unit and a distance d between said first and second mobile units; and

a second scalar obtained by quantizing a relative position vector of said second mobile unit with respect to said first mobile unit.

10. The inter-mobile unit movement classification method according to claim 9, wherein said first scalar is a value obtained by quantizing V·

f (d), where f (d) is a monotone decreasing function of said distance d.

11. The inter-mobile unit movement classification method according to claim 10, wherein said f(d) is expressed as k/(d+ε

), where ε and

k are positive constants.

12. The inter-mobile unit movement classification method according to claims 9, wherein said similarity is a probability that said observation series arises, said probability is a value calculated on the basis of a hidden Markov model, and said reference series is a learning series for determining a model parameter.

14. The inter-mobile unit movement classification apparatus according to claim 13, wherein said first scalar is a value obtained by quantizing V·

f (d), where f (d) is a monotone decreasing function of said distance d.

15. The inter-mobile unit movement classification apparatus according to claim 14, wherein said f(d) is expressed as k/(d+ε

), where ε and

k are positive constants.

18. The mobile unit identification method according to claim 17,

wherein said evaluation value of said step (a) includes a sum over p=1 to Na of a value associated with a value |VCm(t−

1)−

VBp(t−

1)| on the assumption that an identification code of said block i at the time t is IDj, where VCm(t−

1) denotes a motion vector of a block m at the time (t−

1), said block m at the time (t−

1) corresponds to said block i at the time t, and VBp(t−

1) denotes a motion vector of a block whose identification code is IDj and which is adjacent to said block m at the time (t−

1),

wherein said evaluation value of said step (b) includes a sum over q=1 to Nb of a value associated with a value |VCn(t−

1)−

VBq(t−

1)| on the assumption that an identification code of said block i at the time t is IDk, where VCn(t−

1) denotes a motion vector of a block n at the time (t−

1), said block n at the time (t−

1) corresponds to said block i at the time t, and VBq(t−

1) denotes a motion vector of a block whose identification code is IDk and which is adjacent to said block n at the time (t−

1).

19. The mobile unit identification method according to claim 17, further comprising the step of:

prior to processing on said frame picture at the time t, in said frame picture at the time (t−

1), replacing a motion vector of each block having an assigned identification code IDx with a mean motion vector obtained by averaging motion vectors of this block and blocks adjacent to this block and having said identification code IDx.

13. An inter-mobile unit movement classification apparatus comprising:

a picture storage device storing time series pictures; and

a picture processing device processing said time series pictures to detect an anomaly of a mobile unit in at least one of said pictures;

wherein said picture processing device comprises;

an observation amount detecting section detecting a feature amount of a relative movement of a second mobile unit with respect to a first mobile unit as an observation amount;

an observation series storage section storing a time series of said observation amounts as an observation series;

a similarity calculation section calculating a similarity of said observation series to a reference series; and

a classification section classifying a movement of said second object with respect to said first mobile unit according to a value of said similarity;

wherein said observation amounts each include;

a first scalar obtained by quantizing an amount associated with both a relative velocity V of said second mobile unit with respect to said first mobile unit and a distance d between said first and second mobile units; and

a second scalar obtained by quantizing a relative position vector of said second mobile unit with respect to said first mobile unit.

16. The inter-mobile unit movement classification apparatus according to claims 13, wherein said similarity is a probability that said observation series arises, said probability is a value calculated on the basis of a hidden Markov model, and said reference series is a learning series for determining a model parameter.

17. A mobile unit identification method dividing each of time series pictures to blocks each including a plurality of pixels to process said pictures, wherein said method assigns identification codes of a plurality of mobile units included in a frame picture at a time t in units of the block, and obtains motion vectors of said plurality of mobile units in units of the block, in a case where identification codes of said plurality of mobile units included in a frame picture at a time (t−

1) have been assigned in units of the block, and motion vectors of said plurality of mobile units have been obtained in units of the block, said method comprises the steps of;

(a) moving a block j at the time (t−

1), whose identification code is IDj and whose motion vector is Vj, by the vector Vj to obtain a substantially corresponding block i at the time t and moving said block i by a vector −

Vj to obtain a first box at the time (t−

1), to calculate an evaluation value associated with a correlation between an image in said first box at the time (t−

1) and an image of said block i at the time t;

(b) moving a block k at the time (t−

1), whose identification code is IDk and whose motion vector is Vk, by the vector Vk to obtain a substantially corresponding block which is said block i at the time t and moving said block i by a vector −

Vk to obtain a second box at the time (t−

1), to calculate an evaluation value associated with a correlation between an image in said second box at the time (t−

1) and said image of said block i at the time t; and

(c) assigning said IDj or said IDk to said block i at the time t on the basis of magnitudes of said evaluation values calculated in the steps (a) and (b).

20. A mobile unit identification apparatus comprising:

a picture storage device storing time series pictures; and

a picture processing device, dividing each of said time series pictures stored to blocks each including a plurality of pixels to process said pictures, assigning identification code of a plurality of mobile units included in a frame picture at a time t in units of the block, and obtaining motion vectors of said plurality of mobile units in units of the block, in a case where identification code of said plurality of mobile units included in a frame picture at a time (t−

1) have been assigned in units of the block, and motion vectors of said plurality of mobile units have been obtained in units of the block;

wherein said picture processing device performs processing of;

(a) moving a block j at the time (t−

1), whose identification code is IDj and whose motion vector is Vj, by the vector Vj to obtain a substantially corresponding block i at the time t and moving said block i by a vector −

Vj to obtain a first box at the time (t−

1), to calculate an evaluation value associated with a correlation between an image in said first box at the time (t−

1) and an image of said block i at the time t;

(b) moving a block k at the time (t−

1), whose identification code is IDk and whose motion vector is Vk, by the vector Vk to obtain a substantially corresponding block which is said block i at the time t and moving said block i by a vector −

Vk to obtain a second box at the time (t−

1), to calculate an evaluation value associated with a correlation between an image in said second box at the time (t−

1) and said image of said block i at the time t; and

(c) assigning said IDj or said IDk to said block i at the time t on the basis of magnitudes of said evaluation values calculated in the steps (a) and (b).

21. The mobile unit identification apparatus of claim 20,

wherein said evaluation value of said processing (a) includes a value associated with an absolute value of a difference between a motion vector of said block i at the time t on the assumption that an identification code of said block i at the time t is IDj,

wherein said evaluation value of said processing (b) includes a value associated with an absolute value of a difference between a motion vector of said block i at the time t on the assumption that an identification code of said block i at the time t is IDk and a motion vector of a block adjacent to said block i.