PEDESTRIAN BEHAVIOR PREDICTING DEVICE AND PEDESTRIAN BEHAVIOR PREDICTING METHOD

• US 20140219505A1
• Filed: 09/20/2011
• Published: 08/07/2014
• Est. Priority Date: 09/20/2011
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Inventors: Kindo, Toshiki, Ogawa, Masahiro, Funayama, Ryuji
• Granted Patent: US 9,904,846 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: B60W 2420/40 Photo, light or radio wave sensitive means, e.g. infrared sensors
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: G06V 40/25 Recognition of walking or running movements, e.g. gait recognition
• CPC Class Codes: G08G 1/08 according to detected number or speed of vehicles
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

According to the present invention, a pedestrian is detected from an imaged image and a partial image including the pedestrian is extracted, shape information of the pedestrian acquired from the extracted partial image is accumulated and the shape information of a predetermined time before and the current shape information are compared using the accumulated shape information to detect change in the movement of the pedestrian, discontinuous movement estimating information indicating a discontinuous movement of the pedestrian that occurs following the change in the movement of the pedestrian is acquired from a storage means at the time the change in the movement of the pedestrian is detected, and a behavior of the pedestrian is predicted using the acquired discontinuous movement estimating information.

Claims

1-10. -10. (canceled)

11. A pedestrian behavior predicting device comprising:

a pedestrian detecting unit configured to detect a pedestrian from an imaged image and extract a partial image including the pedestrian;

a movement change detecting unit configured to detect change in movement of the pedestrian by accumulating shape information of the pedestrian acquired from the partial image extracted by the pedestrian detecting unit and comparing the shape information of a predetermined time before and the current shape information using the accumulated shape information;

a discontinuous movement estimating unit configured to acquire discontinuous movement estimating information indicating a discontinuous movement of the pedestrian that occurs following the change in the movement of the pedestrian from a storage unit at the time the change in the movement of the pedestrian is detected by the movement change detecting unit; and

a behavior predicting unit configured to predict a behavior of the pedestrian using the discontinuous movement estimating information acquired by the discontinuous movement estimating unit.

12. The pedestrian behavior predicting device according to claim 11, wherein

the movement change detecting unit detects the change in the movement of the pedestrian by acquiring a feature amount distribution representing the shape information of the pedestrian acquired from the partial image extracted by the pedestrian detecting unit, normalizing the shape information represented by the acquired feature amount distribution, accumulating the normalized shape information, and comparing the shape information of a predetermined time before and the current shape information using the accumulated shape information.

13. The pedestrian behavior predicting device according to claim 11, wherein

the discontinuous movement estimating unit calculates an optical flow of the imaged image and acquires the discontinuous movement estimating information using the optical flow at the time the change in the movement of the pedestrian is detected by the movement change detecting unit.

14. The pedestrian behavior predicting device according to claim 12, wherein

the discontinuous movement estimating unit calculates an optical flow of the imaged image and acquires the discontinuous movement estimating information using the optical flow at the time the change in the movement of the pedestrian is detected by the movement change detecting unit.

15. The pedestrian behavior predicting device according to claim 11, wherein

the pedestrian detecting unit acquires a position of the pedestrian on the imaged image,

the pedestrian behavior predicting device further comprises a continuous movement estimating unit configured to generate continuous movement estimating information indicating a continuous movement of the pedestrian that occurs following movement of the position of the pedestrian based on a history of positions of the pedestrian acquired by the pedestrian detecting unit, and

the behavior predicting unit predicts the behavior of the pedestrian based on the continuous movement estimating information generated by the continuous movement estimating unit.

16. The pedestrian behavior predicting device according to claim 12, wherein

the pedestrian detecting unit acquires a position of the pedestrian on the imaged image,

the pedestrian behavior predicting device further comprises a continuous movement estimating unit configured to generate continuous movement estimating information indicating a continuous movement of the pedestrian that occurs following movement of the position of the pedestrian based on a history of positions of the pedestrian acquired by the pedestrian detecting unit, and

the behavior predicting unit predicts the behavior of the pedestrian based on the continuous movement estimating information generated by the continuous movement estimating unit.

17. The pedestrian behavior predicting device according to claim 13, wherein

the pedestrian detecting unit acquires a position of the pedestrian on the imaged image,

the pedestrian behavior predicting device further comprises a continuous movement estimating unit configured to generate continuous movement estimating information indicating a continuous movement of the pedestrian that occurs following movement of the position of the pedestrian based on a history of positions of the pedestrian acquired by the pedestrian detecting unit, and

the behavior predicting unit predicts the behavior of the pedestrian based on the continuous movement estimating information generated by the continuous movement estimating unit.

18. The pedestrian behavior predicting device according to claim 11, wherein

the discontinuous movement estimating information and the continuous movement estimating information include at least one of a moving direction and a moving speed of the pedestrian.

19. The pedestrian behavior predicting device according to claim 12, wherein

the discontinuous movement estimating information and the continuous movement estimating information include at least one of a moving direction and a moving speed of the pedestrian.

20. The pedestrian behavior predicting device according to claim 13, wherein

the discontinuous movement estimating information and the continuous movement estimating information include at least one of a moving direction and a moving speed of the pedestrian.

21. The pedestrian behavior predicting device according to claim 14, wherein

the discontinuous movement estimating information and the continuous movement estimating information include at least one of a moving direction and a moving speed of the pedestrian.

22. The pedestrian behavior predicting device according to claim 11, wherein

the movement change detecting unit further includes

a shape information acquiring unit configured to acquire the feature amount distribution of the pedestrian from the partial image extracted by the pedestrian detecting unit using a predetermined feature amount,

a normalizing unit configured to normalize the feature amount distribution acquired by the shape information acquiring unit and acquire a probability distribution corresponding to the feature amount distribution,

a shape information accumulating unit configured to accumulate the probability distribution acquired by the normalizing unit, and

a change detecting unit configured to calculate a difference between the probability distribution of a predetermined time before and the current probability distribution accumulated by the shape information accumulating unit using a predetermined scale, and detect the change in the movement of the pedestrian at the time the calculated difference is greater than a predetermined threshold value.

23. The pedestrian behavior predicting device according to claim 12, wherein

the movement change detecting unit further includes

a shape information acquiring unit configured to acquire the feature amount distribution of the pedestrian from the partial image extracted by the pedestrian detecting unit using a predetermined feature amount,

a normalizing unit configured to normalize the feature amount distribution acquired by the shape information acquiring unit and acquire a probability distribution corresponding to the feature amount distribution,

a shape information accumulating unit configured to accumulate the probability distribution acquired by the normalizing unit, and

a change detecting unit configured to calculate a difference between the probability distribution of a predetermined time before and the current probability distribution accumulated by the shape information accumulating unit using a predetermined scale, and detect the change in the movement of the pedestrian at the time the calculated difference is greater than a predetermined threshold value.

24. A pedestrian behavior predicting method comprising:

a pedestrian detecting step configured to detect a pedestrian from an imaged image and extract a partial image including the pedestrian;

a movement change detecting step configured to detect change in movement of the pedestrian by accumulating shape information of the pedestrian acquired from the partial image extracted in the pedestrian detecting step, and comparing the shape information of a predetermined time before and the current shape information using the accumulated shape information;

a discontinuous movement estimating step configured to acquire discontinuous movement estimating information indicating a discontinuous movement of the pedestrian that occurs following the change in the movement of the pedestrian from a storage unit at the time the change in the movement of the pedestrian is detected in the movement change detecting step; and

a behavior predicting step configured to predict a behavior of the pedestrian using the discontinuous movement estimating information acquired in the discontinuous movement estimating step.

25. The pedestrian behavior predicting method according to claim 24, wherein

in the movement change detecting step, the change in the movement of the pedestrian is detected by acquiring a feature amount distribution representing the shape information of the pedestrian acquired from the partial image extracted in the pedestrian detecting step, normalizing the shape information represented with the acquired feature amount distribution, accumulating the normalized shape information and comparing the shape information of a predetermined time before and the current shape information.

26. The pedestrian behavior predicting method according to claim 24, wherein

in the discontinuous movement estimating step, an optical flow of the imaged image is calculated, and the discontinuous movement estimating information is acquired using the optical flow at the time the change in the movement of the pedestrian is detected in the movement change detecting step.

27. The pedestrian behavior predicting method according to claim 25, wherein

in the discontinuous movement estimating step, an optical flow of the imaged image is calculated, and the discontinuous movement estimating information is acquired using the optical flow at the time the change in the movement of the pedestrian is detected in the movement change detecting step.

28. The pedestrian behavior predicting method according to claim 24, wherein

a position of the pedestrian on the imaged image is acquired in the pedestrian detecting step,

the pedestrian behavior predicting method further comprises a continuous movement estimating step configured to generate continuous movement estimating information indicating a continuous movement of the pedestrian that occurs following movement of the position of the pedestrian based on a history of positions of the pedestrian acquired in the pedestrian detecting step, and

the behavior of the pedestrian is predicted in the behavior predicting step based on the continuous movement estimating information generated in the continuous movement estimating step.

29. The pedestrian behavior predicting method according to claim 25, wherein

a position of the pedestrian on the imaged image is acquired in the pedestrian detecting step,

the pedestrian behavior predicting method further comprises a continuous movement estimating step configured to generate continuous movement estimating information indicating a continuous movement of the pedestrian that occurs following movement of the position of the pedestrian based on a history of positions of the pedestrian acquired in the pedestrian detecting step, and

the behavior of the pedestrian is predicted in the behavior predicting step based on the continuous movement estimating information generated in the continuous movement estimating step.

30. The pedestrian behavior predicting method according to claim 26, wherein

a position of the pedestrian on the imaged image is acquired in the pedestrian detecting step,

the pedestrian behavior predicting method further comprises a continuous movement estimating step configured to generate continuous movement estimating information indicating a continuous movement of the pedestrian that occurs following movement of the position of the pedestrian based on a history of positions of the pedestrian acquired in the pedestrian detecting step, and

the behavior of the pedestrian is predicted in the behavior predicting step based on the continuous movement estimating information generated in the continuous movement estimating step.