Passenger counting device, passenger counting method, and program recording medium

• US 9,934,442 B2
• Filed: 10/01/2014
• Issued: 04/03/2018
• Est. Priority Date: 10/09/2013
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Oami, Ryoma, Ebiyama, Yukie
• Primary Examiner(s): Tabatabai, Abolfazl
• Application Number:
  US15/025,968
• Publication Number:
  US 20160239714A1
• Time in Patent Office: 1,280 Days
• Field of Search: 382103, 382104, 382107, 340901, 340902, 340903, 340904, 340933, 340938, 340988, 701417, 701519
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 2207/30242 Counting objects in image
• CPC Class Codes: G06T 2207/30268 Vehicle interior
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G06V 20/593 Recognising seat occupancy
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

Disclosed are a passenger counting device, etc., for accurately counting the number of occupants in a vehicle, including persons seated in the rear seats thereof. This vehicle passenger counting device comprises: a specific part detection means for detecting a specific part of a vehicle from acquired images and generating specific part detection results; an association means for associating the specific part detection results across images in order to compute a degree of movement of the vehicle and generate vehicle movement information including the result of the said computation; a person detection means for detecting persons who are occupants of the vehicle in the images and generating person detection results including location information of the detected persons; and an integration means for determining the number of occupants by integrating the person detection results on the basis of the vehicle movement information.

Claims

1. A passenger counting device comprising:

a memory storing instructions; and

at least one processor configured to process the instructions to;

detect a specific part of a vehicle in an image and generating specific-part detection results;

calculate a movement amount of the vehicle by associating the specific-part detection results with each other in images, and generating vehicle movement information including a result of the calculation;

detect a person who is a passenger, in the images and generating person detection results including position information of the detected person; and

integrate the person detection results by associating the images based on the vehicle movement information and thereby determine number of passengers.

2. The passenger counting device according to claim 1, wherein the processor is further configured to process the instructions to:

detect a wheel of the vehicle as the specific part of the vehicle.

3. The passenger counting device according to claim 2, wherein the processor is further configured to process the instructions to:

detect the wheel in the acquired image and generating wheel position information indicating a position of the wheel in the image and

analyze the image based on the wheel position information, estimating whether the detected wheel is a front wheel or a rear wheel of the vehicle, and generating the specific-part detection results including a result of the estimation, and

associate wheels detected in the images with each other by using the results of the estimation included in the specific-part detection results.

4. The passenger counting device according to claim 3, wherein the processor is further configured to process the instructions to:

obtain an edge feature or a gradient feature near the detected wheel based on the wheel position information, calculates likelihood that the detected wheel is a front wheel of the vehicle and likelihood that the detected wheel is a rear wheel of the vehicle, based on the edge feature or the gradient feature, and generates the specific-part detection results including a result of the calculation.

5. The passenger counting device according to claim 4, wherein the processor is further configured to process the instructions to:

group the person detection results for persons located at positions close to each other, each of the positions being indicated by the corresponding position information for each of the images, and generating person grouping information indicating a result of the grouping,

generate group association information indicating association between each group in each of the images and a group in each different one of the images, based on the vehicle movement information and the person grouping information, and

determine number of persons seated in seats in each row of the vehicle based on the group association information and calculating number of passengers by integrating the numbers of persons in the respective rows.

6. The passenger counting device according to claim 4, wherein the processor is further configured to process the instructions to:

estimate geometric distortion in the image, and

perform correction on the image based on the estimated geometric distortion and detects a person in the corrected image.

7. The passenger counting device according to claim 3, wherein the processor is further configured to process the instructions to:

group the person detection results for persons located at positions close to each other, each of the positions being indicated by the corresponding position information for each of the images, and generating person grouping information indicating a result of the grouping,

generate group association information indicating association between each group in each of the images and a group in each different one of the images, based on the vehicle movement information and the person grouping information, and

determine number of persons seated in seats in each row of the vehicle based on the group association information and calculating number of passengers by integrating the numbers of persons in the respective rows.

8. The passenger counting device according to claim 3, wherein the processor is further configured to process the instructions to:

estimate geometric distortion in the image, and

perform correction on the image based on the estimated geometric distortion and detects a person in the corrected image.

9. The passenger counting device according to claim 3, wherein the processor is further configured to process the instructions to:

estimate geometric distortion in the image, and

perform correction on the image based on the estimated geometric distortion and detects a person in the corrected image.

10. The passenger counting device according to claim 3, wherein the processor is further configured to process the instructions to:

estimate geometric distortion in the image, and

detect a person in the image by using a person detector corresponding to the estimated geometric distortion.

11. The passenger counting device according to claim 2, wherein the processor is further configured to process the instructions to:

group the person detection results for persons located at positions close to each other, each of the positions being indicated by the corresponding position information for each of the images, and generating person grouping information indicating a result of the grouping,

generate group association information indicating association between each group in each of the images and a group in each different one of the images, based on the vehicle movement information and the person grouping information, and

determine number of persons seated in seats in each row of the vehicle based on the group association information and calculating number of passengers by integrating the numbers of persons in the respective rows.

12. The passenger counting device according to claim 2, wherein the processor is further configured to process the instructions to:

estimate geometric distortion in the image, and

perform correction on the image based on the estimated geometric distortion and detects a person in the corrected image.

13. The passenger counting device according to claim 2, wherein the processor is further configured to process the instructions to:

estimate geometric distortion in the image, and

perform correction on the image based on the estimated geometric distortion and detects a person in the corrected image.

14. The passenger counting device according to claim 2, wherein the processor is further configured to process the instructions to:

estimate geometric distortion in the image, and

detect a person in the image by using a person detector corresponding to the estimated geometric distortion.

15. The passenger counting device according to claim 1, wherein the processor is further configured to process the instructions to:

group the person detection results for persons located at positions close to each other, each of the positions being indicated by the corresponding position information for each of the images, and generating person grouping information indicating a result of the grouping,

generate group association information indicating association between each group in each of the images and a group in each different one of the images, based on the vehicle movement information and the person grouping information, and

determine number of persons seated in seats in each row of the vehicle based on the group association information and calculating number of passengers by integrating the numbers of persons in the respective rows.

16. The passenger counting device according to claim 15, wherein the processor is further configured to process the instructions to:

estimate geometric distortion in the image, and

perform correction on the image based on the estimated geometric distortion and detects a person in the corrected image.

17. The passenger counting device according to claim 1, wherein the processor is further configured to process the instructions to:

estimate geometric distortion in the image, and

perform correction on the image based on the estimated geometric distortion and detects a person in the corrected image.

18. The passenger counting device according to claim 1, wherein the processor is further configured to process the instructions to:

estimate geometric distortion in the image, and

detect a person in the image by using a person detector corresponding to the estimated geometric distortion.

19. A passenger counting method comprising:

detecting a specific part of a vehicle in an image and generating specific-part detection results;

calculating a movement amount of the vehicle by associating the specific-part detection results with each other in images, and generating vehicle movement information including a result of the calculation;

detecting a person who is a passenger, in the acquired images and generating person detection results including position information of the detected person; and

integrating the person detection results by associating the images based on the vehicle movement information and thereby determining number of passengers.

20. A non-transitory computer-readable program recording medium recording a passenger counting program causing a computer to execute:

a process of detecting a specific part of a vehicle in an image and generating specific-part detection results;

a process of calculating a movement amount of the vehicle by associating the specific-part detection results with each other in images, and generating vehicle movement information including a result of the calculation;

a process of detecting a person who is a passenger, in the acquired images and generating person detection results including position information of the detected person; and

a process of integrating the person detection results by associating the images based on the vehicle movement information and thereby determining number of passengers.