Passenger counting system, passenger counting method and passenger counting program

• US 9,477,881 B2
• Filed: 09/11/2013
• Issued: 10/25/2016
• Est. Priority Date: 10/19/2012
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Konishi, Yusuke
• Primary Examiner(s): Koziol, Stephen R
• Assistant Examiner(s): Gilliard, Delomia L
• Application Number:
  US14/435,326
• Publication Number:
  US 20150294144A1
• Time in Patent Office: 1,140 Days
• Field of Search: 382/118
• US Class Current: 1/1
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30242 Counting objects in image
• CPC Class Codes: G06T 2207/30268 Vehicle interior
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/166 using acquisition arrangements
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

A passenger counting system that can count correctly the number of persons present in a vehicle, including persons sitting on the back seat, is provided. The passenger counting system includes an image capturing device 11 which is arranged outside the vehicle and which captures an image of the vehicle from the lateral direction of the vehicle, an image acquisition unit 31 that controls the image capturing device 11 and acquires an image in which the inside of the vehicle is captured as an input image, a profile detection unit 32 that detects a profile of a person from a partial image obtained from the input image based on an image feature amount calculated by using the partial image and outputs a detection result, and a passenger number determination unit 33 that determines the number of persons present in the vehicle based on the detection result.

Claims

1. A passenger counting system comprising:

an image capturing device having a lens, the image capturing device being arranged outside a vehicle for capturing an image of the vehicle from a lateral direction of the vehicle;

a frontal image capturing device having a lens, the frontal image capturing device being arranged outside the vehicle for capturing an image of the vehicle from a front direction of the vehicle; and

a microprocessor including;

an image acquisition unit that controls the image capturing device during passing timing of the vehicle and acquires an image in which the inside of the vehicle is captured as an input image;

a frontal image acquisition unit that controls the frontal image capturing device during passing timing of the vehicle and acquires an image in which the inside of the vehicle is captured from the front direction as a front input image;

a profile detection unit that detects a profile of a person from a partial image obtained from the input image based on an image feature amount calculated by using the partial image and outputs a detection result;

a frontal face detection unit that detects a frontal face of a person from a partial image in the front input image based on image feature amount calculated by using the partial image obtained from the front input image and outputs a detection result; and

a passenger number determination unit that determines the number of persons present in the vehicle based on the detection results,

wherein the passenger number determination unit determines whether or not a seat position inside the vehicle of each partial image that is determined to include a profile of a person by the profile detection unit overlaps a seat position inside the vehicle of each partial image that is determined to include a frontal face of a person by the frontal face detection unit, determines a total number of seat positions where persons sit, and determines the number of persons present in the vehicle based on the total number of the seat positions.

2. The passenger counting system according to claim 1, wherein

the image acquisition unit acquires an image in which the inside of the vehicle is captured as an input image by controlling the image capturing device according to passing timing of the vehicle, which is obtained by a sensor that is arranged on a course on which the vehicle travels and detects presence of the vehicle.

3. The passenger counting system according to claim 1, wherein

the image acquisition unit determines whether or not the inside of the vehicle is captured in a plurality of images that is continuously captured by detecting a window frame by using image processing using template matching and acquires an image in which the inside of the vehicle is captured as an input image.

4. The passenger counting system according to claim 1, wherein

the profile detection unit uses an image feature amount based on position, orientation, strength, and luminance gradient of an edge, which can be calculated from a partial image including a profile outline from forehead to chin, as the image feature amount.

5. The passenger counting system according to claim 4, wherein

the profile detection unit applies preprocessing to the input image so that vertical edges remain in the input image.

6. The passenger counting system according to claim 1, wherein

the profile detection unit limits a range in the input image, from which a partial image is cut out, to a predetermined range in which inside of a vehicle may be captured, a range in which the inside of a vehicle, which is determined by detecting a window frame by using image processing using template matching, is captured, or an existing range of a vehicle or an existing range of a window frame, which are detected by using a laser range scanner or a passage sensor.

7. The passenger counting system according to claim 1, wherein

the passenger number determination unit determines the number of partial images that are determined to include a profile of a person by the profile detection unit as the number of persons present in the vehicle.

10. The passenger counting system according to claim 2, wherein

the image acquisition unit determines whether or not the inside of the vehicle is captured in a plurality of images that is continuously captured by detecting a window frame by using image processing using template matching and acquires an image in which the inside of the vehicle is captured as an input image.

11. The passenger counting system according to claim 2, wherein

the profile detection unit uses an image feature amount based on position, orientation, strength, and luminance gradient of an edge, which can be calculated from a partial image including a profile outline from forehead to chin, as the image feature amount.

12. The passenger counting system according to claim 3, wherein

the profile detection unit uses an image feature amount based on position, orientation, strength, and luminance gradient of an edge, which can be calculated from a partial image including a profile outline from forehead to chin, as the image feature amount.

13. The passenger counting system according to claim 2, wherein

the profile detection unit limits a range in the input image, from which a partial image is cut out, to a predetermined range in which inside of a vehicle may be captured, a range in which the inside of a vehicle, which is determined by detecting a window frame by using image processing using template matching, is captured, or an existing range of a vehicle or an existing range of a window frame, which are detected by using a laser range scanner or a passage sensor.

14. The passenger counting system according to claim 3, wherein

the profile detection unit limits a range in the input image, from which a partial image is cut out, to a predetermined range in which inside of a vehicle may be captured, a range in which the inside of a vehicle, which is determined by detecting a window frame by using image processing using template matching, is captured, or an existing range of a vehicle or an existing range of a window frame, which are detected by using a laser range scanner or a passage sensor.

15. The passenger counting system according to claim 4, wherein

the profile detection unit limits a range in the input image, from which a partial image is cut out, to a predetermined range in which inside of a vehicle may be captured, a range in which the inside of a vehicle, which is determined by detecting a window frame by using image processing using template matching, is captured, or an existing range of a vehicle or an existing range of a window frame, which are detected by using a laser range scanner or a passage sensor.

16. The passenger counting system according to claim 2, wherein

the passenger number determination unit determines the number of partial images that are determined to include a profile of a person by the profile detection unit as the number of persons present in the vehicle.

17. The passenger counting system according to claim 3, wherein

the passenger number determination unit determines the number of partial images that are determined to include a profile of a person by the profile detection unit as the number of persons present in the vehicle.

8. A passenger counting method comprising:

capturing an image of a vehicle from a lateral direction of the vehicle by an image capturing device, which has a lens, arranged outside the vehicle;

capturing an image of the vehicle from a front direction of the vehicle by a frontal image capturing device, which has a lens, arranged outside the vehicle;

controlling the image capturing device during passing timing of the vehicle and acquiring an image in which the inside of the vehicle is captured as an input image;

controlling the frontal image capturing device during passing timing of the vehicle and acquiring an image in which the inside of the vehicle is captured from the front direction as a front input image;

detecting a profile of a person from a partial image obtained from the input image based on an image feature amount calculated by using the partial image and outputting a detection result;

detecting a frontal face of a person from a partial image in the front input image based on image feature amount calculated by using the partial image obtained from the front input image and outputs a detection result; and

determining the number of persons present in the vehicle based on the detection results,

wherein, when the number of persons present in the vehicle is determined, determining whether or not a seat position inside the vehicle of each partial image that is determined to include a profile of a person by the detecting a profile of a person overlaps a seat position inside the vehicle of each partial image that is determined to include a frontal face of a person by the detecting a frontal face of a person, determining a total number of seat positions where persons sit, and determining the number of persons present in the vehicle based on the total number of the seat positions.

9. A non-transitory computer readable information recording medium storing a passenger counting program that, when executed by a processor, performs a method for:

capturing an image of a vehicle from a lateral direction of the vehicle by an image capturing device, which has a lens, arranged outside the vehicle;

capturing an image of the vehicle from a front direction of the vehicle by a frontal image capturing device, which has a lens, arranged outside the vehicle;

controlling the image capturing device during passing timing of the vehicle and acquiring an image in which the inside of the vehicle is captured as an input image;

controlling the frontal image capturing device during passing timing of the vehicle and acquiring an image in which the inside of the vehicle is captured from the front direction as a front input image;

detecting a profile of a person from a partial image obtained from the input image based on an image feature amount calculated by using the partial image and outputting a detection result;

detecting a frontal face of a person from a partial image in the front input image based on image feature amount calculated by using the partial image obtained from the front input image and outputs a detection result; and

determining the number of persons present in the vehicle based on the detection results,

wherein, when the number of persons present in the vehicle is determined, determining whether or not a seat position inside the vehicle of each partial image that is determined to include a profile of a person by the detecting a profile of a person overlaps a seat position inside the vehicle of each partial image that is determined to include a frontal face of a person by the detecting a frontal face of a person, determining a total number of seat positions where persons sit, and determining the number of persons present in the vehicle based on the total number of the seat positions.