Passenger counting device, system, method and program, and vehicle movement amount calculation device, method and program

• US 10,318,829 B2
• Filed: 03/17/2016
• Issued: 06/11/2019
• Est. Priority Date: 03/17/2016
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Miyamoto, Shinichi
• Primary Examiner(s): Chang, Jon
• Application Number:
  US15/572,826
• Publication Number:
  US 20180144204A1
• Time in Patent Office: 1,181 Days
• CPC Class Codes: G06T 1/00 General purpose image data processing
• CPC Class Codes: G06T 2207/30242 Counting objects in image
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G06V 20/593 Recognising seat occupancy
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation

Abstract

The passenger counting system includes a photographing means 20 for photographing a vehicle and acquiring an image, and a passenger counting device 200. A passenger counting device 200 includes: a movement amount calculation means 21 for calculating a movement amount of a vehicle based on an image of the vehicle; a depth distance calculation means 22 for calculating a distance in a depth direction of a face of a passenger of the vehicle based on the movement amount of the vehicle; and a passenger number determination means 23 for detecting the face of the passenger of the vehicle from the image and determining the number of passengers of the vehicle based on distances in the depth direction of a plurality of detected faces of passengers.

Claims

1. A passenger counting device comprising:

a movement amount calculation unit, implemented by a processor, for calculating a movement amount of a vehicle based on an image of the vehicle;

a depth distance calculation unit, implemented by the processor, for calculating a distance in a depth direction of a face of a passenger of the vehicle based on the movement amount of the vehicle; and

a passenger number determination unit, implemented by the processor, for detecting the face of the passenger of the vehicle from the image and determining a number of passengers of the vehicle based on distances in the depth direction of a plurality of detected faces of passengers.

2. The passenger counting device according to claim 1, wherein

the passenger number determination unit determines presence or absence of erroneous detection based on a distance in the depth direction between a plurality of detected faces of passengers.

3. The passenger counting device according to claim 1, wherein

the movement amount calculation unit estimates an error of the movement amount of the vehicle using a steepest descent method.

4. The passenger counting device according to claim 3, wherein

the movement amount calculation unit measures a distance from a first position to a second position of a face of a specific person for each timing based on images of the vehicle at a plurality of timings and estimates the error of the movement amount of the vehicle when an objective function becomes an extreme value using the steepest descent method by setting a function including a difference between the distance at a first timing and the distance at a second timing as the objective function.

5. The passenger counting device according to claim 1, wherein

the passenger number determination unit determines that a plurality of faces of passengers are faces of different persons when a distance in the depth direction between the plurality of detected faces of passengers is equal to or longer than a first threshold value.

6. The passenger counting device according to claim 1, wherein

the passenger number determination unit determines that a plurality of faces of passengers are faces of different persons when a distance in the depth direction between the plurality of detected faces of passengers is equal to or longer than a first threshold value or when a distance in a traveling direction between the plurality of faces of passengers is equal to or longer than a second threshold value.

7. The passenger counting device according to claim 1, wherein

the depth distance calculation unit calculates a distance in the depth direction from a photographing unit to the face of the passenger based on the calculated movement amount of the vehicle and a direction toward the face of the passenger of the vehicle from a position of the photographing unit that has photographed the vehicle.

8. A passenger counting method comprising:

calculating a movement amount of a vehicle based on an image of the vehicle;

calculating a distance in a depth direction of a face of a passenger of the vehicle based on the movement amount of the vehicle; and

detecting the face of the passenger of the vehicle from the image and determining a number of passengers of the vehicle based on distances in the depth direction of a plurality of detected faces of passengers.

9. The passenger counting method according to claim 8, wherein

presence or absence of erroneous detection is determined based on a distance in the depth direction between a plurality of detected faces of passengers.

10. The passenger counting method according to claim 8, wherein

an error of the movement amount of the vehicle is estimated using a steepest descent method.

11. The passenger counting method according to claim 10, wherein

a distance from a first position to a second position of a face of a specific person is measured for each timing based on images of the vehicle at a plurality of timings, and the error of the movement amount of the vehicle when an objective function becomes an extreme value is estimated using the steepest descent method by setting a function including a difference between the distance at a first timing and the distance at a second timing as the objective function.

12. The passenger counting method according to claim 8, wherein

when a distance in the depth direction between a plurality of detected faces of passengers is equal to or longer than a first threshold value, it is determined that the plurality of faces of passengers are faces of different persons.

13. The passenger counting method according to claim 8, wherein

when a distance in the depth direction between a plurality of detected faces of passengers is equal to or longer than a first threshold value or when a distance in a traveling direction between the plurality of faces of passengers is equal to or longer than a second threshold value, it is determined that the plurality of faces of passengers are faces of different persons.

14. The passenger counting method according to claim 8, wherein

a distance in the depth direction from a photographing unit to the face of the passenger is calculated based on the calculated movement amount of the vehicle and a direction toward the face of the passenger of the vehicle from a position of the photographing unit that has photographed the vehicle.

15. A non-transitory computer-readable recording medium recording a passenger counting program configured to cause a computer to execute:

a movement amount calculation process of calculating a movement amount of a vehicle based on an image of the vehicle;

a depth distance calculation process of calculating a distance in a depth direction of a face of a passenger of the vehicle based on the movement amount of the vehicle; and

a passenger number determination process of detecting the face of the passenger of the vehicle from the image and determining a number of passengers of the vehicle based on distances in the depth direction of a plurality of detected faces of passengers.

16. The non-transitory computer-readable recording medium according to claim 15, wherein

the passenger counting program causes the computer to determine presence or absence of erroneous detection based on a distance in the depth direction between a plurality of detected faces of passengers in the passenger number determination process.

17. The non-transitory computer-readable recording medium according to claim 15, wherein

the passenger counting program causes the computer to estimate an error of the movement amount of the vehicle using a steepest descent method in the movement amount calculation process.

18. The non-transitory computer-readable recording medium according to claim 17, wherein

the passenger counting program causes the computer to measure a distance from a first position to a second position of a face of a specific person for each timing based on images of the vehicle at a plurality of timings, and to estimate the error of the movement amount of the vehicle when an objective function becomes an extreme value using the steepest descent method by setting a function including a difference between the distance at a first timing and the distance at a second timing as the objective function in the movement amount calculation process.

19. The non-transitory computer-readable recording medium according to claim 15, wherein

the passenger counting program causes the computer to determine that a plurality of faces of passengers are faces of different persons when a distance in the depth direction between the plurality of detected faces of passengers is equal to or longer than a first threshold value in the passenger number determination process.

20. The non-transitory computer-readable recording medium according to claim 15, wherein

the passenger counting program causes the computer to determine that a plurality of faces of passengers are faces of different persons when a distance in the depth direction between the plurality of detected faces of passengers is equal to or longer than a first threshold value or when a distance in a traveling direction between the plurality of faces of passengers is equal to or longer than a second threshold value in the passenger number determination process.