Methods and systems for vehicle occupancy confirmation

• US 10,416,671 B2
• Filed: 08/18/2017
• Issued: 09/17/2019
• Est. Priority Date: 07/11/2017
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Waymo LLC (Alphabet Inc.)
• Inventors: Herbach, Joshua
• Primary Examiner(s): Badii, Behrang
• Assistant Examiner(s): Greene, Daniel L
• Application Number:
  US15/681,041
• Publication Number:
  US 20190018411A1
• Time in Patent Office: 760 Days
• CPC Class Codes: B60N 2/002 Seats provided with an occupancy detection means mounted therein or thereon detection means responsive to presence or absence of children B60N2/266
• CPC Class Codes: B60R 1/008 allowing the driver to see passengers, e.g. for busses combined with rear-view mirrors B60R1/08
• CPC Class Codes: B60R 2022/4816 for sensing locking of buckle
• CPC Class Codes: B60R 22/48 Control systems, alarms, or interlock systems, for the correct application of the belt or harness
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60W 40/08 related to drivers or passengers
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/228 Command input arrangements located on-board unmanned vehicles
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06V 20/593 Recognising seat occupancy
• CPC Class Codes: G08G 1/202 Dispatching vehicles on the basis of a location, e.g. taxi dispatching
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

Example implementations relate to vehicle occupancy confirmation. An example implementation involves receiving, at a computing system from a camera positioned inside a vehicle, an image representing an occupancy within the vehicle. The implementation further involves, responsive to receiving the image, displaying the image on a display interface, and receiving an operator input confirming the occupancy meets a desired occupancy. The implementation additionally includes transmitting an occupancy confirmation from the computing system to the vehicle. In some instances, in response to receiving the occupancy confirmation, the vehicle executes an autonomous driving operation.

Claims

1. A method comprising:

receiving, at a computing system from a camera positioned inside a vehicle, an image representing an occupancy within the vehicle, wherein the computing system is positioned remotely from the vehicle;

responsive to receiving the image, estimating, by the computing system, a quantity of passengers inside the vehicle;

displaying, by the computing system on a graphical user interface, the image with corresponding text that represents the estimated quantity of passengers;

receiving, at the graphical user interface, confirmation or denial that the estimated quantity of passengers is accurate and meets a desired occupancy; and

transmitting an indication of the confirmation or denial from the computing system to the vehicle, wherein in response to receiving confirmation that the estimated quantity of passengers is accurate and meets the desired occupancy, the vehicle executes an autonomous driving operation.

2. The method of claim 1, wherein the camera is mounted to an interior of the vehicle such that the camera is oriented towards an interior of the vehicle.

3. The method of claim 1, further comprising:

receiving, at the graphical user interface, an input that indicates respective positions of one or more passengers inside the vehicle; and

wherein the indication of the confirmation or denial further indicates the respective positions of the one or more passengers inside the vehicle.

4. The method of claim 1, wherein receiving the image representing the occupancy within the vehicle further comprises:

receiving audio data representing the occupancy within the vehicle; and

wherein displaying the image on a display interface further comprises;

providing, by the computing system, the audio data via an audio interface.

5. The method of claim 1, wherein the desired occupancy represents a quantity of passengers that should occupy the vehicle prior to the vehicle executing the autonomous driving operation.

6. The method of claim 1, further comprising:

receiving, from a sensor, sensor data indicating motion inside the vehicle; and

responsive to receiving the sensor data indicating motion inside the vehicle, transmitting a signal to the camera to capture one or more images.

7. The method of claim 6, wherein the sensor corresponds to a proximity sensor positioned inside the vehicle.

8. The method of claim 6, wherein the sensor corresponds to a tactile sensor associated with a seatbelt of the vehicle, and

wherein the sensor data indicating motion inside corresponds to detected use of the seatbelt.

9. The method of claim 1, wherein displaying the image with corresponding text that represents the estimated quantity of passengers further comprises:

modifying the image to include one or more enhancement graphics; and

displaying the modified image at the graphical user interface.

10. The method of claim 9, wherein the one or more enhancement graphics includes graphical text indicating a location of the vehicle.

11. The method of claim 1, wherein the indication of the confirmation or denial further includes identification information for at least one passenger positioned in the vehicle.

12. The method of claim 11,

wherein in response to receiving the identification information for the at least one passenger positioned in the vehicle, the vehicle executes a given autonomous driving operation specific for the at least one passenger.

13. A system comprising:

a vehicle;

a computing system positioned remotely from the vehicle, wherein the computing system includes a graphical user interface;

a non-transitory computer readable medium; and

program instructions stored on the non-transitory computer readable medium executable by the computing system to;

receive, from a camera positioned inside a vehicle, an image representing an occupancy within the vehicle;

responsive to receiving the image, estimate a quantity of passengers inside the vehicle;

display, on the graphical user interface, the image with corresponding text that represents the estimated quantity of passengers;

receive, at the graphical user interface, confirmation or denial that the estimated quantity of passengers is accurate and meets a desired occupancy; and

transmit an indication of the confirmation or denial to the vehicle, wherein in response to receiving confirmation that the estimated quantity of passengers is accurate and meets the desired occupancy, the vehicle executes an autonomous driving operation.

14. The system of claim 13, wherein the program instructions are further executable by the computing system to:

receive an image indicative of at least one passenger from a camera coupled to an exterior of the vehicle; and

responsive to receiving the image indicate of the at least one passenger, transmit a signal to the camera positioned inside the vehicle to capture one or more images.

15. The system of claim 13, wherein the program instructions are further executable by the computing system to:

detect a modification to a media system positioned inside the vehicle; and

responsive to detecting the modification to the media system, transmit a signal to the camera positioned inside the vehicle to capture one or more images.

16. The system of claim 13, wherein the program instructions are further executable by the computing system to:

receive sensor measurements indicative of at least one passenger from a sensor positioned inside a seat of the vehicle; and

responsive to receiving the sensor measurements, transmit a signal to the camera positioned inside the vehicle to capture one or more images.

17. A non-transitory computer readable medium having stored therein instructions executable by one or more processors to cause a computing system to perform functions comprising:

receiving, from a camera positioned inside a vehicle, an image representing an occupancy within the vehicle;

responsive to receiving the image, estimating a quantity of passengers inside the vehicle;

displaying, on a graphical user interface, the image with corresponding text that represents the estimated quantity of passengers, wherein the graphical user interface is positioned remotely from the vehicle;

receiving, at the graphical user interface, confirmation or denial that the estimated quantity of passengers is accurate and meets a desired occupancy; and

transmitting an indication of the confirmation or denial to the vehicle, wherein in response to receiving confirmation that the estimated quantity of passengers is accurate and meets the desired occupancy, the vehicle executes an autonomous driving operation.

18. The non-transitory computer readable medium of claim 17, further comprising:

receiving sensor data indicating motion inside the vehicle; and

responsive to receiving the sensor data indicating motion inside the vehicle, transmitting a signal to the camera to capture one or more images.

19. A method comprising:

receiving, at a computing system from a camera positioned inside a vehicle, an image representing an interior environment of the vehicle;

responsive to receiving the image, estimating, by the computing system, a quantity of passengers inside the vehicle;

transmitting, by the computing system to a graphical user interface associated with a second computing system, the image with corresponding text that represents the estimated quantity of passengers, wherein the second computing system is positioned remotely from the vehicle;

receiving, at the computing system and from the second computing system, confirmation or denial that the estimated quantity of passengers is accurate and

meets a desired occupancy; and

in response to receiving confirmation that the estimated quantity of passengers is accurate and meets the desired occupancy, causing, by the computing system, the vehicle to execute an autonomous driving operation.

20. The method of claim 19, wherein the camera is mounted to an interior of the vehicle such that the camera is oriented towards the interior environment of the vehicle.

21. The method of claim 19, wherein the confirmation further indicates respective positions of one or more passengers inside the vehicle.

22. The method of claim 19, wherein the computing system is positioned at a remote location from the vehicle.

23. The method of claim 19, wherein the computing system is positioned inside the vehicle.

24. The method of claim 19, wherein estimating the quantity of passengers inside the vehicle comprises:

performing, using the image a computer vision process to place a box around each passenger inside the vehicle;

estimating the quantity of passengers inside the vehicle based on a quantity of boxes placed during the computer vision process.

25. The method of claim 19, wherein the desired occupancy represents a quantity of passengers that should occupy the vehicle prior to the vehicle executing the autonomous driving operation.

26. The method of claim 19, further comprising:

receiving, at the computing system from a sensor, sensor data indicating motion inside the vehicle; and

responsive to receiving the sensor data indicating motion inside the vehicle, transmitting, by the computing system, a signal to the camera to capture one or more images.

27. The method of claim 26, wherein the sensor corresponds to a tactile sensor associated with a seatbelt of the vehicle, and

wherein the sensor data indicating motion inside corresponds to detected use of the seatbelt.