DETERMINING AND RESPONDING TO AN INTERNAL STATUS OF A VEHICLE

• US 20190258263A1
• Filed: 02/21/2018
• Published: 08/22/2019
• Est. Priority Date: 02/21/2018
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Waymo LLC (Alphabet Inc.)
• Inventors: Wendel, Andreas, Ludwick, Christopher Kennedy, Feenstra, Laurens Andreas
• Granted Patent: US 10,838,425 B2
• Time in Patent Office: Days
• CPC Class Codes: B60K 28/04 responsive to presence or absence of the driver, e.g. to weight or lack thereof
• CPC Class Codes: B60N 2/002 Seats provided with an occupancy detection means mounted therein or thereon detection means responsive to presence or absence of children B60N2/266
• CPC Class Codes: B60N 2/0023 for detection of driver fatigue
• CPC Class Codes: B60N 2/0026 for distinguishing between humans, animals or objects
• CPC Class Codes: B60N 2/0272 using sensors or detectors for detecting the position of seat parts
• CPC Class Codes: B60N 2/268 detecting or alerting means responsive to the installation of the child seats in the vehicle
• CPC Class Codes: B60N 2/271 detecting or alerting means responsive to securing the child in the seat
• CPC Class Codes: B60N 2210/18 Infrared
• CPC Class Codes: B60N 2210/24 Cameras
• CPC Class Codes: B60N 2230/30 Signal processing of sensor data
• CPC Class Codes: B60Q 9/00 Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00 , e.g. haptic signalling
• CPC Class Codes: B60R 16/0234 related to maintenance or repairing of vehicles
• CPC Class Codes: B60R 21/01538 for image processing, e.g. cameras or sensor arrays
• CPC Class Codes: B60R 21/01556 Child-seat detection systems
• CPC Class Codes: B60R 22/48 Control systems, alarms, or interlock systems, for the correct application of the belt or harness
• CPC Class Codes: B60W 2050/143 Alarm means
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2540/043 Identity of occupants
• CPC Class Codes: B60W 2540/049 Number of occupants
• CPC Class Codes: B60W 2540/221 Physiology, e.g. weight, heartbeat, health or special needs
• CPC Class Codes: B60W 2540/223 : Posture, e.g. hand, foot, or seat position, turned or inclined
• CPC Class Codes: B60W 2540/227 : Position in the vehicle
• CPC Class Codes: B60W 50/14 : Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: B60W 60/00253 : Taxi operations
• CPC Class Codes: G01C 21/3602 : Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
• CPC Class Codes: G05D 1/0221 : involving a learning process
• CPC Class Codes: G05D 1/0246 : using a video camera in combination with image processing means

Abstract

Aspects of the disclosure relate to determining and responding to an internal state of a self-driving vehicle. For instance, an image of an interior of the vehicle captured by a camera mounted in the vehicle is received. The image is processed in order to identify one or more visible markers at predetermined locations within the vehicle. The internal state of the vehicle is determined based on the identified one or more visible markers. A responsive action is identified action using the determined internal state, and the vehicle is controlled in order to perform the responsive action.

Claims

1. A method of determining and responding to an internal state of a self-driving vehicle, the method comprising:

receiving an image of an interior of the vehicle captured by a camera mounted in the vehicle;

processing the image in order to identify one or more visible markers at predetermined locations within the vehicle;

determining the internal state of the vehicle based on the identified one or more visible markers;

identifying a responsive action using the determined internal state; and

controlling the vehicle in order to perform the responsive action.

2. The method of claim 1, wherein the image is processed using a machine learning model which provides the determined internal state.

3. The method of claim 1, further comprising using the one or more visible markers to determine a height of a passenger of the vehicle, and wherein determining the internal state of the vehicle is based on the determined height.

4. The method of claim 1, wherein the internal state indicates that a seat of the vehicle is occupied.

5. The method of claim 1, wherein the internal state of the vehicle indicates that a seat of the vehicle is unoccupied.

6. The method of claim 1, wherein the one or more visible markers are located on a seat back of a seat of the vehicle, and the internal state of the vehicle indicates that the seat is unoccupied.

7. The method of claim 1, wherein the one or more visible markers are located on a seatbelt of a seat of the vehicle, and the internal state of the vehicle indicates that the seat is occupied by a passenger.

8. The method of claim 7, wherein the internal state of the vehicle indicates that the passenger is wearing a seatbelt.

9. The method of claim 1, further comprising qualifying the determined internal state based on whether the vehicle is currently on a trip to a destination with one or more passengers.

10. The method of claim 1, wherein the determined internal state indicates a number of passengers within the vehicle.

11. The method of claim 1, wherein when the determined internal state indicates that the vehicle is occupied by a passenger and a trip has just ended, the responsive action includes providing an audible notification to the passenger to indicate that it is time to leave the vehicle.

12. The method of claim 1, wherein when the determined internal state indicates that the vehicle is unoccupied and a trip is in progress, the responsive action includes cancelling the trip.

13. The method of claim 1, wherein when the determined internal state indicates that a child is in the vehicle, the responsive action includes providing a notification that indicates that the child must be in a specific seat within the vehicle.

14. The method of claim 13, wherein the responsive action includes not starting a trip until the child is the specific seat of the vehicle.

15. The method of claim 1, wherein the determined internal state indicates that a child is in the vehicle, the responsive action includes providing a notification that indicates that the child must be in an appropriate child restraint within the vehicle.

16. The method of claim 15, wherein the responsive action includes not starting a trip until the child is the appropriate child restraint.

17. The method of claim 1, wherein the determined internal state indicates that a child is in the vehicle, the responsive action includes providing a notification to a client device of an account holders associated with the trip requesting confirmation that the child is allowed to ride in the vehicle.

18. The method of claim 1, wherein the determined internal state indicates that a child is in the vehicle, and the method further comprises turning off an airbag adjacent to a seat in which the child is located.

19. The method of claim 1, wherein the determined internal state indicates that a passenger is not wearing a seatbelt, the responsive action includes providing a notification requests that the passenger use the seatbelt.

20. A system for determining and responding to an internal state of a self-driving vehicle, the system comprising one or more processors configured to:

receive an image of an interior of the vehicle captured by a camera mounted in the vehicle;

process the image in order to identify one or more visible markers at predetermined locations within the vehicle;

determine the internal state of the vehicle based on the identified one or more visible markers;

identify a responsive action using the determined internal state; and

control the vehicle in order to perform the responsive action.