Using vehicle sensor data to monitor pedestrian health

• US 10,694,357 B2
• Filed: 12/01/2016
• Issued: 06/23/2020
• Est. Priority Date: 11/11/2016
• Status: Active Grant

• Current Assignee: NIO Technology Company Limited (Shanghai Anbin Technology Co., Ltd.)
• Original Assignee: Nio Usa Inc.
• Inventors: Newman, Austin
• Primary Examiner(s): Geroleo, Francis
• Application Number:
  US15/366,623
• Publication Number:
  US 20180137372A1
• Time in Patent Office: 1,300 Days
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source
• CPC Class Codes: H04W 4/40 for vehicles, e.g. vehicle-to-pedestrians [V2P]
• CPC Class Codes: H04W 4/80 Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication

Abstract

A system and method for monitoring pedestrians based upon information is collected by a wide array of sensors already included in modern motor vehicles. Also included, is a system of monitoring pedestrians by aggregating data collected by an array of vehicles.

Claims

1. An apparatus of a vehicle for monitoring pedestrian health comprising:

one or more thermal imagers,

one or more high definition imagers,

a real time communication interface,

a short range communication interface, and

a vehicle computer communicatively coupled to the one or more thermal imagers, the one or more high definition imagers, the real time communication interface and the short range communication interface;

wherein the vehicle computer;

acquires a plurality of thermal images of a pedestrian sidewalk from the one or more thermal imagers as the vehicle is driving on a roadway located adjacent the pedestrian sidewalk,

acquires a plurality of high definition images of the pedestrian sidewalk from the one or more high definition imagers as the vehicle is driving on the roadway located adjacent the pedestrian sidewalk,

determines whether the plurality of thermal images and the plurality of high definition images contain one or more pedestrians on the pedestrian sidewalk,

determines, based on the plurality of thermal images and the plurality of high definition images containing the one or more pedestrians on the pedestrian sidewalk, whether a pedestrian of the one or more pedestrians who was previously walking on the pedestrian sidewalk is falling or lying on the pedestrian sidewalk and exhibiting an injured pedestrian behavior,

selectively displays an alert on a display communicatively coupled to the vehicle computer in response to determining that the pedestrian is falling or lying on the pedestrian sidewalk, and

selectively transmits via the real time interface the plurality of thermal images and the plurality of high definition images to a database server in response to determining that the pedestrian is falling or lying on the pedestrian sidewalk.

2. The apparatus of claim 1, wherein the vehicle computer further:

selectively transmits via the real time communication interface the plurality of thermal images and the plurality of high definition images along with an alert to emergency services, wherein the alert includes one or more of time, location, and an identification of the injured pedestrian.

3. The apparatus of claim 1, wherein the vehicle computer further:

when the plurality of thermal images and the plurality of high definition images are determined to contain the one or more pedestrians, analyzes the plurality of thermal images and the plurality of high definition images to determine if the one or more pedestrians are displaying a predetermined pedestrian behavior indicating the one or more pedestrians have an elevated thermal profile.

4. The apparatus of claim 3, wherein the injured pedestrian behavior is associated with a criminal activity.

5. The apparatus of claim 1, wherein the vehicle computer further:

determines, based on the plurality of thermal images, a thermal profile of the pedestrian, and

determines, based on the thermal profile of the pedestrian, that the pedestrian is having a heart attack.

6. The apparatus of claim 1, wherein

determining whether the plurality of thermal images and the plurality of high definition images contain one or more pedestrians on the pedestrian sidewalk further comprises determining whether a moving object on the pedestrian sidewalk has a kinematic profile that corresponds to a pedestrian movement associated with the one or more pedestrians.

7. A method for monitoring pedestrian health comprising:

acquiring, by a vehicle computer of a vehicle, a plurality of thermal images of a pedestrian sidewalk from one or more thermal imagers as the vehicle is driving on a roadway located adjacent the pedestrian sidewalk;

acquiring, by the vehicle computer, a plurality of high definition images of the pedestrian sidewalk from one or more high definition imagers as the vehicle is driving on the roadway located adjacent the pedestrian sidewalk,

determining, by the vehicle computer, whether the plurality of thermal images and the plurality of high definition images contain one or more pedestrians on the pedestrian sidewalk,

determining, based on the plurality of thermal images and the plurality of high definition images contain the one or more pedestrians on the pedestrian sidewalk, whether a pedestrian of the one or more pedestrians who was previously walking on the pedestrian sidewalk is falling or lying on the pedestrian sidewalk and exhibiting an injured pedestrian behavior,

selectively displaying, by the vehicle computer, an alert on a display communicatively coupled to the vehicle computer in response to determining the pedestrian is falling or lying on the pedestrian sidewalk; and

selectively transmitting, via a real time interface of the vehicle computer, the plurality of thermal images and the plurality of high definition images to a database server in response to determining that the pedestrian is falling or lying on the pedestrian sidewalk.

8. The method of claim 7, further comprising:

selectively transmitting, via the real time communication interface of the vehicle computer, the plurality of thermal images and the plurality of high definition images along with an alert to emergency services, wherein the alert includes one or more of time, location, and an identification of the injured pedestrian behavior.

9. The method of claim 7, further comprising:

when the plurality of thermal images and the plurality of high definition images are determined to contain the one or more pedestrians, analyzing, by the vehicle computer, the plurality of thermal images and the plurality of high definition images to determine if the one or more pedestrians are displaying a predetermined pedestrian behavior indicating the one or more pedestrians have an elevated thermal profile.

10. The method of claim 9, wherein the injured pedestrian behavior is associated with a criminal activity.

11. The method of claim 7, further comprising:

determining, by the vehicle computer and based on the plurality of thermal images a thermal signature of the pedestrian, and

determining, based on the thermal signature of the pedestrian, that the pedestrian is having a heart attack.

12. The method of claim 7, wherein

determining whether the plurality of thermal images and the plurality of high definition images contain one or more pedestrians on the pedestrian sidewalk further comprises determining whether a moving object on the pedestrian sidewalk has a kinematic profile that corresponds to a pedestrian movement associated with the one or more pedestrians.