Enhanced safety through augmented reality and shared data

• US 10,088,676 B2
• Filed: 06/30/2016
• Issued: 10/02/2018
• Est. Priority Date: 06/30/2016
• Status: Active Grant

• Current Assignee: PayPal, Inc. (PayPal Holdings, Inc.)
• Original Assignee: PayPal, Inc. (PayPal Holdings, Inc.)
• Inventors: O'Toole, Christopher Diebold, Luk, Bryant Genepang, Das, Ananya, Ziaja, Jason, Warman, Shaun
• Primary Examiner(s): Buttram, Todd
• Application Number:
  US15/198,861
• Publication Number:
  US 20180003965A1
• Time in Patent Office: 824 Days
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 27/0101 characterised by optical features G02B27/0172 takes precedence
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G08G 1/052 with provision for determining speed or overspeed speed measuring in general G01P
• CPC Class Codes: G08G 1/08 according to detected number or speed of vehicles
• CPC Class Codes: G08G 1/163 involving continuous checking
• CPC Class Codes: G08G 1/164 Centralised systems, e.g. external to vehicles
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

Systems and methods for enhancing traffic safety through augmented reality and shared data are disclosed. An example method includes, identifying a second transportation vehicle that at least partially blocks a first view of a first user associated with a first transportation vehicle; and obtaining second view data from the second transportation vehicle. The second view data is descriptive of a second view of a second user associated with the second transportation vehicle, and the second view includes an additional view that was blocked in the first view by the second transportation vehicle. In some implementations, the method includes presenting the second view on a first display associated with the first transportation vehicle. In other implementations, for example where the first transportation vehicle is a driver-less vehicle, the method includes providing an electronic signal identifying the second view to the first transportation vehicle.

Claims

1. A device, comprising:

a non-transitory memory; and

one or more hardware processors coupled to the non-transitory memory and configured to read instructions from the non-transitory memory to cause the device to perform operations comprising;

determining, based on a first view associated with a first transportation vehicle, that a second transportation vehicle at least partially blocks the first view associated with the first transportation vehicle;

obtaining second view data from a first set of sensors of a first sensor type associated with the second transportation vehicle, wherein the second view data corresponds to a second view associated with the second transportation vehicle, and wherein the second view includes an additional view that was blocked in the first view by the second transportation vehicle;

determining, from the second view data, that an object is approaching the first transportation vehicle;

in response to determining that the object is approaching the first transportation vehicle from the second view data and before presenting the second view data in the first transportation vehicle, identifying an external device that provides a third view that includes at least a portion of the additional view based on a location of the external device, wherein the external device is external to the first transportation vehicle and the second transportation vehicle;

obtaining third view data corresponding to the third view from a second set of sensors of the external device, wherein the second set of sensors are of a second sensor type different from the first sensor type;

validating the second view data by identifying data within the third view data indicating the object is approaching the first transportation vehicle; and

in response to validating the second view data, presenting the second view data on a first display associated with the first transportation vehicle.

2. The device of claim 1, wherein the external device includes one of:

a traffic control device or a lighting device.

3. The device of claim 2, wherein the external device is a traffic control device, and wherein the operations further comprise:

determining a traffic control signal shown by the traffic control device; and

determining that the object present in the second view is violating the traffic control signal, wherein

the second view data is presented on the first display associated with the first transportation vehicle in response to the determining that the object present in the second view is violating the traffic control signal.

4. The device of claim 1, wherein the operations further comprise:

obtaining rating data associated with the second view data; and

determining whether to present the second view data on the first display in accordance with the rating data.

5. The device of claim 1, wherein the operations further comprise:

determining, based on the third view data, a travel speed associated with the object present in the second view is more than a predetermined speed, wherein

the second view data is presented on the first display associated with the first transportation vehicle in response to the determining that the travel speed associated with the object is more than the predetermined speed.

6. The device of claim 1, wherein the operations are performed without any user input, in response to determining that the first transportation vehicle is moving below a predetermined speed.

7. The device of claim 1, wherein the operations are performed without any user input, in response to determining that the second transportation vehicle is at least partially blocking the first view.

8. The device of claim 1, wherein the second view data is obtained from the second transportation vehicle through a directional Wi-Fi or Bluetooth communication channel between the first transportation vehicle and the second transportation vehicle.

9. The device of claim 1, wherein the second view data includes image data captured by a camera installed on the second transportation vehicle, and wherein the third view data comprises data obtained from a radar sensor.

10. The device of claim 1, wherein determining that the object is approaching the first transportation vehicle from the second view data includes:

obtaining an image from the second view data; and

applying an object recognition technique to the obtained image included in the second view data to determine that the obtained image includes the object.

11. The device of claim 1, wherein validating the second view data includes:

obtaining sound data from the third view data; and

applying a sound analysis technique to the obtained sound data to verify that vehicle sound is present in the obtained sound data.

12. The device of claim 1, wherein the third view data includes one of:

sound data, image data, or radar data.

13. The device of claim 1, wherein the operations further comprise in response to validating the second view data, automatically adjusting a parameter of the first transportation vehicle.

14. The device of claim 13, wherein adjusting the parameter comprises applying brakes on the first transportation vehicle.

15. The device of claim 1, wherein the operations further comprise:

analyzing the second view data to determine a location of the object with respect to the first transportation vehicle; and

determining the first display from a plurality of displays associated with the first transportation vehicle for presenting the second view data based on the determined location of the object.

16. A method, comprising:

determining, by one or more hardware processors based on a first view associated with a first transportation vehicle, that a second transportation vehicle at least partially blocks the first view associated with the first transportation vehicle;

obtaining, by the one or more hardware processors, second view data from a first set of sensors of a first sensor type associated with the second transportation vehicle, wherein the second view data is descriptive of a second view associated with the second transportation vehicle, and wherein the second view includes an additional view that was blocked in the first view by the second transportation vehicle;

determining, by the one or more hardware processors from the second view data, that an object is approaching the first transportation vehicle;

in response to determining that the object is approaching the first transportation vehicle from the second view data and before presenting the second view data in the first transportation vehicle, identifying, by the one or more hardware processors, an external device that provides a third view that includes at least a portion of the additional view based on a location of the external device, wherein the external device is external to the first transportation vehicle and the second transportation vehicle;

obtaining, by the one or more hardware processors, third view data corresponding to the third view from a second set of sensors of the external device, wherein the second set of sensors is of a second sensor type different from the first sensor type;

validating, by the one or more hardware processors, the second view data by identifying data within the third view data indicating the object is approaching the first transportation vehicle; and

in response to validating the second view data, providing, by the one or more hardware processors, an electronic signal identifying the second view to the first transportation vehicle.

17. The method of claim 16, wherein the first transportation vehicle is a driver-less vehicle.

18. The method of claim 16 wherein the external device includes one of:

a traffic control device or a lighting device.

19. The method of claim 16, further comprising in response to validating the second view data, automatically adjusting a parameter of the first transportation vehicle.

20. The method of claim 19, wherein adjusting the parameter comprises applying brakes on the first transportation vehicle.