Determining threats based on information from road-based devices in a transportation-related context

• US 10,079,929 B2
• Filed: 06/09/2016
• Issued: 09/18/2018
• Est. Priority Date: 12/01/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Lord, Richard T., Lord, Robert W., Myhrvold, Nathan P., Tegreene, Clarence T., Hyde, Roderick A., Wood, Jr., Lowell L., Ishikawa, Muriel Y., Wood, Victoria Y. H., Whitmer, Charles, Bahl, Paramvir, Burger, Douglas C., Chandra, Ranveer, Gates, III, William H., Holman, Paul, Kare, Jordin T., Mundie, Craig J., Paek, Tim, Tan, Desney S., Zhong, Lin, Dyor, Matthew G.
• Primary Examiner(s): Rahaman, Mohammed
• Application Number:
  US15/177,535
• Publication Number:
  US 20160286026A1
• Time in Patent Office: 831 Days
• Field of Search: None
• CPC Class Codes: G01S 3/802 Systems for determining direction or deviation from predetermined direction
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G08G 1/04 using optical or ultrasonic detectors
• CPC Class Codes: G08G 1/042 using inductive or magnetic detectors
• CPC Class Codes: G08G 1/052 with provision for determining speed or overspeed speed measuring in general G01P
• CPC Class Codes: G08G 1/056 with provision for distinguishing direction of travel
• CPC Class Codes: G08G 1/087 Override of traffic control, e.g. by signal transmitted by an emergency vehicle
• CPC Class Codes: G08G 1/096716 where the received information does not generate an automatic action on the vehicle control
• CPC Class Codes: G08G 1/096741 where the source of the transmitted information selects which information to transmit to each vehicle
• CPC Class Codes: G08G 1/096775 where the origin of the information is a central station
• CPC Class Codes: G08G 1/096791 where the origin of the information is another vehicle
• CPC Class Codes: G08G 1/162 event-triggered
• CPC Class Codes: G08G 1/164 Centralised systems, e.g. external to vehicles
• CPC Class Codes: H04B 1/385 Transceivers carried on the body, e.g. in helmets
• CPC Class Codes: H04B 2001/3866 carried on the head
• CPC Class Codes: H04M 1/724097 Worn on the head
• CPC Class Codes: H04M 1/724098 Interfacing with an on-board device of a vehicle for hands-free speaking in vehicles H04M1/6075
• CPC Class Codes: H04W 4/40 for vehicles, e.g. vehicle-to-pedestrians [V2P]
• CPC Class Codes: H04W 4/70 : Services for machine-to-machine communication [M2M] or machine type communication [MTC]

Abstract

Techniques for ability enhancement are described. Some embodiments provide an ability enhancement facilitator system (“AEFS”) configured to enhance a user'"'"'s ability to operate or function in a transportation-related context as a pedestrian or a vehicle operator. In one embodiment, the AEFS is configured perform vehicular threat detection based on information received at a road-based device, such as a sensor or processor that is deployed at the side of a road. An example AEFS receives, at a road-based device, information about a first vehicle that is proximate to the road-based device. The AEFS analyzes the received information to determine threat information, such as that the vehicle may collide with the user. The AEFS then informs the user of the determined threat information, such as by transmitting a warning to a wearable device configured to present the warning to the user.

Claims

1. A computer-readable storage device including contents that are configured, when executed, to cause a computing system to perform a method for enhancing ability in a transportation-related context, the method comprising:

receiving, from a first road-based device, information about a first vehicle that is proximate to the first road-based device;

receiving, from a wearable device of an occupant of a second vehicle, motion-related information about the occupant of the second vehicle wearing the wearable device, the motion-related information including position information obtained from a position sensor of the wearable device and including at least one of velocity information or acceleration information;

determining threat information that is relevant to safe operation of the first vehicle based on the information about the first vehicle and the motion-related information about the occupant of the second vehicle wearing wearable device;

presenting the threat information via a wearable device of a driver of the first vehicle; and

transmitting the threat information directly from the computing system to a second road-based device that is along a projected course of travel of the first vehicle,

wherein a road-based device is a fixed device situated on, in, or about a roadway.

2. The computer-readable storage device of claim 1, the method further comprising at least one of:

receiving motion-related information about the first vehicle and/or other objects moving about a roadway;

transmitting to the first vehicle a warning based on the threat information;

presenting the threat information via an output device of the second vehicle of the occupant wearing the wearable device, the output device including a visual display and/or an audio speaker;

transmitting the threat information to the second vehicle;

or

causing the second road-based device to warn drivers that the first vehicle is driving erratically.

3. The computer-readable storage device of claim 2, wherein the transmitting the threat information to the second vehicle includes:

transmitting the threat information to an intermediary server system for distribution to other vehicles in proximity to the occupant of the second vehicle wearing the wearable device.

4. The computer-readable storage device of claim 1, the method further comprising:

performing, at the road-based device, the determining threat information; and

transmitting the threat information includes transmitting the threat information from the road-based device to the wearable device of the driver of the first vehicle.

5. The computer-readable storage device of claim 1, wherein the determining threat information includes at least one of:

determining a likelihood that the first vehicle will collide with the occupant of the second vehicle wearing the wearable device;

determining a likelihood that the first vehicle will collide with some other object;

or

determining that the first vehicle is driving with excessive speed.

6. The computer-readable storage device of claim 1, the method further comprising:

receiving data representing threat information relevant to a third vehicle, the third vehicle not being used for travel by the driver of the first vehicle wearing the wearable device and the occupant of the second vehicle wearing the wearable device; and

determining the threat information based on the data representing threat information relevant to the third vehicle.

7. The computer-readable storage device of claim 6, wherein the receiving data representing threat information relevant to third vehicle includes at least one of:

receiving from the third vehicle at least one of an indication of stalled or slow traffic encountered by the third vehicle, an indication of poor driving conditions experienced by the third vehicle, an indication that the first vehicle is driving erratically, or an image of the first vehicle.

8. The computer-readable storage device of claim 1, wherein the road-based device is a sensor:

attached to a utility pole, a traffic control signal support, a building, and/or a structure proximate to the first vehicle;

or

embedded in a roadway being traveled over by the first vehicle.

9. The computer-readable storage device of claim 1, wherein the receiving information about the first vehicle includes at least one of:

receiving an image of the first vehicle from a camera deployed at an intersection;

receiving ranging data from a range sensor deployed at an intersection, the ranging data representing a distance between the first vehicle and the intersection;

receiving the information about the first vehicle from a sensor attached to the first vehicle;

receiving the information about the first vehicle from a sensor attached to the second vehicle;

receiving the information about the first vehicle from a sensor of the wearable device of the driver of the first vehicle wearing;

or

receiving data representing an audio signal emitted or reflected by the first vehicle.

10. The computer-readable storage device of claim 1, wherein the presenting the threat information via the wearable device of the driver of the first vehicle includes at least one of:

presenting the threat information via an audio output device of the wearable device of the driver of the first vehicle;

presenting the threat information via a visual display device of the wearable device of the driver of the first vehicle, wherein the visual display device displays an indicator that instructs the driver of the first vehicle to accelerate, decelerate, and/or turn.

11. A computing system comprising:

a processor;

memory storing instructions that, when executed by the processor, cause the computing system to perform operations comprising;

receiving, from a road-based device, information about a first vehicle that is proximate to the road-based device;

receiving, from a wearable device of an occupant of a second vehicle, motion-related information about the occupant of the second vehicle wearing the wearable device, the motion-related information including position information obtained from a position sensor of the wearable device and including at least one of velocity information or acceleration information;

determining threat information that is relevant to safe operation of the first vehicle based on the information about the first vehicle and the motion-related information about the occupant of the second vehicle wearing wearable device;

presenting the threat information via a wearable device of a driver of the first vehicle; and

transmitting the threat information directly from the computing system to a second road-based device situated along a projected course of travel of the first vehicle,

wherein a road-based device is a fixed device situated on, in, or about a roadway.

12. The computing system of claim 11, further comprising:

identifying multiple threats to the driver of the first vehicle wearing the wearable device;

identifying a first one of the multiple threats that is more significant than at least one other of the multiple threats; and

instructing the driver of the first vehicle wearing the wearable device to avoid the first one of the multiple threats.

13. The computing system of claim 11, further comprising at least one of:

receiving motion-related information about the first vehicle and/or other objects moving about a roadway;

transmitting to the first vehicle a warning based on the threat information;

presenting the threat information via an output device of the second vehicle of the occupant wearing the wearable device, the output device including a visual display and/or an audio speaker;

transmitting the threat information to the second vehicle;

or

causing the second road-based device to warn drivers that the first vehicle is driving erratically.

14. The computing system of claim 11, further comprising:

performing, at the road-based device, the determining threat information; and

transmitting the threat information includes transmitting the threat information from the road-based device to the wearable device of the driver of the first vehicle.

15. The computing system of claim 11, further comprising:

receiving data representing threat information relevant to a third vehicle, the third vehicle not being used for travel by the driver of the first vehicle wearing the wearable device and the occupant of the second vehicle wearing the wearable device; and

determining the threat information based on the data representing threat information relevant to the third vehicle.

16. The computing system of claim 11, wherein the road-based device is a sensor:

attached to a structure proximate to the first vehicle, wherein the structure is one of a utility pole, a traffic control signal support, or a building;

or

embedded in a roadway being traveled over by the first vehicle.

17. The computer-readable medium of claim 16, wherein the road-based device includes at least one of:

a camera, a microphone, a radar-based speed sensor, light detection and ranging-based speed sensor, a range sensor, or a receiver operable to receive motion-related information about the first vehicle transmitted from the first vehicle.

18. The computing system of claim 11, wherein the receiving information about the first vehicle includes at least one of:

receiving ranging data from a range sensor deployed at an intersection, the ranging data representing a distance between the first vehicle and the intersection;

receiving the information about the first vehicle from a sensor attached to the first vehicle;

receiving the information about the first vehicle from a sensor attached to the second vehicle;

receiving the information about the first vehicle from a sensor of the wearable device of the driver of the first vehicle wearing;

receiving image data from a camera, the image data representing an image of the first vehicle;

receiving an image from a camera that is attached to a road-side structure or the wearable device of the driver of the first vehicle wearing;

or

receiving data representing an audio signal emitted or reflected by the first vehicle.

19. The computing system of claim 11, wherein the determining threat information includes at least one of:

determining a likelihood that the first vehicle will collide with some other object;

determining that the first vehicle is driving with excessive speed;

or

determining that there is slow traffic in proximity to the driver of the first vehicle wearing the wearable device and the occupant of the second vehicle wearing the wearable device.

20. The computing system of claim 11, wherein the wearable device of the driver of the first vehicle wearing comprises a helmet, goggles, eyeglasses, and/or a hat worn by the driver of the first vehicle wearing.