Presentation of shared threat information in a transportation-related context

• US 9,159,236 B2
• Filed: 03/29/2012
• Issued: 10/13/2015
• Est. Priority Date: 12/01/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Elwha LLC (Intellectual Ventures LLC)
• Inventors: Lord, Richard T., Lord, Robert W., Myhrvold, Nathan P., Tegreene, Clarence T., Hyde, Roderick A., Wood, Lowell L. Jr., Ishikawa, Muriel Y., Wood, Victoria Y. H., Whitmer, Charles, Bahl, Paramvir, Burger, Douglas C., Chandra, Ranveer, Gates, William H. III, Holman, Paul, Kare, Jordin T., Mundie, Craig J., Paek, Tim, Tan, Desney S., Zhong, Lin, Dyor, Matthew G.
• Primary Examiner(s): Nguyen, Kim T
• Application Number:
  US13/434,475
• Publication Number:
  US 20130144490A1
• Time in Patent Office: 1,293 Days
• Field of Search: 701/1, 701/41, 701/48, 701/51, 701/53, 701/65, 701/70, 701/99, 701/301, 701/412, 701/414
• US Class Current: 1/1
• CPC Class Codes: B60T 17/22 Devices for monitoring or checking brake systems; Signal devices
• CPC Class Codes: B60T 2201/022 Collision avoidance systems
• CPC Class Codes: B60T 2210/36 Global Positioning System [GPS]
• CPC Class Codes: G08G 1/164 Centralised systems, e.g. external to vehicles

Abstract

Techniques for ability enhancement are described. In some embodiments, devices and systems located in a transportation network share threat information with one another, in order to enhance a user'"'"'s ability to operate or function in a transportation-related context. In one embodiment, a process in a vehicle receives threat information from a remote device, the threat information based on information about objects or conditions proximate to the remote device. The process then determines that the threat information is relevant to the safe operation of the vehicle. Then, the process modifies operation of the vehicle based on the threat information, such as by presenting a message to the operator of the vehicle and/or controlling the vehicle itself.

Claims

1. A method for enhancing ability in a transportation-related context, the method comprising:

at a first vehicle, receiving threat information from a remote device, the threat information based at least in part on information about objects and/or conditions proximate to the remote device, wherein the receiving threat information includes receiving threat information determined by a wearable device of an occupant of a second vehicle;

determining that the threat information is relevant to safe operation of the first vehicle; and

modifying operation of the first vehicle based on the threat information, including presenting a message based on the threat information to an operator of the first vehicle, wherein the presenting the message includes presenting the message via a wearable device of the operator of the first vehicle.

2. The method of claim 1, wherein the receiving threat information includes:

receiving threat information determined based on information about driving conditions proximate to the remote device.

3. The method of claim 2, wherein the information about driving conditions indicates that icy surface conditions are present proximate to the remote device.

4. The method of claim 2, wherein the information about driving conditions indicates that wet surface conditions are present proximate to the remote device, wherein the wet surface conditions are inferred based on reports of precipitation received from a weather system.

5. The method of claim 2, wherein the information about driving conditions indicates that oily surface conditions are present proximate to the remote device.

6. The method of claim 2, wherein the information about driving conditions indicates that a limited visibility condition is present proximate to the remote device, wherein the limited visibility condition is determined based on a light sensor indicating glare from the sun.

7. The method of claim 2, wherein the information about driving conditions indicates that there is an accident proximate to the remote device, wherein presence of the accident is determined based on information received from an accelerometer that is configured to detect accidents based on sudden deceleration.

8. The method of claim 1, wherein the receiving threat information includes:

receiving threat information determined based on information about the second vehicle proximate to the remote device, wherein the information about the second vehicle indicates at least one of;

that the second vehicle is driving erratically, that the second vehicle is driving with excessive speed; and

/or that the second vehicle is driving too slowly.

9. The method of claim 1, wherein the receiving threat information includes:

receiving threat information determined based on information about a pedestrian proximate to the remote device.

10. The method of claim 1, wherein the receiving threat information includes:

receiving threat information determined at the second vehicle with respect to information about objects and/or conditions received at the second vehicle.

11. The method of claim 10, wherein the receiving threat information determined at the second vehicle includes:

receiving threat information determined by a computing device installed in the second vehicle.

12. The method of claim 10, wherein the receiving threat information determined at the second vehicle includes:

receiving motion-related information from a velocity and/or position sensor attached to the second vehicle.

13. The method of claim 1, wherein the receiving threat information includes:

receiving threat information determined by a road-based device with respect to information about objects and/or conditions received at the road-based device.

14. The method of claim 13, wherein the receiving threat information determined by a road-based device includes:

receiving threat information determined by a road-based computing device configured to receive the information about objects and/or conditions from vehicles proximate to the road-based computing device.

15. The method of claim 13, wherein the receiving threat information determined by a road-based device includes:

receiving threat information determined by a road-based computing device configured to receive the information about objects and/or conditions from road-based sensors.

16. The method of claim 13, wherein the road-based device is a sensor attached to a structure proximate to the first vehicle.

17. The method of claim 16, wherein the structure proximate to the first vehicle is one of a utility pole, a traffic control signal support, a building, a street light, a tunnel wall, a bridge, an overpass, a flyover, a communication tower, a traffic kiosk, an advertisement structure, a roadside sign, an information/regulatory display, and/or a vehicle toll reader.

18. The method of claim 16, wherein the receiving threat information determined by a road-based device includes:

receiving an image of the second vehicle from a camera deployed at an intersection.

19. The method of claim 16, wherein the receiving threat information determined by a road-based device includes:

receiving ranging data from a range sensor deployed at an intersection, the ranging data representing a distance between the second vehicle and the intersection.

20. The method of claim 16, wherein the road-based device includes at least one of:

a camera, a microphone, a radar gun, and/or a range sensor.

21. The method of claim 16, wherein the road-based device includes a receiver operable to receive motion-related information transmitted from the second vehicle, the motion-related information including at least one of a position of the second vehicle, a velocity of the second vehicle, and/or a trajectory of the second vehicle.

22. The method of claim 13, wherein the road-based device is embedded in a roadway.

23. The method of claim 22, wherein the road-based device includes an induction loop embedded in the roadway, the induction loop configured to detect the presence and/or velocity of the second vehicle.

24. The method of claim 23, wherein the receiving threat information determined by a road-based device includes:

receiving motion-related information from the induction loop, the motion-related information including at least one of a position of the second vehicle, a velocity of the second vehicle, and/or a trajectory of the second vehicle.

25. The method of claim 1, wherein the determining that the threat information is relevant to safe operation of the first vehicle includes:

determining a location associated with the remote device; and

determining whether the first vehicle is approaching the location.

26. The method of claim 25, wherein the location associated with the remote device includes the location of a road-based device.

27. The method of claim 25, wherein the location associated with the remote device includes a location of the second vehicle that includes the remote device.

28. The method of claim 25, wherein the determining whether the first vehicle is approaching the location includes:

determining whether the first vehicle is within a threshold distance from the location associated with the remote device.

29. The method of claim 25, wherein the determining whether the first vehicle is approaching the location includes:

determining whether the first vehicle is moving on trajectory that intersects the location associated with the remote device.

30. The method of claim 1, wherein the determining that the threat information is relevant to safe operation of the first vehicle includes:

determining a threat to the first vehicle based on the threat information;

determining a likelihood associated with the threat; and

determining that the likelihood is greater than a threshold level.

31. The method of claim 30, wherein the determining a threat to the first vehicle based on the threat information includes:

predicting a path of an object identified by the threat information;

predicting a path of the first vehicle; and

determining, based on the paths of the object and the first vehicle, whether the first vehicle and the object will come within a threshold distance of one another.

32. The method of claim 30, wherein the determining a likelihood associated with the threat includes:

determining a likelihood that the first vehicle will collide with the second vehicle identified by the threat information.

33. The method of claim 30, wherein the determining a likelihood associated with the threat includes:

determining a likelihood that the first vehicle will collide with an animal identified by the threat information.

34. The method of claim 30, wherein the determining a likelihood associated with the threat includes:

determining a likelihood that surface conditions identified by the threat information will cause the operator to lose control of the first vehicle.

35. The method of claim 1, wherein the determining that the threat information is relevant to safe operation of the first vehicle includes:

determining that the threat information is relevant based on gaze information associated with the operator of the first vehicle.

36. The method of claim 35, further comprising:

determining that the operator has not looked at the road for more than a threshold amount of time.

37. The method of claim 1, wherein the presenting a message based on the threat information to the operator of the first vehicle includes at least one of:

presenting a warning to the operator;

presenting an instruction to the operator;

directing the operator to accelerate or decelerate;

directing the operator to turn or not to turn.

38. The method of claim 1, wherein the presenting a message based on the threat information to the operator of the first vehicle includes:

presenting the message via an audio output device that is a speaker of a wearable device of the operator.

39. The method of claim 1, wherein the presenting a message based on the threat information to the operator of the first vehicle includes:

presenting the message via a visual display device of a wearable device of the operator, the message instructing the operator to accelerate, decelerate, turn, and/or look towards an oncoming vehicle identified by the threat information.

40. The method of claim 1, wherein the presenting a message based on the threat information to the operator of the first vehicle includes:

providing tactile feedback to the operator of the first vehicle.

41. The method of claim 40, wherein the providing tactile feedback to the operator of the first vehicle includes:

causing a steering device, seat, and/or pedal of the first vehicle to vibrate.

42. The method of claim 1, wherein the modifying operation of the first vehicle includes:

controlling the first vehicle by performing at least one of;

decreasing speed of the first vehicle by applying brakes of the first vehicle and/or by reducing output of an engine of the first vehicle;

increasing speed of the first vehicle by releasing brakes of the first vehicle and/or by increasing output of an engine of the first vehicle; and

/or changing direction of the first vehicle.

43. The method of claim 1, wherein the receiving threat information includes:

receiving threat information determined based on image data received from a camera that is attached to at least one of;

a road-side structure, a wearable device of a pedestrian, or the second vehicle, the image data including multiple images of the second vehicle taken at different times.

44. The method of claim 1, wherein the receiving the threat information includes:

receiving threat information determined based on acoustic source localization performed to determine a position of one of the objects based on multiple audio signals received via multiple microphones.

45. The method of claim 1, further comprising:

identifying multiple threats to the first vehicle, at least one of which is based on the threat information;

identifying a first one of the multiple threats that is more significant than at least one other of the multiple threats; and

instructing the operator of the first vehicle to avoid the first one of the multiple threats.

46. A non-transitory computer-readable medium including contents that are configured, when executed, to cause a computing system to perform a method for enhancing ability in a transportation-related context, the method comprising:

at a first vehicle, receiving threat information from a remote device, the threat information based at least in part on information about objects and/or conditions proximate to the remote device, wherein the receiving threat information includes receiving threat information determined by a wearable device of an occupant of a second vehicle;

determining that the threat information is relevant to safe operation of the first vehicle; and

modifying operation of the first vehicle based on the threat information, including presenting a message based on the threat information to an operator of the first vehicle, wherein the presenting the message includes presenting the message via a wearable device of the operator of the first vehicle.

47. A computing system for enhancing ability in a transportation-related context, the computing system comprising:

a processor;

a memory; and

a module that is stored in the memory and that is configured, when executed by the processor, to perform a method comprising;

at a first vehicle, receiving threat information from a remote device, the threat information based at least in part on information about objects and/or conditions proximate to the remote device, wherein the receiving threat information includes receiving threat information determined by a wearable device of an occupant of a second vehicle;

determining that the threat information is relevant to safe operation of the first vehicle; and

modifying operation of the first vehicle based on the threat information, including presenting a message based on the threat information to an operator of the first vehicle, wherein the presenting the message includes presenting the message via a wearable device of the operator of the first vehicle.