Inter-vehicle authentication using visual contextual information

• US 9,842,263 B2
• Filed: 11/10/2015
• Issued: 12/12/2017
• Est. Priority Date: 11/10/2015
• Status: Active Grant

• Current Assignee: Carnegie Mellon University, Ford Global Technologies, LLC (Ford Motor Company)
• Original Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Inventors: Kim, Yu Seung, Han, Jun, Tague, Patrick
• Primary Examiner(s): Mistry, Oneal R
• Application Number:
  US14/937,583
• Publication Number:
  US 20170132477A1
• Time in Patent Office: 763 Days
• CPC Class Codes: G06F 21/445 by mutual authentication, e.g. between devices or programs
• CPC Class Codes: G06V 20/62 Text, e.g. of license plates, overlay texts or captions on TV images
• CPC Class Codes: G08G 1/096716 where the received information does not generate an automatic action on the vehicle control
• CPC Class Codes: G08G 1/096741 where the source of the transmitted information selects which information to transmit to each vehicle
• CPC Class Codes: G08G 1/096791 where the origin of the information is another vehicle
• CPC Class Codes: H04L 2209/84 Vehicles
• CPC Class Codes: H04L 9/0844 with user authentication or key authentication, e.g. ElGamal, MTI, MQV-Menezes-Qu-Vanstone protocol or Diffie-Hellman protocols using implicitly-certified keys
• CPC Class Codes: H04W 12/069 using certificates or pre-shared keys
• CPC Class Codes: H04W 12/65 Environment-dependent, e.g. using captured environmental data
• CPC Class Codes: H04W 4/46 for vehicle-to-vehicle communication [V2V]

Abstract

Authentication of vehicles in preparation for V2V communication includes vehicle A taking a picture of vehicle B (V_A) and vehicle B taking a picture of vehicle A (V_B). These pictures are then shared. Vehicle A determines the relative location of vehicle B indicated in V_A and V_B. If they agree, then vehicle A and B authenticate one another, such as using Diffie-Hellman key exchange. Identifying vehicle A and vehicle B in the pictures may include identifying the license plates of vehicle A and vehicle B in the pictures. Vehicles A and B may exchange license plate numbers prior to taking of the pictures. Image spoofing may be prevented by taking both forward and rearward facing pictures by both vehicles. Objects in pictures facing the same direction may be identified to verify that the pictures were taken nearly simultaneously.

Claims

1. A method comprising performing, by a controller of a vehicle A:

receiving a first image from a camera of the vehicle A;

receiving a second image from a vehicle B;

performing at least one of (a) and (b),

wherein (a) comprises—

(i) determining a first estimated distance between vehicle A and vehicle B according to analysis of the first image;

(ii) determining a second estimated distance between vehicle A and vehicle B according to analysis of the second image; and

(iii) determining that a difference between the first estimated distance and the second estimated distance meets a threshold distance condition; and

wherein (b) comprises—

(iv) determining a first estimated angle between vehicle A and vehicle B according to analysis of the first image;

(v) determining a second estimated angle between vehicle A and vehicle B according to analysis of the second image; and

(vi) determining that a difference between the first estimated angle and the second estimated angle meets a threshold angle condition; and

in response to at least one of (iii) and (vi), authenticating vehicle B.

2. The method of claim 1, wherein determining the first estimated distance comprises:

identifying, by the controller of the vehicle A, an image of the vehicle B in the first image;

determining the first estimated distance according to a size and location of the image of the vehicle B in the first image;

wherein determining the first estimated angle comprises;

identifying, by the controller of the vehicle A, the image of the vehicle B in the first image;

determining the first estimated angle according to the size and location of the image of the vehicle B in the first image.

3. The method of claim 2,

wherein performing at least one of (a) and (b) comprises determining both of (a) and (b);

and

wherein authenticating vehicle B in response to at least one of (iii) and (vi) comprises authenticating vehicle B in response to both of (iii) and (vi).

4. The method of claim 2, wherein:

receiving the first image from the camera of the vehicle A comprises receiving a first forward image from a forward facing camera mounted to the vehicle A and receiving a first rearward image from a rearward facing camera mounted to the vehicle A; and

receiving the second image from the vehicle B comprises receiving a second forward image from a forward facing camera mounted to the vehicle B and receiving a second rearward image from a rearward facing camera mounted to the vehicle B.

5. The method of claim 2, wherein:

identifying the image of the vehicle B in the first image comprises identifying a license plate number of the vehicle B in the first image;

wherein identifying the image of the vehicle A in the second image comprises identifying a license plate number of the vehicle A in the second image.

6. The method of claim 5, further comprising receiving, by the controller of the vehicle A from the vehicle B, a message including the license plate number of the vehicle B.

7. The method of claim 3, wherein:

determining the first estimated distance comprises—

determining a first dimension (h_image1) of a license plate of the vehicle B in the first image in pixels; and

determining the first estimated distance (d₁) as equal to d_init*h_init/h_image1, where d_init is a calibration distance and h_init is a test dimension in pixels of a test license plate positioned at d_init from a test camera; and

determining the second estimated distance comprises—

determining a second dimension (h_image2) of a license plate of the vehicle A in the second image in pixels; and

determining the second estimated distance (d₂) as equal to d_init*h_init/h_image2.

8. The method of claim 7, wherein:

determining the first estimated angle comprises—

determining a first pixel offset (d_shift1) of the license plate of the vehicle B in the first image from a center of the first image in pixels;

determine a first distance offset (d_shift1m) as equal to (h_m*d_shift1/h_image1), where h_m is a measured dimension of a test license plate;

and

determining the first estimated angle as equal to Arccos(d₁/d_shift1m); and

determining the second estimated angle comprises—

determining a second pixel offset (d_shift2) of the license plate of the vehicle A in the second image from a center of the second image in pixels;

determine a second distance offset (d_shift2m) as equal to (h_m*d_shift2/h_image2);

determining the second estimated angle as equal to Arccos(d₂/d_shift2m).

9. The method of claim 1, wherein the camera of the vehicle A is a first camera, the method further comprising authenticating the vehicle B in response to determining that one or more background objects in the second image correspond to objects in an image received from a second camera mounted to the vehicle A and facing in an opposite direction to the first camera.

10. The method of claim 1, wherein authenticating the vehicle B comprises performing Diffie-Hellman key exchange between the vehicle A and the vehicle B.

20. The method of claim 1, wherein authenticating the vehicle B comprises performing Diffie-Hellman key exchange between the vehicle A and the vehicle B according to the algorithm of Table 1.

11. A system comprising:

a vehicle;

a camera mounted to the vehicle;

a controller mounted to the vehicle, the controller programmed to;

receive a first image from the camera;

receive a second image from a vehicle B;

identify an image of the vehicle B in the first image;

identify an image of the vehicle A in the second image;

determine at least one of a first estimated distance to the vehicle B from the vehicle A and a first estimated angle to the vehicle B from the vehicle A according to a location of the image of the vehicle B in the first image;

determine at least one of a second estimated distance to the vehicle A from the vehicle B and a second estimated angle to the vehicle A from the vehicle B according to a location of the image of the vehicle A in the second image; and

if at least one of (a) and (b) is true, authenticate vehicle B and perform secured communication with vehicle B, wherein (a) is a difference between the second estimated distance and the first estimated distance being within a distance tolerance and wherein (b) is a difference between the first estimated angle and the second estimated angle being within an angle tolerance.

12. The system of claim 11, wherein the controller is programmed to authenticate vehicle B and perform secured communication with vehicle B only if both of (a) and (b) are true.

13. The system of claim 11, wherein the controller is further programmed to:

receive a first image from the camera of the vehicle A by receiving a first forward image from a forward facing camera mounted to the vehicle A and receiving a first rearward image from a rearward facing camera mounted to the vehicle A; and

receive a second image from the vehicle B by receiving a second forward image from a forward facing camera mounted to the vehicle B and receiving a second rearward image from a rearward facing camera mounted to the vehicle B.

14. The system of claim 11, wherein the controller is further programmed to:

identify the image of the vehicle B in the first image by identifying a license plate number of the vehicle B in the first image;

identify the image of the vehicle A in the second image by identifying a license plate number of the vehicle A in the second image.

15. The system of claim 14, wherein the controller is further programmed to:

receive a message including the license plate number of the vehicle B.

16. The system of claim 15, wherein the controller is further programmed to:

determine the at least one of the first estimated distance to the vehicle B from the vehicle A and the first estimated angle to the vehicle B from the vehicle A according to the location of the image of the vehicle B in the first image by—

determining a first dimension (h_image1) of a license plate of the vehicle B in the first image in pixels; and

determining the first estimated distance (d₁) as equal to d_init*h_init/h_image1, where d_init is a calibration distance and h_init is a test dimension in pixels of a test license plate positioned at d_init from a test camera;

determine the at least one of the second estimated distance to the vehicle A from the vehicle B and the second estimated angle to the vehicle A from the vehicle B according to the location of the image of the vehicle A in the second image by—

determining a second dimension (h_image2) of a license plate of the vehicle A in the second image in pixels; and

determining the second estimated distance (d₂) as equal to d_init*h_init/h_image2.

17. The system of claim 16, wherein the controller is further programmed to:

determine the at least one of the first estimated distance to the vehicle B from the vehicle A and the first estimated angle to the vehicle B from the vehicle A according to the location of the image of the vehicle B in the first image by—

determining a first pixel offset (d_shift1) of the license plate of the vehicle B in the first image from a center of the first image in pixels;

determine a first distance offset (d_shift1m) as equal to (h_m*d_shift1/h_image1), where h_m is a measured dimension of a test license plate;

and

determining the first estimated angle as equal to Arccos(d₁/d_shift1m); and

determine the at least one of the second estimated distance to the vehicle A from the vehicle B and the second estimated angle to the vehicle A from the vehicle B according to the location of the image of the vehicle A in the second image by—

determining a second pixel offset (d_shift2) of the license plate of the vehicle A in the second image from a center of the second image in pixels;

determine a second distance offset (d_shift2m) as equal to (h_m*d_shift2/h_image2);

determining the second estimated angle as equal to Arccos(d₂/d_shift2m).

18. The system of claim 11, wherein the camera of the vehicle A is a first camera;

wherein the controller is further programmed to authenticate the vehicle B only if one or more background objects in the second image correspond to objects in an image received from a second camera mounted to the vehicle A and facing in an opposite direction to the first camera.

19. The method of claim 11, wherein authenticating the vehicle B comprises performing Diffie-Hellman key exchange between the vehicle A and the vehicle B.