Image and video compression for remote vehicle assistance

• US 9,384,402 B1
• Filed: 04/10/2014
• Issued: 07/05/2016
• Est. Priority Date: 04/10/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Furman, Vadim, Chatham, Andrew Hughes, Ogale, Abhijit, Dolgov, Dmitri
• Primary Examiner(s): Chen, Shelley A
• Application Number:
  US14/249,611
• Time in Patent Office: 817 Days
• Field of Search: 701/211
• US Class Current: 1/1
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60R 2011/0026 Windows, e.g. windscreen
• CPC Class Codes: B60R 2011/004 outside the vehicle
• CPC Class Codes: B60R 2011/008 Adjustable or movable supports
• CPC Class Codes: G01C 21/30 Map- or contour-matching
• CPC Class Codes: G01C 21/3896 Transmission of map data from central databases
• CPC Class Codes: G06T 9/00 Image coding bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/167 Position within a video image, e.g. region of interest [ROI]
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression

Abstract

A vehicle may receive one or more images of an environment of the vehicle. The vehicle may also receive a map of the environment. The vehicle may also match at least one feature in the one or more images with corresponding one or more features in the map. The vehicle may also identify a given area in the one or more images that corresponds to a portion of the map that is within a threshold distance to the one or more features. The vehicle may also compress the one or more images to include a lower amount of details in areas of the one or more images other than the given area. The vehicle may also provide the compressed images to a remote system, and responsively receive operation instructions from the remote system.

Claims

1. A method comprising:

receiving, by a vehicle, one or more images of an environment of the vehicle;

receiving, by the vehicle, a map of the environment;

matching at least one feature in the one or more images with corresponding one or more features in the map;

based on the matching, identifying a given area in the one or more images that corresponds to a portion of the map that is within a threshold distance to the one or more features;

compressing, by the vehicle, the one or more images to include a lower amount of details in areas of the one or more images other than the given area;

providing, by the vehicle, the compressed images to a remote system; and

responsive to providing the compressed images, receiving operation instructions for the vehicle from the remote system.

2. The method of claim 1, further comprising:

altering the received one or more images by replacing areas of the one or more images other than the given area with a given color, wherein compressing the one or more images comprises compressing the altered images.

3. The method of claim 1, further comprising:

blurring, by the vehicle, areas of the one or more images other than the given area, wherein compressing the one or more images comprises compressing the blurred images.

4. The method of claim 1, further comprising:

compressing, by the vehicle, an unaltered version of the received one or more images to generate one or more intermediate images; and

reducing, by the vehicle, an amount of details in areas of the intermediate images that correspond to areas of the received one or more images other than the given area, wherein compressing the one or more images comprises providing one or more final images having the reduced amount of details.

5. The method of claim 1, wherein the one or more images are indicative of a video stream from a camera coupled to the vehicle.

6. The method of claim 1, further comprising:

receiving, by the vehicle from the remote system, configuration data indicative of the threshold distance.

7. The method of claim 1, wherein receiving the operation instructions comprises receiving an input from a human operator.

8. A vehicle comprising:

a camera coupled to the vehicle and configured to capture one or more images of an environment of the vehicle;

one or more processors; and

data storage configured to store instructions executable by the one or more processors to cause the vehicle to;

obtain the one or more images of the environment from the camera;

receive a map of the environment;

match at least one feature in the one or more images with corresponding one or more features in the map;

based on the match, identify a given area in the one or more images that corresponds to a a portion of the map that is within a threshold distance to the one or more features;

compress the one or more images to include a lower amount of details in areas of the one or more images other than the given area;

provide the compressed images to a remote system; and

responsive to providing the compressed images, receive operation instructions for the vehicle from the remote system.

9. The vehicle of claim 8, wherein the instructions executable by the one or more processors further cause the vehicle to:

alter the received one or more images by replacing areas of the one or more images other than the given area with a given color, wherein compressing the one or more images comprises compressing the altered images.

10. The vehicle of claim 8, wherein the instructions executable by the one or more processors further cause the vehicle to:

blur areas of the one or more images other than the given area, wherein compressing the one or more images comprises compressing the blurred images.

11. The vehicle of claim 8, wherein the instructions executable by the one or more processors further cause the vehicle to:

compress an unaltered version of the received one or more images to generate one or more intermediate images; and

reduce an amount of details in areas of the intermediate images that correspond to areas of the received one or more images other than the given area, wherein compressing the one or more images comprises providing one or more final images having the reduced amount of details.

12. The vehicle of claim 8, wherein the one or more images are indicative of a video stream from the camera.

13. The vehicle of claim 8, wherein the instructions executable by the one or more processors further cause the vehicle to:

receive, from the remote system, configuration data indicative of the threshold distance.

14. The vehicle of claim 8, wherein receiving the operation instructions comprises receiving an input from a human operator.

15. A non-transitory computer readable medium having stored therein instructions, that when executed by a device, cause the device to perform functions, the functions comprising:

receiving one or more images of an environment of the device;

receiving a map of the environment;

matching at least one feature in the one or more images with corresponding one or more features in the map;

based on the matching, identifying a given area in the one or more images that corresponds to a portion of the map that is within a threshold distance to the one or more features;

compressing the one or more images to include a lower amount of details in areas of the one or more images other than the given area;

providing the compressed images to a remote system; and

responsive to providing the compressed images, receiving operation instructions for the device from the remote system.

16. The non-transitory computer readable medium of claim 15, the functions further comprising:

altering the received one or more images by replacing areas of the one or more images other than the given area with a given color, wherein compressing the one or more images comprises compressing the altered images.

17. The non-transitory computer readable medium of claim 15, the functions further comprising:

blurring areas of the one or more images other than the given area, wherein compressing the one or more images comprises compressing the blurred images.

18. The non-transitory computer readable medium of claim 15, the functions further comprising:

compressing an unaltered version of the received one or more images to generate one or more intermediate images; and

reducing an amount of details in areas of the intermediate images that correspond to areas of the received one or more images other than the given area, wherein compressing the one or more images comprises providing one or more final images having the reduced amount of details.

19. The non-transitory computer readable medium of claim 15, wherein the one or more images are indicative of a video stream from a camera mounted to the device.

20. The non-transitory computer readable medium of claim 15, the functions further comprising:

receiving, from the remote system, configuration data indicative of the threshold distance.