Image and Video Compression for Remote Vehicle Assistance

• US 20160283804A1
• Filed: 06/03/2016
• Published: 09/29/2016
• Est. Priority Date: 04/10/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Furman, Vadim, Chatham, Andrew Hughes, Ogale, Abhijit, Dolgov, Dmitri
• Granted Patent: US 9,767,369 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60R 2011/0026 Windows, e.g. windscreen
• CPC Class Codes: B60R 2011/004 outside the vehicle
• CPC Class Codes: B60R 2011/008 Adjustable or movable supports
• CPC Class Codes: G01C 21/30 Map- or contour-matching
• CPC Class Codes: G01C 21/3896 Transmission of map data from central databases
• CPC Class Codes: G06T 9/00 Image coding bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/167 Position within a video image, e.g. region of interest [ROI]
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression

Abstract

A vehicle may receive one or more images of an environment of the vehicle. The vehicle may also receive a map of the environment. The vehicle may also match at least one feature in the one or more images with corresponding one or more features in the map. The vehicle may also identify a given area in the one or more images that corresponds to a a portion of the map that is within a threshold distance to the one or more features. The vehicle may also compress the one or more images to include a lower amount of details in areas of the one or more images other than the given area. The vehicle may also provide the compressed images to a remote system, and responsively receive operation instructions from the remote system.

Claims

1. A method comprising:

a vehicle accessing a stored representation of an environment of the vehicle;

receiving, from a sensor of the vehicle, sensor data indicative of the environment;

matching a feature of the environment indicated in the sensor data to a corresponding feature indicated in the stored representation;

identifying a first portion of the sensor data, the first portion being associated with a region of the environment that is within a threshold distance to the matched feature;

compressing the sensor data to reduce a second portion of the sensor data; and

providing the compressed sensor data for receipt by a remote system.

2. The method of claim 1, wherein the second portion of the sensor data is associated with a region of the environment that is at a distance greater than the threshold distance to the matched feature.

3. The method of claim 1, further comprising:

receiving, from the remote system, operation instructions for the vehicle, wherein the operation instructions are based on the compressed sensor data.

4. The method of claim 1, wherein the sensor is a camera, and wherein the sensor data comprises one or more images.

5. The method of claim 1, wherein the sensor is a radar unit, and wherein the sensor data comprises an indication of one or more objects in the environment of the vehicle.

6. The method of claim 1, wherein the sensor is a microphone, and wherein the sensor data comprises an indication of sounds in the environment of the vehicle.

7. The method of claim 1, further comprising:

obtaining, via a second sensor of the vehicle, an image of the environment, wherein the image is associated with the sensor data, and wherein matching the feature indicated in the sensor data with the corresponding feature indicated in the stored representation is based on a comparison between the image and the stored representation.

8. The method of claim 7, further comprising:

compressing the image to reduce an amount of details in a portion of the image that corresponds to the region of the environment that is at least at the threshold distance to the matched feature; and

transmitting the compressed image for receipt by the remote system.

9. The method of claim 1, further comprising:

receiving, from the remote system, a selection of the sensor associated with the sensor data, wherein receiving the sensor data from the sensor is based on the received selection.

10. The method of claim 1, further comprising:

receiving, from the remote system, an indication of the threshold distance to the matched feature, wherein identifying the first portion of the sensor data is based on the received indication.

11. The method of claim 1, further comprising:

determining the threshold distance to the matched feature based on lighting conditions associated with the received sensor data, wherein identifying the first portion of the sensor data is based on the determined threshold distance.

12. The method of claim 1, further comprising:

determining the threshold distance to the matched feature based on information included in the stored representation, wherein identifying the first portion of the sensor data is based on the determined threshold distance.

13. A vehicle comprising:

a sensor to provide sensor data indicative of an environment of the vehicle;

one or more processors; and

data storage storing instructions executable by the one or more processors to cause the vehicle to;

access a stored representation of an environment of the vehicle;

obtain the sensor data from the sensor;

match a feature of the environment indicated in the sensor data to a corresponding feature indicated in the stored representation;

identify a first portion of the sensor data, the first portion being associated with a region of the environment that is within a threshold distance to the matched feature;

compress the sensor data to reduce a second portion of the sensor data; and

provide the compressed sensor data for receipt by a remote system.

14. The vehicle of claim 13, further comprising a second sensor to provide an image of the environment, wherein the image is associated with the sensor data, and wherein the instructions executable by the one or more processors further cause the vehicle to:

obtain, via the second sensor, the image of the environment, wherein matching the feature indicated in the sensor data with the corresponding feature indicated in the stored representation is based on a comparison between the image and the stored representation.

15. The vehicle of claim 14, wherein the instructions executable by the one or more processors further cause the vehicle to:

compress the image to reduce an amount of details in a portion of the image that corresponds to the region of the environment that is at least at the threshold distance to the matched feature; and

transmit the compressed image for receipt by the remote system.

16. The vehicle of claim 14, wherein the instructions executable by the one or more processors further cause the vehicle to:

determine the threshold distance to the matched feature based on lighting conditions associated with the image, wherein identifying the first portion of the sensor data is based on the determined threshold distance.

17. A non-transitory computer readable medium having stored therein instructions, that when executed by a device, cause the device to perform functions comprising:

accessing a stored representation of an environment of a vehicle;

receiving sensor data indicative of the environment of the vehicle;

matching a feature of the environment indicated in the sensor data to a corresponding feature indicated in the stored representation;

identifying a first portion of the sensor data, the first portion being associated with a region of the environment that is within a threshold distance to the matched feature;

compressing the sensor data to reduce a second portion of the sensor data; and

causing the vehicle to wirelessly transmit the compressed sensor data.

18. The non-transitory computer readable medium of claim 17, wherein the sensor data is associated with a camera, and wherein the sensor data comprises one or more images.

19. The non-transitory computer readable medium of claim 17, wherein the sensor data is associated with a radar unit, and wherein the sensor data comprises an indication of one or more objects in the environment of the vehicle.

20. The non-transitory computer readable medium of claim 17, wherein the sensor data is associated with a microphone, and wherein the sensor data comprises an indication of sounds in the environment of the vehicle.