Doppler-based segmentation and optical flow in radar images

• US 10,042,047 B2
• Filed: 09/17/2015
• Issued: 08/07/2018
• Est. Priority Date: 09/19/2014
• Status: Active Grant

• Current Assignee: GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: GM Global Technology Operations LLC (General Motors Company)
• Inventors: Stainvas Olshansky, Inna, Bilik, Igal, Bialer, Oded
• Primary Examiner(s): Bythrow, Peter M
• Application Number:
  US14/856,612
• Publication Number:
  US 20160084956A1
• Time in Patent Office: 1,055 Days
• CPC Class Codes: G01S 13/50 Systems of measurement based on relative movement of target
• CPC Class Codes: G01S 13/58 Velocity or trajectory determination systems; Sense-of-movement determination systems
• CPC Class Codes: G01S 13/89 for mapping or imaging
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G06T 2207/10044 Radar image
• CPC Class Codes: G06T 7/269 using gradient-based methods

Abstract

In an embodiment, a method for processing a sequence of images is provided. The method receives a sequence of images generated by a radar. Each image includes a plurality of pixels. Each pixel includes radial velocity information. The method estimates optical flow for the pixels within an image in the sequence of images by using the radial velocity information for the pixels as a constraint.

Claims

1. A computer-implemented method for processing a sequence of images, comprising:

receiving a sequence of images generated by a radar, each image including a plurality of pixels, each pixel including radial velocity information;

estimating, by a computer, optical flow for the pixels within an image in the sequence of images by using the radial velocity information for the pixels as a constraint, wherein the optical flow is indicative of an object; and

implementing, by the computer, a response at a vehicle based on the object indicated by the optical flow.

2. The method of claim 1, wherein the radial velocity information indicates a radial velocity of the pixel measured by a radar based on a motion of an object relative to the radar.

3. The method of claim 1, further comprising using an optical flow technique that is adapted to using the radial velocity information of the pixels.

4. The method of claim 1, wherein each pixel further includes intensity information, wherein the estimation is performed by using the intensity information in addition to the radial velocity information for the pixels.

5. The method of claim 1, wherein the radial velocity information for the pixel is used as a constraint to solve the intensity constancy constraint equation for the optical flow for the pixel.

6. The method of claim 1, wherein the estimating comprises assuming the radial velocities of a plurality of pixels that are spatially adjacent to a particular pixel in an image are constant.

7. The method of claim 1, wherein each pixel further includes location information comprising an x-coordinate value, a y-coordinate value and a z-coordinate value.

8. A system for processing a sequence of images, comprising:

a radar configured to generate a sequence of images, each image including a plurality of pixels, each pixel including radial velocity information; and

an image processing module configured to estimate optical flow for the pixels within an image in the sequence of images by using the radial velocity information for the pixels as a constraint.

9. The system of claim 8, wherein the radial velocity information indicates a radial velocity of the pixel measured by a radar based on a motion of an object relative to the radar.

10. The system of claim 8, wherein the image processing module is configured to use an optical flow technique that is adapted to using the radial velocity information of the pixels.

11. The system of claim 8, wherein each pixel further includes intensity information, wherein the estimation is performed by using the intensity information in addition to the radial velocity information for the pixels.

12. The system of claim 8, wherein the radial velocity information for the pixel is used as a constraint to solve the intensity constancy constraint equation for the optical flow for the pixel.

13. The system of claim 8, wherein the image processing module is configured to estimate by assuming the radial velocities of a plurality of pixels that are spatially adjacent to a particular pixel in an image are constant.

14. The system of claim 8, wherein each pixel further includes location information comprising an x-coordinate value, a y-coordinate value and a z-coordinate value.

15. A computer program product for processing a sequence of images, the computer program product comprising:

a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions readable by a processing circuit to cause the processing circuit to perform a method comprising;

receiving a sequence of images generated by a radar, each image including a plurality of pixels, each pixel including radial velocity information; and

estimating optical flow for the pixels within an image in the sequence of images by using the radial velocity information for the pixels as a constraint.

16. The computer program product of claim 15, wherein the radial velocity information indicates a radial velocity of the pixel measured by a radar based on a motion of an object relative to the radar.

17. The computer program product of claim 15, wherein the method further comprises using an optical flow technique that is adapted to using the radial velocity information of the pixels.

18. The computer program product of claim 15, wherein each pixel further includes intensity information, wherein the estimation is performed by using the intensity information in addition to the radial velocity information for the pixels.

19. The computer program product of claim 15, wherein the radial velocity information for the pixel is used as a constraint to solve the intensity constancy constraint equation for the optical flow for the pixel.

20. The computer program product of claim 15, wherein the estimating comprises assuming the radial velocities of a plurality of pixels that are spatially adjacent to a particular pixel in an image are constant.