Coupled range and intensity imaging for motion estimation

• US 9,069,075 B2
• Filed: 02/10/2012
• Issued: 06/30/2015
• Est. Priority Date: 02/10/2012
• Status: Active Grant

• Current Assignee: GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: GM Global Technology Operations LLC (General Motors Company)
• Inventors: Zeng, Shuqing
• Primary Examiner(s): Black, Thomas G
• Assistant Examiner(s): Nolan, Peter D
• Application Number:
  US13/370,685
• Publication Number:
  US 20130211657A1
• Time in Patent Office: 1,236 Days
• Field of Search: 701/300, 701/301, 348/139, 348/169, 348/170, 356/4.01, 340/435, 340/436
• US Class Current: 1/1
• CPC Class Codes: G01S 17/58 Velocity or trajectory determination systems; Sense-of-movement determination systems
• CPC Class Codes: G01S 17/894 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/269 using gradient-based methods
• CPC Class Codes: G08G 1/16 Anti-collision systems road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08
• CPC Class Codes: G08G 1/165 for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

A method and system may operate a coupled range imager and intensity imager to obtain a sequence of at least two automatically registered images, each automatically registered image including concurrently acquired range data and intensity data with regard to an imaged scene. The registered range and intensity data is analyzed to yield an estimated motion of an element of an imaged object of the imaged scene.

Claims

1. A method comprising:

directing modulated light at an object in motion over at least one set time interval;

using a beam splitter to direct a first portion of the modulated light after reflection from the object onto a range imager and a second portion of the modulated light onto an intensity imager so as to generate time-dependent, pixel-specific, intensity data and range data of the object; and

using a processor to;

perform pixel-level fusion of the intensity data and the range data, and

calculate three-dimensional, pixel-level motion flow from the pixel-level fusion of the intensity data and the range data,

wherein the pixel-level fusion is implemented by minimizing

2. A method comprising:

directing modulated light at an object in motion over at least one set time interval;

using a beam splitter to direct a first portion of the modulated light after reflection from the object onto a range imager and a second portion of the modulated light onto an intensity imager so as to generate time-dependent, pixel-specific, intensity data and range data of the object; and

using a processor to;

perform pixel-level fusion of the intensity data and the range data,

calculate three-dimensional, pixel-level motion flow from the pixel-level fusion of the intensity data and the range data,

wherein the pixel-level fusion is implemented by minimizing

3. A system comprising:

a modulated light source configured to direct modulated light at an object in motion over at least one set time interval;

a beam splitter configured to direct a first portion of the modulated light after reflection from the object onto a range imager detector and a second portion of the modulated light onto an intensity imager detector so as to generate time-dependent, pixel-specific, intensity data and range data of the object; and

a processor configured to;

perform pixel-level fusion of the intensity data and the range data, and

calculate three-dimensional, pixel-level motion flow from the pixel-level fusion of the intensity data and the range data,

the pixel-level fusion is implemented by minimizing

4. A system comprising:

a modulated light source configured to direct modulated light at an object in motion over at least one set time interval;

a beam splitter configured to direct a first portion of the modulated light after reflection from the object onto a range imager detector and a second portion of the modulated light onto an intensity imager detector so as to generate time-dependent, pixel-specific, intensity data and range data of the object; and

a processor configured to;

perform pixel-level fusion of the intensity data and the range data, and

calculate three-dimensional, pixel-level motion flow from the pixel-level fusion of the intensity data and the range data, pixel-level fusion is implemented by minimizing