Stereo correspondence model fitting

• US 10,249,052 B2
• Filed: 12/19/2012
• Issued: 04/02/2019
• Est. Priority Date: 12/19/2012
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Cohen, Scott D., Price, Brian L., Zhang, Chenxi
• Primary Examiner(s): Vazquez Colon, Maria E
• Application Number:
  US13/720,316
• Publication Number:
  US 20140168215A1
• Time in Patent Office: 2,295 Days
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 17/10 Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 7/593 from stereo images

Abstract

Stereo correspondence model fitting techniques are described. In one or more implementations, a model may be fit to a region in at least one of a plurality of stereoscopic images of an image scene. The model may then be used as part of a stereo correspondence calculation, which may include computing disparities for the region based at least in part on correspondence to the model.

Claims

1. A method implemented by at least one computing device, the method comprising:

selecting a model from a plurality of models automatically and without user intervention by the at least one computing device, the selecting based at least in part on an indication that pixels included in respective ones of a plurality of stereoscopic images correspond to each other;

fitting the model to the plurality of stereoscopic images by the at least one computing device based at least in part on the indication that the pixels correspond to each other;

calculating stereo correspondence of image data of the plurality of stereoscopic images by the at least one computing device, the calculating based on minimizing a cost function determined from disparities indicating a difference in locations of the pixels, the disparities determined at least in part based on the model, the cost function including a first cost of color similarity in the pixels that correspond to each other in the stereoscopic images and a second cost of smoothness of neighboring pixels in one of the stereoscopic images; and

determining, based on the stereo correspondence, a region of one of the plurality of stereoscopic images that corresponds to another region of an additional one of the plurality of stereoscopic images.

2. The method as described in claim 1, further comprising computing the correspondence of the pixels included in the respective ones of the plurality of stereoscopic images automatically and without user intervention.

3. The method as described in claim 2, wherein the computing of the correspondence of the pixels is based at least in part on identification of features in the plurality of stereoscopic images.

4. The method as described in claim 2, further comprising outputting a user interface that includes the indication of the pixels that correspond to each other, the user interface configured to support user interaction to verify the correspondence.

5. The method as described in claim 4, wherein the user interface is configured to remove the indication that the pixels correspond to each other.

6. The method as described in claim 1, further comprising outputting a user interface that is configured to support user interaction to generate the indication that the pixels correspond to each other.

7. The method as described in claim 6, wherein the user interface is configured to support the user interaction by detecting selection of a plurality of different individual locations in respective ones of the plurality of stereoscopic images.

8. The method as described in claim 6, wherein the user interface is configured to support the user interaction by detecting selection of a boundary in respective ones of the plurality of stereoscopic images.

9. The method as described in claim 1, further comprising outputting a user interface that is configured to change the automatically selected model.

10. The method as described in claim 1, wherein the model describes a three-dimensional object.

11. The method as described in claim 1, wherein the model describes a surface.

12. The method as described in claim 1, wherein the calculating is further based on a weight based on correspondence to the model.

13. A system comprising:

at least one module implemented at least partially in hardware, the at least one module configured to output a user interface to support user interaction to specify a plurality of models and regions in one or more of a plurality of stereoscopic images to which the respective models are to be fit; and

one or more modules implemented at least partially in hardware, the one or more modules configured to;

calculate stereo correspondence of the regions of the plurality of stereoscopic images, the calculation based on minimizing a cost function determined from disparities indicating a difference in locations of pixels in the regions, the disparities determined at least in part based on the respective models that are fit to the regions, the cost function including a first cost of color similarity in pixels that correspond to each other in the stereoscopic images and a second cost of smoothness of neighboring pixels in one of the stereoscopic images; and

determine, based on the stereo correspondence, a region of one of the plurality of stereoscopic images that corresponds to another region of an additional one of the plurality of stereoscopic images.

14. The system as described in claim 13, wherein the user interface is configured to support user interaction to specify the regions by indicating correspondence of pixels between the plurality of stereoscopic images.

15. The system as described in claim 13, wherein the respective models describe a surface or a three dimensional object.

20. The system as described in claim 13, wherein the user interface is configured to support the user interaction by detecting selection of a boundary in respective ones of the plurality of stereoscopic images.

16. A method implemented by at least one computing device, the method comprising:

outputting, by the computing device, a user interface that is configured to support user interaction to select a model from a plurality of models integrated within an application corresponding to the user interface;

fitting, by the computing device, the model to the plurality of stereoscopic images based at least in part on an indication that pixels included in respective ones of a plurality of stereoscopic images correspond to each other;

calculating, by the computing device, stereo correspondence of image data of the plurality of stereoscopic images, the calculating based on minimizing a cost function determined from disparities indicating a difference in locations of the pixels, the disparities determined at least in part based on the model, the cost function including a first cost of color similarity in the pixels that correspond to each other in the stereoscopic images and a second cost of smoothness of neighboring pixels in one of the stereoscopic images; and

determining, based on the stereo correspondence, a region of one of the plurality of stereoscopic images that corresponds to another region of an additional one of the plurality of stereoscopic images.

17. The method as described in claim 16, further comprising computing the correspondence of the pixels included in the respective one of the plurality of stereoscopic images automatically and without user intervention.

18. The method as described in claim 17, further comprising outputting a user interface that is configured to support user interaction to generate the indication that the pixels correspond to each other.

19. The method as described in claim 16, wherein the calculating is further based on a weight based on correspondence to the model.