SYSTEM AND METHOD FOR CONVERTING A FISH-EYE IMAGE INTO A RECTILINEAR IMAGE

• US 20080129723A1
• Filed: 11/30/2006
• Published: 06/05/2008
• Est. Priority Date: 11/30/2006
• Status: Active Grant

• Current Assignee: The MathWorks, Inc.
• Original Assignee: The MathWorks, Inc.
• Inventors: Sharma, Anurag, Comer, Robert P.
• Granted Patent: US 8,670,001 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06T 3/047 Fisheye or wide-angle transformations
• CPC Class Codes: G06T 5/80 Geometric correction

Abstract

A system and method are disclosed for converting a region of a Fish-Eye or other wide-angle image into a rectilinear image. According to an illustrative embodiment of the present disclosure, a Fish-Eye to rectilinear image conversion block is defined for use in a block diagram of a diagrammatic programming environment. The Fish-Eye to rectilinear image conversion block may use any of a variety of models for Fish-Eye systems, and thus is “model-independent.” Further, intuitive zoom, pan, tilt, and attitude adjustment parameters may be used with the image conversion block, to allow control of the conversion by users lacking advanced understanding of optics.

Claims

1. A graphical image conversion component of a diagrammatic programming environment, for converting a region of a wide-angle image into a rectilinear image, the conversion component comprising:

a first input data interface configured to receive the wide-angle image;

a second input data interface configured to receive selected pan and tilt parameters that cause the image conversion component to select a region of the wide-angle image to convert; and

an output data interface configured to transmit a rectilinear image constructed from the specified region, the rectilinear image including a number of pixels, each pixel constructed from one or more corresponding pixels in the wide-angle image.

2. The image conversion component of claim 1, wherein the wide-angle image is a Fish-Eye image.

3. The image conversion component of claim 1, further comprising:

a computer-implemented representation of a rectilinear system configured to convert a location of a pixel in the rectilinear image into a 3-D position relative to the rectilinear image.

4. The image conversion component of claim 3, further comprising:

a computer-implemented representation of a wide-angle system configured to map the 3-D position to a location of one or more pixels in the wide-angle image.

5. The image conversion component of claim 4, wherein the representation of the wide-angle system comprises:

a model of a wide-angle system selected from the group consisting of an equidistant model, a stereographic model, an equi-solid angle model, and an orthogonal model.

6. The image conversion component of claim 1, wherein the second interface is further configured to receive an approximate orientation of an image acquisition device used to acquire the wide-angle image.

7. The image conversion component of claim 6, wherein the second input data interface is further configured to receive a first adjustment parameter to rotate about a first axis and a second adjustment parameter to rotate about a second axis, wherein the first and second axis are orthogonal to each other.

8. A method for converting a region of a wide-angle image into a rectilinear image, the method comprising the steps of:

converting a location of a pixel in the rectilinear image into a 3-D position relative to the rectilinear image;

in response to a pan parameter, applying a first rotation matrix to the 3-D position;

in response to a tilt parameter, applying a second rotation matrix to the 3-D position;

mapping the 3-D position to a location of one or more pixels in the wide-angle image; and

setting attributes of the pixel in the rectilinear image using attributes of the one or more pixels in the wide angle image.

9. The method of claim 8, wherein the wide-angle image is a Fish-Eye image.

10. The method of claim 9, in which the step of mapping the 3-D position is performed using a model of a wide-angle system selected from the group consisting of an equidistant model, a stereographic model, an equi-solid angle model, and an orthogonal model.

11. The method of claim 8, further comprising the steps of:

in response to an indication that an image acquisition device that captured the wide-angle image was oriented approximately horizontal, the first rotation matrix performing a rotation about a first axis orthogonal to an optical axis of the image acquisition device.

12. The method of claim 11, further comprising the step of:

the second rotation matrix performing a rotation about a second axis that is orthogonal to the optical axis of the image acquisition device, and that is also orthogonal to the first axis.

13. The method of claim 8, further comprising the steps of:

in response to an indication that an image acquisition device that captured the wide-angle image was oriented approximately vertical down-looking, the first rotation matrix performing a rotation about an optical axis of the image acquisition device.

14. The method of claim 13, further comprising the step of:

the second rotation matrix performing a rotation about an axis orthogonal to the optical axis of the image acquisition device.

15. The method of claim 8, further comprising the steps of:

in response to a first adjustment parameter, applying a third rotation matrix to the 3-D position to rotate in a first direction prior to the application of the first and second rotation matrices; and

in response to a second adjustment parameter, applying a fourth rotation matrix to the 3-D position to rotate in a second direction prior to the application of the first and second rotation matrices.

16. The method of claim 15, further comprising the steps of:

in response to an indication that an image acquisition device that captured the wide-angle image was oriented approximately horizontal, the third and forth rotation matrices aligning the 3-D position in accord with approximately true horizontal.

17. The method of claim 16, wherein the third rotation matrix performs a rotation about an optical axis of the image acquisition device.

18. The method of claim 17, wherein the fourth rotation matrix performs a rotation about an axis orthogonal to the optical axis of the image acquisition device.

19. The method of claim 15, further comprising the steps of:

in response to an indication that an image acquisition device that captured the wide-angle image was oriented approximately down-looking vertical, the first and second rotation matrices aligning the 3-D position in accord with approximately true down-looking vertical.

20. The method of claim 19, wherein the third rotation matrix performs a rotation about a first axis that is orthogonal to an optical axis of the image acquisition device.

21. The method of claim 20, wherein the forth rotation matrix performs a rotation about a second axis that is orthogonal to an optical axis of the image acquisition device, and that is also orthogonal to the first axis.

22. The method of claim 8, further comprising the step of:

specifying, by a user, the pan and tilt parameters by manipulating an input device.

23. The method of claim 22, wherein the input device is a joystick.

24. The method of claim 8, wherein the steps are implemented by a graphical image conversion component within a diagrammatic programming environment.

25. The method of claim 8, wherein the steps are implemented by functionality of a text-based programming environment.

26. The method of claim 8, wherein the steps are implemented by an embedded hardware target.

27. An apparatus for converting a region of a wide-angle image into a rectilinear image, the apparatus comprising:

a computer-implemented representation of a rectilinear system configured to convert a location of a pixel in the rectilinear image into a 3-D position relative to the rectilinear image;

a first rotation matrix configured to rotate the 3-D position in response to a pan parameter;

a second rotation matrix configured to rotate the 3-D position in response to a tilt parameter;

a computer-implemented representation of a wide-angle system configured to map the 3-D position to a location of one or more pixels in the wide-angle image and set attributes of the pixel in the rectilinear image using attributes of the one or more pixels in the wide angle image.

28. The apparatus of claim 27, wherein the wide-angle image is a Fish-Eye image.

29. The apparatus of claim 28, wherein the computer-implemented representation of a wide-angle system includes a model of a wide-angle system selected from the group consisting of an equidistant model, a stereographic model, an equi-solid angle model, and an orthogonal model.

30. The apparatus of claim 27, wherein the first rotation matrix is further configured to, in response to an indication that an image acquisition device that captured the wide-angle image was oriented approximately horizontal, perform a rotation about a first axis orthogonal to an optical axis of the image acquisition device.

31. The apparatus of claim 30, wherein the second rotation matrix is further configured to perform a rotation about a second axis that is orthogonal to the optical axis of the image acquisition device, and that is also orthogonal to the first axis.

32. The apparatus of claim 27, wherein the first rotation matrix is further configured to, in response to an indication that an image acquisition device that captured the wide-angle image was oriented approximately vertical down-looking. perform a rotation about an optical axis of an image acquisition device.

33. The apparatus of claim 32, wherein the second rotation matrix is further configured to perform a rotation about an axis orthogonal to the optical axis of the image acquisition device.

34. The apparatus of claim 27, further comprising:

a third rotation matrix configured to rotate the 3-D position in a first direction;

and

a forth rotation matrix configured to rotate the 3-D position in a second direction.

35. The apparatus of claim 34, wherein the third and forth rotation matrices are configured to aligning the 3-D position in accord with approximately true horizontal in response to an indication that an image acquisition device that captured the wide-angle image was oriented approximately horizontal.

36. The apparatus of claim 34, wherein the third and forth rotation matrices are configured to align the 3-D position in accord with approximately true down-looking vertical, in response to an indication that an image acquisition device that captured the wide-angle image was oriented approximately down-looking vertical.

37. The apparatus of claim 27 further comprising:

an input device configured to specify pan and tilt parameters in response to manipulation by a user.

38. The apparatus of claim 37, wherein the input device is a joystick.

39. The apparatus of claim 37, wherein the computer-implemented representation of a rectilinear system and the computer-implemented representation of a wide-angle system are implemented by a graphical image conversion component within a diagrammatic programming environment.

40. The apparatus of claim 37, wherein the computer-implemented representation of a rectilinear system and the computer-implemented representation of a wide-angle system are implemented by functionality of a text-based programming environment.

41. The apparatus of claim 37, wherein the computer-implemented representation of a rectilinear system and the computer-implemented representation of a wide-angle system are implemented by an embedded hardware target.

42. A computer readable medium containing executable program instructions for converting a region of a wide-angle image into a rectilinear image, the executable program instructions comprising program instructions configured to:

convert a location of a pixel in the rectilinear image into a 3-D position relative to the rectilinear image;

in response to a pan parameter, apply a first rotation matrix to the 3-D position;

in response to a tilt parameter, apply a second rotation matrix to the 3-D position;

map the 3-D position to a location of one or more pixels in the wide-angle image; and

set attributes of the pixel in the rectilinear image using attributes of the one or more pixels in the wide angle image.

43. The computer readable medium of claim 42, wherein the wide-angle image is a Fish-Eye image.

44. An apparatus for converting a wide-angle image into a rectilinear image, the apparatus comprising:

a wide angle camera mounted to a vehicle and configured to capture a wide angle image;

a display device internal to the vehicle configured to display a rectilinear image; and

a processor configured to map a location of each pixel in the rectilinear image to a location of one or more pixels in the wide-angle image, and set attributes of each pixel in the rectilinear image from the one or more pixels in the wide angle image.

45. The apparatus of claim 44 wherein the processor is further configured to convert the location of each pixel to a 3-D position relative to the rectilinear image, apply a first rotation matrix to rotate the 3-D position in response to a pan parameter, apply a second rotation matrix to rotate the 3-D position in response to a tilt parameter, and map the 3-D position to the one or more pixels in the wide-angle image.

46. The apparatus of claim 45 further comprising:

an input device internal to the vehicle configured to specify the pan and tilt parameters in response to manipulation by a user.