Image transformation and multi-view output systems and methods

• US 9,196,022 B2
• Filed: 03/10/2014
• Issued: 11/24/2015
• Est. Priority Date: 03/10/2014
• Status: Active Grant

• Current Assignee: Omnivision Technologies Incorporated (Will Semiconductor Co., Ltd. Shanghai)
• Original Assignee: Omnivision Technologies Incorporated (Will Semiconductor Co., Ltd. Shanghai)
• Inventors: Li, Jin, Shan, Jizhang
• Primary Examiner(s): Do, Anh
• Application Number:
  US14/203,016
• Publication Number:
  US 20150254818A1
• Time in Patent Office: 624 Days
• Field of Search: 382/275, 382/276, 382/154, 382/293, 382/295, 382/296, 239/1, 239/11, 239/155, 348/42, 348/84, 348/E7.002, 356/124, 345/581, 345/586, 345/419, 345/427, 345/590, 345/611, 345/640, 345/648
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20172 Image enhancement details
• CPC Class Codes: G06T 3/02 Affine transformations for image registration G06T3/147; for image mosaicing G06T3/4038
• CPC Class Codes: G06T 5/80 Geometric correction

Abstract

An image transformation and multi-view output system and associated method generates output view data from raw image data using a coordinate mapping that reverse maps pixels of the output view data onto the raw image data. The coordinate mapping is stored in a lookup table and incorporates perspective correction and/or distortion correction for a wide angle lens used to capture the raw image data. The use of the lookup table with reverse mapping improves performance of the image transformation and multi-view output system to allow multi-view video streaming of images corrected for one or both of perspective and distortion.

Claims

1. A method for image transformation and multi-view output to a display device, comprising:

receiving raw image data, generated by an image sensor, into a memory of a system that includes a lookup table stored in non-volatile memory and a reverse mapper;

mapping, using the lookup table and a coordinate mapping module of the reverse mapper, each pixel coordinate of output view data, stored in the memory, to a location in raw image data using coordinate mapping that corrects at least one of (a) perspective and (b) distortion within the raw image;

determining, using an intensity setting module of the reverse mapper, an intensity of each output pixel of the output view data based upon information within the raw image data proximate the correspondingly mapped location; and

outputting the output view data to the display device.

2. The method of claim 1, further comprising selecting the coordinate mapping from a plurality of coordinate mappings stored within the lookup table.

3. The method of claim 1, wherein the step of mapping generates at least two sub-views within the output view data.

4. The method of claim 1, wherein the location within the raw image data identifies a pixel in the raw image data.

5. The method of claim 1, wherein the location within the raw image data identifies a group of pixels in the raw image data.

6. The method of claim 1, wherein the step of mapping corrects distortion along an axis of the image plane.

7. The method of claim 1, wherein the step of mapping corrects distortion along two orthogonal axes of the image plane.

8. The method of claim 1, wherein the step of mapping comprises image transformations selected from the group consisting of scaling and rotation.

9. The method of claim 1, the step of determining comprising an interpolation algorithm.

10. An image transformation and multi-view output system, comprising:

a lookup table, stored in a non-volatile memory and containing coordinate mapping that maps pixels of output view data to locations within raw image data; and

a reverse mapper for generating the output view data from the raw image data using the coordinate mapping;

wherein the coordinate mapping comprises at least one of (a) perspective correction and (b) distortion correction.

11. The system of claim 10, the reverse mapper comprising an intensity setting module for interpolating between at least two input pixels of the raw image data to generate each output pixel intensity value of the output view data.

12. The system of claim 10, wherein the reverse mapper corrects distortion along an axis of the image plane.

13. The system of claim 10, wherein the reverse mapper corrects distortion along two orthogonal axes of the image plane.

14. The system of claim 10, wherein the output view data comprises two independent sub-views of the raw image data.

15. The system of claim 10, further comprising:

at least one additional coordinate mapping stored within the lookup table; and

a multi-view selection module for (a) receiving an input indicative of a selected output view, and (b) selecting one of the coordinate mapping and the additional coordinate mapping for use by the reverse mapper to generate the output view data.

16. The system of claim 10, wherein the coordinate mapping implements one or more of scaling and rotation.

17. The system of claim 10, wherein the lookup table and the reverse mapper are configured as a single integrated circuit.

18. The system of claim 10, wherein the output view data is sent to a display device for display to a user.

19. The system of claim 10, wherein the raw image data is a frame of an input video sequence data and wherein the output view data is displayed as a frame in a live video stream.

20. A method for generating a coordinate mapping for use in a lookup table used by an image transformation and multi-view output system, comprising:

receiving parameters of an imaging lens, the parameters including both a distortion curve and a distortion center of the imaging lens;

receiving parameters of raw image data;

receiving at least one perspective angle corresponding to an output view of the image transformation and multi-view output system;

applying inverse perspective correction to an output view data based upon the at least one perspective angle to generate an intermediate mappines;

applying inverse distortion correction to the intermediate mapping data based upon at least one of the distortion curve and the distortion center to generate the coordinate mapping; and

outputting the coordinate mapping for use within the image transformation and multi-view output system.

21. The method of claim 20 wherein the parameters include the field of view of the imaging lens.

22. The method of claim 20 wherein raw image data parameters includes pixel positions.

23. The method of claim 20, wherein the coordinate mapping comprises a scaling operation between the raw image data and the output view data.

24. The method of claim 20, wherein the coordinate mapping maps each pixel of the output view data to a position in the raw image data.