Imaging systems and methods using square image sensor for flexible image orientation

• US 9,531,970 B2
• Filed: 04/25/2014
• Issued: 12/27/2016
• Est. Priority Date: 04/26/2013
• Status: Active Grant

• Current Assignee: Omnivision Technologies Incorporated (Will Semiconductor Co., Ltd. Shanghai)
• Original Assignee: Omnivision Technologies Incorporated (Will Semiconductor Co., Ltd. Shanghai)
• Inventors: Haji-Khamneh, Bahman, Iyer, Harish, Margasahayam, Vinoo
• Primary Examiner(s): NGUYEN, LUONG TRUNG
• Application Number:
  US14/262,180
• Publication Number:
  US 20140320715A1
• Time in Patent Office: 977 Days
• US Class Current: 1/1
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

An imaging system for generating flexibly oriented electronic images includes an image sensor having a square pixel array, imaging optics for forming an optical image on at least a portion of the square pixel array, wherein the portion is within an image circle of the imaging optics and includes at least two rectangular sub-portions differing from each other in aspect ratio and/or orientation, and a processing module capable of generating an electronic image from each of the at least two rectangular sub-portions. An imaging method for generating electronic images of flexible orientation, using a square image sensor pixel array, includes forming an optical image on at least a portion of the square image sensor pixel array, selecting, according to a desired orientation, a rectangular sub-portion of the at least a portion of the square image sensor pixel array, and generating a final electronic image from the sub-portion.

Claims

1. An imaging system for generating flexibly oriented electronic images, comprising:

a mobile phone including;

(a) an image sensor having a square pixel array;

(b) imaging optics for forming an optical image on at least a portion of the square pixel array, the at least a portion being within an image circle of the imaging optics and the at least a portion including a plurality of rectangular sub-portions of mutually different orientations including at least one orientation that is not squarely aligned with the square pixel array; and

(c) a processing module capable of generating an electronic image from any of the plurality of rectangular sub-portions, wherein the processing module is configured to, for each pixel of the electronic image and when a rectangular sub-portion is not squarely aligned with the square pixel array, populate the each pixel of the electronic image with values associated with signals from at least one pixel of the square pixel array located nearest a location corresponding to the each pixel of the electronic image.

2. The imaging system of claim 1, the rectangular sub-portions including a first sub-portion having aspect ratio a:

b, where a is different from b, and a second sub-portion having size and aspect ratio identical to the first sub-portion and orientation orthogonal to the first sub-portion.

3. The imaging system of claim 2, the square pixel array comprising N×

N pixels, the first and second sub-portions having sides parallel with sides of the square pixel array, the first sub-portion including columns of N pixels parallel to a first side of the square pixel array, and the second sub-portion including rows of N pixels, the rows being orthogonal to the columns.

4. The imaging system of claim 2, the rectangular sub-portions further including a third sub-portion having aspect ratio c:

d, where c is different from d and c/d is different from a/b, and a fourth sub-portion having size and aspect ratio identical to and orientation orthogonal to the third sub-portion.

5. The imaging system of claim 4, the first, second, third, and fourth sub-portions having sides parallel with sides of the square pixel array, the first and third sub-portions including columns of N pixels parallel to a first side of the square pixel array, and the second and fourth sub-portions including rows of N pixels, the rows being orthogonal to the columns.

6. The imaging system of claim 2, the rectangular sub-portions further including a third sub-portion having aspect ratio 1:

1, the third sub-portion including all pixels of the square pixel array.

7. The imaging system of claim 1, the mobile phone further comprising a gravitational orientation sensor for detecting gravitational orientation of the imaging system relative to gravity, and wherein the processing module is capable of selecting any one of the plurality of rectangular sub-portions, according to the gravitational orientation, such that the electronic image reflects a desired orientation relative to gravity.

8. The imaging system of claim 7, the plurality of rectangular sub-portions including a discrete set of rectangular sub-portions evenly distributed throughout range from 0 to 360 degrees and including at least one rectangular sub-portion not squarely aligned with the square pixel array.

21. The system of claim 1, the processing module further being capable of, for each one of the mutually different orientations, generating the electronic image with a plurality of mutually different aspect ratios.

9. An imaging method for generating electronic images of flexible orientation using a square image sensor pixel array, comprising:

onboard a mobile phone, forming an optical image on at least a portion of the square image sensor pixel array;

selecting, according to a desired orientation and from a set of mutually different orientations, a rectangular sub-portion of the at least a portion of the square image sensor pixel array, the set of mutually different orientations including at least one orientation that is not squarely aligned with the square image sensor pixel array; and

generating a final electronic image from the rectangular sub-portion, said generating including for each pixel of the final electronic image, when the rectangular sub-portion is not squarely aligned with the square pixel array, populating the pixel of the final electronic image with values associated with signals from at least one pixel of the square image sensor pixel array located nearest a location corresponding to the pixel of the final electronic image.

10. The method of claim 9, said selecting comprising selecting a rectangular sub-portion of the at least a portion of the square image sensor pixel array, the rectangular sub-portion having sides parallel with sides of the square image sensor pixel array.

11. The method of claim 10, said selecting a rectangular sub-portion comprising selecting a rectangular sub-portion from a set of a landscape-oriented sub-portion and a portrait-oriented sub-portion.

12. The method of claim 9, said selecting comprising selecting the rectangular sub-portion, of the at least a portion of the square image sensor pixel array, according to the desired orientation.

13. The method of claim 9, further comprising receiving the desired orientation.

14. The method of claim 13, said selecting being performed by a processing module using the desired orientation received in said receiving.

15. The method of claim 9, further comprising determining gravitational orientation of the square image sensor pixel array using a gravitational orientation sensor implemented onboard the mobile phone, the step of selecting comprising selecting, according to a desired orientation relative to the gravitational orientation, the rectangular sub-portion of the at least a portion of the square image sensor pixel array, the rectangular sub-portion being non-square with the square image sensor pixel array.

16. The method of claim 15, the electronic images being a video stream, the method comprising repeating said determining, forming, selecting, and generating for each of the electronic images in the video stream to produce an orientation-stabilized video stream.

17. The method of claim 9, further comprising, prior to said generating, capturing a square electronic image, said generating comprising cropping the square electronic image in accordance with the rectangular sub-portion selected in said selecting.

18. The method of claim 9, said generating comprising capturing a rectangular image using the rectangular sub-portion selected in said selecting.

19. The method of claim 9, said selecting further comprising selecting the rectangular sub-portion according to a desired one of a plurality of mutually different aspect ratios.

20. The method of claim 19, said selecting comprising selecting the rectangular sub-portion according to any combination of (a) the desired one of a plurality of mutually different aspect ratios and (b) the desired orientation.