Apparatus and method for rotating the display orientation of a captured image

• US RE38,896 E1
• Filed: 01/04/2002
• Issued: 11/29/2005
• Est. Priority Date: 01/19/1996
• Status: Expired due to Term

• Current Assignee: Apple Inc.
• Original Assignee: Apple Computer Incorporated (Apple Inc.)
• Inventors: Anderson, Eric C.
• Primary Examiner(s): Garber, Wendy R.
• Assistant Examiner(s): WHIPKEY, JASON T
• Application Number:
  US10/040,249
• Time in Patent Office: 1,425 Days
• Field of Search: 382/296, 382/297, 348/208.99, 348/208.1, 348/208.2, 348/208.3, 348/208.4, 348/208.6, 348/208.12, 348/246, 348/272, 348/222.1, 345/656
• US Class Current: 348/272
• CPC Class Codes: H04N 2101/00 Still video cameras
• CPC Class Codes: H04N 2201/0084 Digital still camera
• CPC Class Codes: H04N 5/232 Devices for controlling television cameras, e.g. remote control ; Control of cameras comprising an electronic image sensor H04N5/235 takes precedence ; Mountings, adjusting means, or light-tight connections, for optical elements G02B7/00
• CPC Class Codes: H04N 5/23293 Electronic viewfinders
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Abstract

The apparatus of the present invention preferably comprises an image sensor, an orientation sensor, a memory and a processing unit. The image sensor is used for generating captured image data. The orientation sensor is coupled to the image sensor, and is used for generating signals relating to the position of the image sensor. The memory, has an auto-rotate unit comprising program instructions for transforming the captured image data into rotated image data in response to the orientation sensor signals. The processing unit, executes program instructions stored in the memory, and is coupled to the image sensor, the orientation sensor and the memory. The method of the present invention preferably comprises the steps of: generating image data representative of an object with an image sensor; identifying an orientation of the image sensor relative to the object during the generating step; and selectively transferring the image data to an image processing unit in response to the identifying step.

Claims

1. An apparatus for rotating a display orientation of captured image data representative of an object, the apparatus comprising:

an image sensor, for generating said captured image data;

an orientation sensor coupled to said image sensor, for generating a signal corresponding to the position of the image sensor relative to said object;

a memory, having an auto-rotate unit comprising program instructions for selectively transforming said captured image data into rotated image data in response to said position signal, said memory coupled to said image sensor and to said orientation sensor; and

an image processing unit coupled to said memory for executing program instructions stored in said memory;

wherein (a) said image processing unit processes an i-by-j array of said captured image data and said image sensor generates an i+1-by-j+1 array of said image data, or (b) an image capture unit generates an additional row and column of pixels for said captured image data from said image sensor; and

an image capture unit generates an additional row and column of pixels for said captured image data from said image sensor.

2. The apparatus of claim 1, wherein the memory further comprises:

an image processing unit comprising program instructions for transforming one from a group consisting of captured image data and portrait image data, into processed image data.

3. The apparatus of claim 1, wherein:

the signal is a portrait_left signal if the image sensor is rotated clockwise from a landscape orientation relative to the object, and the signal is a portrait_right signal if the image sensor is rotated counter-clockwise from the landscape orientation relative to the object; and

the auto-rotate unit comprises program instructions for transforming the captured image data into portrait_left image data in response to the portrait_left signal and into portrait_right image data in response to the portrait_right signal.

4. The apparatus of claim 1, wherein:

the signal is a landscape signal if the image sensor is positioned in a level orientation relative to the object; and

the auto-rotate unit comprises program instructions for transforming the captured image data into landscape image data in response to the landscape signal.

5. The apparatus of claim 3, wherein:

the image sensor has a top, a bottom, a right side and a left side;

the auto-rotate unit program instructions transform the captured image data into the portrait_left image data by transferring a prior portrait_left line of image data which starts further toward the bottom of the image sensor and ends further toward the top of the image sensor, then transferring a subsequent portrait_left line of image data, located closer to the right side of the image sensor than the prior portrait_left line of image data, and also starting further toward the bottom of the image sensor and ending further toward the top of the image sensor; and

the auto-rotate unit program instructions transform the captured image data into the portrait_right image data by transferring a prior portrait_right line of image data which starts further toward the top of the image sensor and ends further toward the bottom of the image sensor, then transferring a subsequent portrait_right line of image data, located closer to the left side of the image sensor than the prior portrait_right line of image data, and also starting further toward the top of the image sensor and ending further toward the bottom of the image sensor.

6. The apparatus of claim 4, wherein:

the image sensor has a top, a bottom, a right side and a left side; and

the auto-rotate unit program instructions transform the captured image data into the landscape image data by transferring a prior landscape line of image data which starts further toward the left side of the image sensor and ends further toward the right side of the image sensor, then transferring a subsequent landscape line of image data, located closer to the bottom of the image sensor than the prior landscape line of image data, and also starting further toward the left side of the image sensor and ending further toward the right side of the image sensor.

7. The apparatus of claim 3, wherein:

the portrait_left signal is generated by the orientation sensor if the image sensor is rotated approximately 45°

clockwise from the level orientation, and the portrait_right signal is generated by the orientation sensor if the imaging subsystem is rotated approximately 45°

counter-clockwise from the level orientation.

8. The apparatus of claim 5, wherein:

the prior portrait_left line of image data and the prior portrait_right line of image data comprise a “

green, red, green, red”

pixel pattern; and

the subsequent portrait_left line of image data and the subsequent portrait_right line of image data comprise a “

blue, green, blue, green”

pixel pattern.

9. An apparatus for rotating a display orientation of multicolor captured image data having an i-by-j pixel matrix with a pattern representative of an object, comprising:

an image sensor, for generating the multicolor captured image data;

an input device, for generating a portrait_left signal in response to a first user selection, a portrait_right signal in response to a second user selection, and a landscape signal in response to a third user selection;

a memory, having;

an auto-rotate unit comprising program instructions for selectively transforming the multicolor captured image data into portrait_left image data in response to the portrait_left signal, portrait_right image data in response to the protrait_right signal, and landscape image data in response to the landscape signal; and

an image processing unit comprising program instructions for transforming the portrait_left image data, the portrait_right image data and the landscape image data into processed image data; and

a processing unit, coupled to the image sensor, to the input device, and to the memory, for executing program instructions stored in the memory;

wherein said image processing unit changes the number of pixel rows and pixel columns of the multicolor captured image data such that, from a defined referenced viewpoint, the portrait left image data, the portrait right image data, and the landscape image data, each includes the an (i−

1)-by-(j−

1) pixel matrix having said pattern.

10. The apparatus of claim 9, wherein the image processing unit has a first line length for processing the portrait_left image data and the portrait_right image data and a second line length for processing the landscape image data.

11. A method for rotating a display orientation of image data representative of an object, comprising the steps of:

generating image data with an image sensor;

identifying an orientation of the image sensor relative to the object at a time substantially simultaneous with the generating step, where said identifying is performed by an orientation sensor; and

selectively transferring data to an image processing unit in response to the identifying step;

wherein said image processing unit rotates said display orientation of said image data and (a) said image processing unit processes an i-by-j array of said captured image data and said image sensor generates an i+1-by-j+1 array of said image data, or (b) an image capture unit generates an additional row and column of pixels for said captured image data from said image sensor

wherein said image processing unit rotates said orientation of said image data by generating an additional row and column of pixels for said captured image data from said image sensor.

12. The method of claim 11, further comprising the step of:

generating an additional row and column of image data.

13. The method of claim 11, further comprising the step of correcting defects within the image data caused by defects within the image sensor.

14. The method of claim 11, wherein the image sensor comprises a top, a right side and a left side, wherein the image comprises a “

top portion,” and

wherein the step of identifying an orientation further comprises the steps of;

identifying a portrait_left orientation, if the left side of the image sensor corresponds to the “

top portion”

of the object;

identifying a portrait_right orientation, if the right side of the image sensor corresponds to the “

top portion”

of the object; and

identifying a landscape orientation, if the top of the image sensor corresponds to the “

top portion”

of the object.

15. The method of claim 11, wherein the step of identifying an orientation further comprises the steps of:

identifying a portrait_left orientation, in response to a user selection of the portrait_left orientation on an input device;

identifying a portrait_right orientation, in response to a user selection of the protrait_right orientation on the input device; and

identifying a landscape orientation, in response to a user selection of the landscape orientation on the input device.

16. The method of claim 11, wherein the orientation is a portrait_left orientation, wherein the image data is comprised of an array of pixel colors ordered in rows and columns, and wherein the step of selectively transferring comprises the steps of:

initializing a column variable to a first column of pixel colors required by the image processing unit;

initializing a row variable to a row containing a first pixel color required by the image processing unit;

transferring pixel color at an array location defined by the row variable and the column variable to and the image processing unit;

decrementing the row variable to a row containing a next pixel color required by the image processing unit;

returning to the transferring step, if a row containing a last pixel color has not been transferred;

incrementing the column variable to a next column of pixel colors required by the image processing unit; and

returning to the initializing a row variable step, if a last column of pixel colors has not been transferred.

17. The method of claim 16, wherein the image data is replaced by defective image sensor information, further comprising the step of repeating the steps of claim 16.

18. The method of claim 16, further comprises the steps of:

configuring the image processing unit to accept an image data line length corresponding to the portrait_left orientation; and

performing image processing on a line of transferred image data.

19. The method of claim 11, wherein the orientation is a portrait_right orientation, wherein the image data is comprised of an array of pixel colors ordered in rows and columns, and wherein the step of selectively transferring comprises the steps of:

initializing a column variable to a first column of pixel colors required by the image processing unit;

initializing a row variable to a row containing a first pixel color required by the image processing unit;

transferring pixel color at an array location defined by the row variable and the column variable, to the image processing unit;

incrementing the row variable to a row containing a next pixel color required by the image processing unit;

returning to the transferring step, if a row containing a last pixel color has not been transferred;

decrementing the column variable to a next column of pixel colors required by the image processing unit; and

returning to the initializing a row variable step, if a last column of pixel colors has not been transferred.

20. The method of claim 11, wherein the orientation is a landscape orientation, wherein the image data is comprised of an array of pixel colors ordered in rows and columns, and wherein the step of selectively transferring further comprises the steps of:

initializing a row variable to a first row of pixel colors required by the image processing unit;

initializing a column variable to a column containing a first pixel color required by the image processing unit;

transferring pixel color at an array location defined by the row variable and the column variable, to the image processing unit;

incrementing the column variable to a column containing a next pixel color required by the image processing unit;

returning to the transferring step, if a column containing a last color pixel has not been transferred;

incrementing the row variable to a next row of pixel colors required by the image processing unit; and

returning to the initializing a column variable step, if a last row of pixel colors has not been transferred.

21. An apparatus for rotating a display orientation of multicolor image data having an i-by-j pixel matrix with a pattern representative of an object, comprising:

means for generating multicolor image data with an image sensor, the image data having a Bayer pattern;

orientation sensor means for identifying an orientation of said image sensor relative to said object at a time substantially simultaneous with said generating said multicolor image data; and

means for selectively transferring said multicolor image data to an image processing unit in response to said means for identifying;

wherein said image processing unit rotates said display orientation of said multicolor image data for providing rotated multicolor image data, and changes the number of pixel rows and pixel columns of said multicolor image data such that, from a defined referenced viewpoint, said rotated multicolor image data includes an (i−

1)-by-(j−

1) pixel matrix having said pattern.

22. The apparatus of claim 21, further comprising means for generating an additional row and column of image data.

23. The apparatus of claim 21, further comprising means for correcting defects within the image data caused by defects within the image sensor.

24. The apparatus of claim 21, wherein the image sensor comprises a top, a right side and a left side, wherein the image comprises a “

top portion,” and

wherein the means for identifying an orientation further comprises;

means for identifying a portrait_left orientation, if the left side of the image sensor corresponds to the “

top portion”

of the object;

means for identifying a portrait_right orientation, if the right side of the image sensor corresponds to the “

to portion”

of the object; and

means for identifying a landscape orientation, if the top of the image sensor corresponds to the “

top portion”

of the object.

25. The apparatus of claim 21, wherein the orientation is a portrait_left orientation, wherein the image data is comprised of an array of pixel colors ordered in rows and columns, and wherein the means for selectively transferring comprises:

means for initializing a column variable to a first column of pixel colors required by the image processing unit;

means for initializing a row variable to a row containing a first pixel color required by the image processing unit;

means for transferring pixel color at an array location, defined by the row variable and the column variable, to the image processing unit;

means for decrementing the row variable to a row containing a next pixel color required by the image processing unit;

means for returning to the means for transferring, if a row containing a last pixel color has not been transferred;

means for incrementing the column variable to a next column of pixel colors required by the image processing unit; and

means for returning the means for initializing a row variable, if a last column of pixel colors has not been transferred.

26. The apparatus of claim 21, wherein the orientation is a portrait_right orientation, wherein the image data is comprised of an array of pixel colors ordered in rows and columns, and wherein the means for selectively transferring comprises:

means for initializing a column variable to a first column of pixel colors required by the image processing unit;

means for initializing a row variable to a row containing a first pixel color required by the image processing unit;

means for transferring pixel color at an array location, defined by the row variable and the column variable, to the image processing unit;

means for incrementing the row variable to a row containing a next pixel color required by the image processing unit;

means for returning to the means for transferring, if a row containing a last pixel color has not been transferred;

means for decrementing the column variable to a next column of pixel colors required by the image processing unit; and

means for returning to the means for initializing a row variable, if a last column of pixel colors has not been transferred.

27. The apparatus of claim 21, wherein the orientation is a landscape orientation, wherein the image data is comprised of an array of pixel colors ordered in rows and columns, and wherein the means for selectively transferring comprises:

means for initializing a row variable to a first row of pixel colors required by the image processing unit;

means for initializing a column variable to a column containing a first pixel color required by the image processing unit;

means for transferring pixel color at an array location, defined by the row variable and the column variable, to the image processing unit;

means for incrementing the column variable to a column containing a next pixel color required by the image processing unit;

means for returning to the means for transferring, if a column containing a last color pixel has not been transferred;

means for incrementing the row variable to a next row of pixel colors required by the image processing unit; and

means for returning to the means for initializing a column variable, if a last row of pixel colors has not been transferred.

28. A computer useable medium embodying computer readable program code for causing a computer to rotate a display orientation of multicolor image data having an i-by-j pixel matrix with a pattern representative of an object, by performing steps comprising:

generating said multicolor image data with an image sensor, the image data having a Bayer pattern;

identifying an orientation of the image sensor relative to the object at a time substantially simultaneous with the generating step, wherein said identifying of said orientation is performed with an orientation sensor; and

selectively transferring image data to an image processing unit in response to the identifying step,

wherein said image processing unit rotates said display orientation of said multicolor image data for providing rotated multicolor image data, and changes the number of pixel rows and pixel columns of said multicolor image data such that, from a defined referenced viewpoint, said rotated multicolor image data includes having an (i−

1)-by-(j−

1) pixel matrix having said pattern.

29. The computer useable medium of claim 28, further comprising program code for generating an additional row and column of image data.

30. The computer useable medium of claim 28, further comprising program code for correcting defects within the image data caused by defects within the image sensor.

31. The computer useable medium of claim 28, wherein the image sensor comprises a top, a right side and a left side, wherein the image comprises a “

top portion,” and

wherein the program code for performing the step of identifying an orientation further comprises program code for;

identifying a portrait_left orientation, if the left side of the image sensor corresponds to the “

top portion”

of the object;

identifying a portrait_right orientation, if the right side of the image sensor corresponds to the “

top portion”

of the object; and

identifying a landscape orientation, if the top of the image sensor corresponds to the “

top portion”

of the object.

32. The computer useable medium of claim 28, wherein the orientation is a portrait_left orientation, wherein the image data is comprised of an array of pixel colors ordered in rows and columns, and wherein the program code for performing the step of selectively transferring comprises program code for:

initializing a column variable to a first column of pixel colors required by the image processing unit;

initializing a row variable to a row containing a first pixel color required by the image processing unit;

transferring pixel color at an array location, defined by the row variable and the column variable, to the image processing unit;

decrementing the row variable to a row containing a next pixel color required by the image processing unit;

returning to the transferring step, if a row containing a last pixel color has not been transferred;

incrementing the column variable to a next column of pixel colors required by the image processing unit; and

returning to the initializing a row variable step, if a last column of pixel colors has not been transferred.

33. The computer useable medium of claim 28, wherein the orientation is a protrait_right orientation, wherein the image data is comprised of an array of pixel colors ordered in rows and columns, and wherein the program code for performing the step of selectively transferring comprises program code for:

initializing a column variable to a first column of pixel colors required by the image processing unit;

initializing a row variable to a row containing a first pixel color required by the image processing unit;

transferring pixel color at an array location, defined by the row variable and the column variable, to the image processing unit;

incrementing the row variable to a row containing a next pixel color required by the image processing unit;

returning to the transferring step, if a row containing a last pixel color has not been transferred;

decrementing the column variable to a next column of pixel colors required by the image processing unit; and

returning to the initializing a row variable step, if a last column of pixel colors has not been transferred.

34. The computer useable medium of claim 28, wherein the orientation is a landscape orientation, wherein the image data is comprised of an array of pixel colors ordered in rows and columns, and wherein the program code for performing the step of selectively transferring comprises program code for:

initializing a row variable to a first row of pixel colors required by the image processing unit;

initializing a column variable to a column containing a first pixel color required by the image processing unit;

transferring pixel color at an array location, defined by the row variable and the column variable, to the image processing unit;

incrementing the column variable to a column containing a next pixel color required by the image processing unit;

returning to the transferring step, if a column containing a last color pixel has not been transferred;

incrementing the row variable to a next row of pixel colors required by the image processing unit; and

returning to the initializing a column variable step, if a last row of pixel colors has not been transferred.

35. An apparatus for rotating a display orientation of multicolor captured image data having an i-by-j pixel matrix with a pattern representative of an object, comprising:

an image sensor, for generating said multicolor captured image data, the image data having a Bayer pattern;

an orientation sensor coupled to said image sensor, for generating a signal corresponding to the position of said image sensor relative to said object; and

a hardware device, having an auto-rotate unit comprising circuits for selectively transforming said multicolor captured image data into rotated multicolor image data in response to said position signal, said hardware device coupled to said image sensor and to said orientation sensor;

wherein, from a defined referenced viewpoint, said rotated multicolor image data includes an (i−

1)-by-(j−

1) pixel matrix having said pattern.

36. An apparatus for rotating a display orientation of captured image data representative of an object, the apparatus comprising:

an image sensor, for generating said captured image data;

an input device, for generating an orientation signal in response to a user selection;

a memory, having an auto-rotate unit for selectively transforming said captured image data into rotated image data in response to said orientation signal from said input device; and

an image processing unit coupled to said memory for processing the image data by generating at least one additional row and column of pixels for said captured image data from said image sensor.

37. A digital image capture device, comprising:

an image sensor, for capturing image data;

an orientation sensor, for generating an orientation signal indicating whether the image sensor is in a portrait or landscape position; and

an auto-rotate unit coupled to the image sensor and the orientation sensor, for automatically rotating a subset of the image data in response to the orientation signal.

38. The digital image capture device of claim 37, further comprising:

an image capture unit coupled to the image sensor, for adding m additional rows and n additional columns to an i-by-j array of image data to form an i+m-by-j+n array of image data to be rotated by the auto-rotate unit in response to the orientation signal.

39. A method of rotating image data in a digital image capture device, comprising:

capturing image data from an image sensor;

providing an orientation signal indicating whether the image sensor is in a portrait or landscape position; and

automatically rotating a subset of the captured image data in response to the orientation signal.

40. The method of claim 39, further comprising:

adding m additional rows and n additional columns to an i-by-j array of the image data to form an i+m-by-j+n array of image data.

41. A computer-readable medium having stored thereon instructions which, when executed by a processor, cause the processor to perform the steps of:

capturing image data from an image sensor;

providing an orientation signal indicating whether the image sensor is in a portrait or landscape orientation; and

automatically rotating a subset of the captured image data in response to the orientation signal.

42. The computer-readable medium of claim 41, further comprising:

adding m additional rows and n additional columns to an i-by-j array of the image data to form an i+m-by-j+n array of image data.

43. A digital image capture device, comprising:

image sensor means for generating image data;

means for generating an orientation signal indicating either a portrait orientation or a landscape orientation of the image sensor; and

means for automatically rotating a subset of the image data in response to the orientation signal.

44. A digital image capture device, comprising:

an image sensor, for generating said captured image data including a plurality of rows and columns of pixels;

an orientation sensor coupled to said image sensor, for generating a position signal indicating whether the image sensor is in a portrait or landscape position;

a memory, having an auto-rotate unit comprising program instructions for selectively transforming said captured image data into rotated image data in response to said position signal by processing at least one row of pixels and at least one column of pixels less than the plurality of rows and columns of pixels in the captured image data, said memory coupled to said image sensor and to said orientation sensor; and

an image processing unit coupled to said memory for executing the stored program instructions to rotate said capture image data.

45. A method for rotating a display orientation of image data, comprising:

generating image data with an image sensor including a plurality of rows and columns of pixels;

determining with an orientation sensor a portrait orientation or a landscape orientation for the image data substantially simultaneously with generating the image data; and

processing the image data with an image processing unit in response to the orientation signal to rotate the image data, by rotating less than all of the plurality of rows and columns of pixels of the image data.