Image sensor apparatus and method for embedding recoverable data on image sensor pixel arrays

• US 9,521,292 B2
• Filed: 03/12/2008
• Issued: 12/13/2016
• Est. Priority Date: 03/12/2008
• Status: Active Grant

• Current Assignee: Omnivision Technologies Incorporated (Will Semiconductor Co., Ltd. Shanghai)
• Original Assignee: Omnivision Technologies Incorporated (Will Semiconductor Co., Ltd. Shanghai)
• Inventors: Emery, Robbert
• Primary Examiner(s): CUTLER, ALBERT H
• Application Number:
  US12/047,267
• Publication Number:
  US 20090231455A1
• Time in Patent Office: 3,198 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: H04N 1/32203 Spatial or amplitude domain methods H04N1/32309 takes precedence
• CPC Class Codes: H04N 1/32208 involving changing the magnitude of selected pixels, e.g. overlay of information or super-imposition H04N1/32219, H04N1/32224 take precedence
• CPC Class Codes: H04N 1/32229 with selective or adaptive application of the additional information, e.g. in selected regions of the image H04N1/32213 takes precedence
• CPC Class Codes: H04N 2201/3215 of a time or duration
• CPC Class Codes: H04N 2201/3226 of identification information or the like, e.g. ID code, index, title, part of an image, reduced-size image
• CPC Class Codes: H04N 2201/3252 Image capture parameters, e.g. resolution, illumination conditions, orientation of the image capture device H04N2201/3256 and subgroups take precedence
• CPC Class Codes: H04N 2201/3253 Position information, e.g. geographical position at time of capture, GPS data
• CPC Class Codes: H04N 2201/327 which are undetectable to the naked eye, e.g. embedded codes
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

An image sensor apparatus is disclosed. The image sensor apparatus includes an image sensor for generating image data in a pixel array corresponding to an optical image. A processor alters the image data to embed a feature-dependent code associated with a feature of the image data in a feature-dependent location in the pixel array and generate a digital image from the altered image data.

Claims

1. An image sensor apparatus, comprising:

an image sensor for generating image data in a pixel array corresponding to an optical image; and

a processor for;

altering the image data to embed a first feature-dependent code associated with a first feature of the image data and a second feature-dependent code associated with a second feature of the image data, and

generating a digital image from the altered image data,

wherein the first feature-dependent code is embedded at a first fixed location of the pixel array, the second feature-dependent code is embedded at a second fixed location of the pixel array, the first fixed location and second fixed location each corresponding to a different non-overlapping group of pixels within the pixel array, the first and second fixed locations being predetermined based upon a respective type of the first and second features and distinct from a respective location of the first and second features within the image data.

2. The image sensor apparatus of claim 1, wherein the image data comprises a CRC of the first and second feature-dependent codes embedded in a perimeter of the pixel array.

3. The image sensor apparatus of claim 1, wherein each of the first and second feature-dependent codes comprises an n-bit word, wherein n is at least two.

4. The image sensor apparatus of claim 3, wherein the processor alters the value of one or more pixels in the pixel array to represent the n-bit word.

5. The image sensor apparatus of claim 4, wherein the one or more pixels are spatially apart in quadrants of the pixel array.

6. The image sensor apparatus of claim 5, wherein each of the first and second feature-dependent codes is perceptually insignificant to an observer of the digital image.

7. The image sensor apparatus of claim 6, wherein each of the first and second feature-dependent codes comprises a user-selectable code.

8. The image sensor apparatus of claim 1, wherein at least one of the first and second features of the image data comprises a content-feature and a context-feature.

9. The image sensor apparatus of claim 1, the embedded first feature-dependent code and the embedded second feature-dependent code being imperceptible to a human observer of the optical image.

10. A method for embedding codes in a digital image, comprising:

acquiring image data in a pixel array of an image sensor device corresponding to an optical image;

identifying a first feature of interest associated with the image data and a second feature of interest associated with the image data; and

altering the image data in the pixel array to generate a digital image having a first feature-dependent code representing the first feature of interest and a second feature-dependent code representing the second feature of interest, wherein the first feature-dependent code is embedded at a first fixed location of the pixel array, the second feature dependent code is embedded at a second fixed location of the pixel array, the first and second fixed locations each corresponding to a different respective non-overlapping group of pixels within the pixel array, the first and second fixed locations being predetermined based upon a type of the first and second feature of interest and distinct from a location of each of the first and second features within the image data.

11. The method of claim 10, further comprising altering one or more pixels in a perimeter of the pixel array to embed a CRC of each of the first and second feature-dependent codes.

12. The method of claim 11, further comprising decoding the CRC to recover each of the first and second feature-dependent codes.

13. The method of claim 10, wherein altering the image data in the pixel array comprises altering one or more pixels in the pixel array to represent an n-bit word corresponding to each of the first and second feature-dependent codes, wherein n is at least two.

14. The method of claim 13, further comprising processing the digital image to recover each of the first and second feature-dependent codes at the first and second fixed locations.

15. The method of claim 10, the step of altering the image data comprising embedding first feature-dependent code and the second feature-dependent code so as to be imperceptible to a human observer of the optical image.

16. A processor for use in an image sensor device, comprising:

a feature module for identifying a first and second feature of interest associated with image data captured by the image sensor device in a pixel array;

a signature module for creating a first feature-dependent code associated with the first feature of interest and a second feature-dependent code associated with the second feature of interest; and

an embedding module for altering the image data to embed the first feature-dependent code in a first fixed location in the pixel array, and the second feature-dependent code in a second fixed location in the pixel array, the first and second fixed locations each corresponding to a different respective non-overlapping group of pixels, the first and second fixed locations being predetermined based upon a type of each of the first and second features of interest and distinct from a location of each of the first and second features within the image data.

17. The processor of claim 16, wherein the feature module comprises routines selected from a list comprising:

an edge detection routine;

a segmentation routine;

a scene identification routine;

an object detection routine;

a metadata identification routine; and

a Global Positioning System routine.

18. The processor of claim 17, wherein each of the first and second feature-dependent codes comprises an n-bit word to identify the respective features of interest, wherein n is at least two.

19. The processor of claim 18, further comprising a routine for altering one or more pixels in the perimeter of the image to represent a CRC of each of the first and second feature-dependent codes.

20. The processor of claim 16, the embedded first feature-dependent code and the embedded second feature-dependent code being imperceptible to a human observer of the image data when displayed as an image.

21. A system for identifying a feature of a digital image, comprising:

an imaging device with executable instructions to;

generate image data in a pixel array corresponding to an optical image; and

alter the image data in the pixel array to generate a digital image having a plurality of feature-dependent codes, each one of the plurality associated with a different feature of the image data, wherein each one of the plurality of feature-dependent codes are embedded at a respective fixed location of the pixel array, each fixed location corresponding to a different non-overlapping group of pixels, the fixed locations being predetermined based upon a type of the feature and distinct from a location of the feature within the image data; and

a processing device with executable instructions to;

receive the digital image from the imaging device; and

identify the feature of the digital image based on the plurality of feature-dependent codes embedded at different fixed locations in the pixel array.

22. The system of claim 21, wherein the processing device comprises a detection module for extracting the plurality of feature-dependent codes embedded at the different fixed locations in the pixel array.

23. The system of claim 22, wherein the imaging device further comprises executable instructions to embed a CRC of the plurality of feature-dependent codes in the perimeter of the pixel array.

24. The system of claim 23, wherein the processing device further comprises executable instructions to decode the CRC to recover the plurality of feature-dependent codes.

25. The system of claim 21, the embedded plurality of feature-dependent codes being imperceptible to a human observer of the digital image.

26. A method for identifying a feature in a digital image generated in a pixel array, comprising:

receiving a digital image having a plurality of feature-dependent codes embedded in fixed locations in the pixel array, each fixed location corresponding to a different non-overlapping group of pixels, wherein the plurality of feature-dependent codes are associated with the feature, the fixed location being predetermined based upon a type of the feature and distinct from a location of the feature within the image data;

processing the digital image to recover one or more pixels in each feature-dependent location; and

comparing the one or more pixels to each feature-dependent code to identify the feature.

27. The method of claim 26, further comprising decoding a CRC code embedded in the digital image to verify whether the plurality of feature-dependent codes is present in the image.

28. The method of claim 27, further comprising displaying the digital image.

29. The method of claim 26, the plurality of embedded feature-dependent codes being imperceptible to a human observer of the digital image.

30. An image sensor apparatus, comprising:

an image sensor for generating image data in a pixel array corresponding to an optical image; and

a processor,

wherein the processor generates altered image data by embedding at least a first plurality of feature-dependent codes and a second plurality of feature-dependent codes in the pixel array, the first plurality of feature-dependent codes associated with a first feature of the image data and embedded in a first fixed feature-dependent location of the pixel array, the second plurality of feature-dependent codes associated with a second feature of the image data and embedded in a second fixed feature-dependent location of the pixel array,

wherein the processor generates a digital image from the altered image data, and

wherein the first feature of the image data is different from the second feature of the image data, and wherein the first fixed feature-dependent location corresponds to a different non-overlapping pixel group from the second fixed feature-dependent location, the first and second fixed feature-dependent locations being predetermined based upon a type of the first and second features and distinct from locations of the first and second features within the image data.

31. The image sensor apparatus of claim 30, the embedded first plurality of feature-dependent codes and the embedded second plurality of feature-dependent codes being imperceptible to a human observer of the digital image.