Method And System For Inserting Software Processing In A Hardware Image Sensor Pipeline

• US 20080292132A1
• Filed: 11/15/2007
• Published: 11/27/2008
• Est. Priority Date: 05/24/2007
• Status: Active Grant

• Current Assignee: Clive Walker, David Plowman, Gary Keall
• Original Assignee: Broadcom Corporation (Broadcom, Inc.)
• Inventors: Keall, Gary, Plowman, David, Walker, Clive
• Granted Patent: US 9,058,668 B2
• Time in Patent Office: Days
• US Class Current: 382/100
• CPC Class Codes: G06F 15/7807 System on chip, i.e. computer system on a single chip; System in package, i.e. computer system on one or more chips in a single package
• CPC Class Codes: G06F 9/3867 using instruction pipelines
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: H04N 1/00307 with a mobile telephone apparatus constructional features of portable telephone sets H04M1/0202; mobile radio systems H04W84/00
• CPC Class Codes: H04N 2201/0096 Portable devices
• CPC Class Codes: H04N 23/76 by influencing the image signals
• CPC Class Codes: H04N 25/00 Circuitry of solid-state image sensors [SSIS]; Control thereof

Abstract

Image data may be processed via one or more stages by a hardware image sensor pipeline (ISP) wherein one or more software processing steps may be inserted at any point within the hardware ISP. Output from any stage of the hardware ISP may be stored in memory. Stored hardware ISP output may be retrieved from memory and processed via one or more software processes. Results from the one or more software processes may be stored in memory and communicated to any stage of the hardware ISP for additional processing. In this regard, the hardware ISP and one or more processors may simultaneously process portions of image data. In addition, the hardware ISP and the one or more processors may be integrated within a chip.

Claims

1. A method for processing images, the method comprising:

processing image data via one or more steps or stages handled by a hardware image sensor pipeline (ISP) wherein said hardware ISP enables insertion of one or more software processing steps or stages at any point within said hardware ISP.

2. The method according to claim 1, comprising storing output from any portion of said hardware ISP.

3. The method according to claim 2, comprising retrieving said stored output for handling via one or more software processes.

4. The method according to claim 3, comprising processing said retrieved stored output for processing via said one or more software processes.

5. The method according to claim 4, comprising storing results from said processing via said one or more software processes.

6. The method according to claim 5, comprising communicating said stored results from said processing via said one or more software processes to any portion of said hardware ISP for processing.

7. The method according to claim 1, comprising simultaneously processing a portion of said image data via said hardware ISP and a portion of said image data via one or more processors handling said software processes.

8. The method according to claim 1, wherein said hardware ISP and one or more processors enabled to handle said one or more software processing steps are integrated within a chip.

9. A system for processing images, the system comprising:

one or more circuits comprising a hardware image sensor pipeline (ISP), said one or more circuits enable processing of image data via one or more steps or stages handled by said hardware image sensor pipeline (ISP) and wherein said one or more circuits enable the insertion of one or more software processing steps at any point in said hardware ISP.

10. The system according to claim 9, wherein said one or more circuits enables storage of output from any portion of said hardware ISP.

11. The system according to claim 10, wherein said one or more circuits enables retrieval of said stored output for handling via one or more software processes.

12. The system according to claim 11, wherein said one or more circuits enables processing of said retrieved stored output for processing via said one or more software processes.

13. The system according to claim 12, wherein said one or more circuits enables storage of results from said processing via said one or more software processes.

14. The system according to claim 13, wherein said one or more circuits enables communication of said stored results from said processing via said one

15. The system according to claim 9, wherein said one or more circuits enables simultaneously processing of a portion of said image data via said hardware ISP and a portion of said image data via one or more processors handling said software processes.

16. The system according to claim 9, wherein said hardware ISP and one or more processors enabled to handle said one or more software processing steps are integrated within a chip.

17. A machine-readable storage having stored thereon, a computer program having at least one code section for processing images, the at least one code section being executable by a machine for causing the machine to perform steps comprising:

processing image data via one or more steps or stages handled by a hardware image sensor pipeline (ISP) wherein said hardware ISP enables the insertion of one or more software processing steps at any point in said hardware ISP.

18. The machine-readable storage according to claim 17, wherein said at least one code section comprises code that enables storing of output from any portion of said hardware ISP.

19. The machine-readable storage according to claim 18, wherein said at least one code section comprises code that enables retrieving of said stored output for handling via one or more software processes.

20. The machine-readable storage according to claim 19, wherein said at least one code section comprises code that enables processing of said retrieved stored output for processing via said one or more software processes.

21. The machine-readable storage according to claim 20, wherein said at least one code section comprises code that enables storing of results from said processing via said one or more software processes.

22. The machine-readable storage according to claim 21, wherein said at least one code section comprises code that enables communicating of said stored results from said processing via said one or more software processes to any portion of said hardware ISP for processing

23. The machine-readable storage according to claim 17, wherein said at least one code section comprises code that enables simultaneously processing of a portion of said image data via said hardware ISP and a portion of said image data via one or more processors handling said software processes.

24. The machine-readable storage according to claim 17, wherein said hardware ISP and one or more processors enabled to handle said one or more software processing steps are integrated within a chip.