METHOD AND APPARATUS FOR TIGHTLY COUPLED, LOW POWER IMAGE PROCESSING

• US 20130120593A1
• Filed: 11/11/2011
• Published: 05/16/2013
• Est. Priority Date: 11/11/2011
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Shen, Jian, Chua-Eoan, Lew G., Hsiao, Yi-Pin
• Granted Patent: US 9,001,267 B2
• Time in Patent Office: Days
• US Class Current: 348/207.1
• CPC Class Codes: G06F 13/00 Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units interface circuits for specific input/output devices G06F3/00 ; multiprogram control therefor G06F9/46; multiprocessor systems G06F15/16
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: H04N 1/2112 using still video cameras
• CPC Class Codes: H04N 19/423 characterised by memory arrangements H04N19/433 takes precedence
• CPC Class Codes: H04N 25/00 Circuitry of solid-state image sensors [SSIS]; Control thereof
• CPC Class Codes: H04N 25/443 by reading pixels from selected 2D regions of the array, e.g. for windowing or digital zooming
• CPC Class Codes: H04N 5/772 the recording apparatus and the television camera being placed in the same enclosure

Abstract

An image divided into N pixel blocks, stored block wise in a camera core and transferred block wise from the camera core to a downstream processing engine local to the local memory. A direct handshaking is communicated, between the camera core and the downstream processing engine, in the block wise transfers. Optionally an optical sensor scanner divides the image with a scan rate N times a frame rate, each scan providing a block of the frame. Optionally, the block wise transfer includes a transfer through a local memory, local to the camera core, controlled by the direct handshaking.

Claims

1. A method of processing an image, comprising:

receiving the image at a camera core; and

transferring the image on a block basis from the camera core to a downstream processing engine, wherein the transferring includes communicating a handshaking signal between the camera core and the downstream processing engine.

2. The method of claim 1, wherein the transferring transfers the image as N image blocks over a local interconnect coupled to the camera core and to the downstream processing engine.

3. The method of claim 2, wherein the communicating a handshaking signal between the camera core and the downstream processing engine is over a direct handshaking path coupled to the camera core and to the downstream processing engine.

4. The method of claim 1, further comprising an image processing at the downstream processing engine according to a given order, wherein the transferring transfers the image as a sequence of image blocks according to the given order.

5. The method of claim 1, wherein the downstream processing engine is a video coder/decoder (VCODEC), graphics processing unit (GPU) or display.

6. The method of claim 1, wherein the transferring includes sending the image from the camera core over a local interconnect to a local memory, and transferring the image as image blocks from the local memory, over the local interconnect, to the downstream processing engine.

7. The method of claim 1, wherein the transferring includes transferring the image to a local memory over a local interconnect coupled to the camera core, the local memory and the downstream processing engine, and transferring the image as N image blocks from the local memory to the downstream processing engine over the local interconnect.

8. The method of claim 7, wherein the communicating a handshake signal between the camera core and the downstream processing engine is associated with the transferring the image as N image blocks from the local memory to the downstream processing engine.

9. The method of claim 8, wherein communicating the handshake signals between the camera core and the downstream processing engine includes communicating handshake signals indicating a data availability.

10. The method of claim 8, wherein communicating the handshake signals between the camera core and the downstream processing engine is based on preventing an overflow in a buffer associated with the camera core.

11. The method of claim 10, wherein the buffer is in the local memory or in the downstream processing engine.

12. The method of claim 1, wherein receiving the image at the camera core comprises:

generating the image at an optical sensor;

dividing the image into N image blocks;

transferring the N image blocks sequentially to the camera core; and

concurrently storing a number of the N image blocks in the camera core.

13. The method of claim 12, wherein N is in the range of 1-20.

14. The method of claim 12, wherein the number of the image blocks in the camera core is less than N.

15. The method of claim 1, wherein receiving the image at the camera core comprises:

generating the image at an optical sensor;

dividing the image into a plurality of N image blocks;

transferring the image blocks sequentially to at least one of the camera core and a local memory associated with the camera core; and

concurrently storing a quantity of the N image blocks in at least one of the camera core and the local memory.

16. The method of claim 15, wherein transferring the image on a block basis from the camera core to the downstream processing engine includes transferring the image blocks from at least one of the camera core and the local memory, over a local interconnect, to the local downstream processing engine.

17. The method of claim 1, wherein receiving the image at the camera core comprises:

scanning the image N times at an optical sensor to generate N scans;

transferring, for each of the N scans, at least one image block of the image to at least one of the camera core and a local memory associated with the camera core; and

concurrently storing a quantity of the N image blocks in at least one of the camera core and the local memory.

18. The method of claim 17, wherein transferring the image on a block basis wise from the camera core to the downstream processing engine includes transferring the image blocks of the image from at least one of the camera core and the local memory, over a local interconnect, to the local downstream processing engine.

19. A method of transferring an image to a processing engine, comprising:

scanning an optical sensor to obtain one scan of the image,

extracting an image block of a given N-block division of the one scan of the image;

storing the extracted image block in at least one of a camera core or a local memory associated with the camera core;

transferring the extracted image block from at least one of the camera core or the local memory to a downstream processing engine; and

repeating, N times, the scanning, the extracting, the storing, and the transferring to provide the image to the downstream processing engine.

20. The method of claim 19, wherein the transferring the extracted image block from at least one of the camera core or the local memory to a downstream processing engine includes communicating a handshaking signal between the camera core and the downstream processing engine.

21. The method of claim 20, wherein transferring the one extracted image block transfers the extracted image block over a local interconnect coupled to the downstream processing engine and to at least one of the camera core and the local memory.

22. The method of claim 21, wherein communicating a handshaking signal between the camera core and the downstream processing engine is over a direct handshaking path coupled to the camera core and to the downstream processing engine.

23. The method of claim 22, wherein storing the extracted image block in at least one of the camera core or the local memory includes:

storing the extracting image block in the camera core; and

transferring the extracted image block from the camera core to the local memory.

24. The method of claim 20, wherein the image is a video frame, and wherein the method further comprises providing a sequence of the video frames to the processing engine by the scanning being at a scan rate of N times FS and, for each of the video frames, performing the repeating N times of the scanning, the extracting, the storing, and the transferring, at rate to provide the sequence of the video frames at a frame rate of FS.

25. An image processor, comprising:

a camera core;

a downstream processor coupled over a data interconnect to the camera core; and

a direct handshaking path coupled to the camera core and to the downstream processor,

wherein the camera core is configured to receive a video frame and to transfer the video frame by N block transfers over the data interconnect to the downstream processor, each of the block transfers transferring a corresponding one of N pixel blocks of the video frame, and to control the N block transfers over the direct handshaking path.

26. The image processor of claim 25, wherein the camera core is configured to receive the video frame as a succession of N scan frames.

27. The image processor of claim 26, wherein the camera core is configured to extract from each of the N scan frames a corresponding one of the N pixel blocks.

28. The image processor of claim 25, wherein the downstream processor is any from among a video coder/decoder (VCODEC), graphics processing unit (GPU) or display.

29. The image processor of claim 25, further comprising a local memory coupled to the data interconnect, wherein the camera core and the local memory are configured to perform the block transfers by transferring pixel blocks from the camera core to the local memory over the data interconnect and by block transfers over the data interconnect from the local memory to the downstream processor, and wherein the camera core is configured to control the block transfers from the local memory to the downstream processor through the direct handshaking path.

30. The image processor of claim 25, wherein the camera core includes a buffer to store at least a portion of the video frame, wherein the camera core is configured to detect a fill level of the buffer and to control the block transfers to the downstream processor based on the detected fill level.

31. The image processor of claim 30, wherein said control of the block transfers is based, at least in part, on the detected fill level includes communicating direct handshake signals based on the detected fill level.

32. The image processor of claim 30, wherein said control of the block transfers based on the detected fill level controls the block transfers to preventing an overflow of the buffer.

33. The image processor of claim 25, wherein the image processor is integrated in at least one semiconductor die.

34. The image processor of claim 25, further comprising a device, selected from a group consisting of a set top box, music player, video player, entertainment unit, navigation device, communications device, personal digital assistant (PDA), fixed location data unit, and a computer, into which the image processor is integrated.

35. An image processor, comprising:

means for scanning an optical sensor to obtain one scan of the image,

means for extracting a block of a given N-block division of the one scan of the image;

means for storing an extracted block, resulting from the extracting, in at least one of a camera core or a local memory associated with the camera core; and

means for transferring the extracted block from at least one of the camera core or the local memory to a downstream processing engine, wherein transferring the extracted block from at least one of the camera core or the local memory to the downstream processing engine includes communicating handshaking signals between the camera core and the downstream processing engine.

36. The image processor of claim 35, wherein the image processor is integrated in at least one semiconductor die.

37. The image processor of claim 35, further comprising a device, selected from a group consisting of a set top box, music player, video player, entertainment unit, navigation device, communications device, personal digital assistant (PDA), fixed location data unit, and a computer, into which the image processor is integrated.

38. A computer program product, having a computer-readable medium comprising:

code for causing at least one computer to receive at a camera core an image; and

code for causing at least one computer to transfer the image on a block basis, to a downstream processing engine, and to communicate, in association with the transfer on a block basis, a handshaking signal between the camera core and the downstream processing engine, over a direct handshaking path coupled to the camera core and to the downstream processing engine.

39. The computer program product of claim 38, further comprising code for causing a processing of the image at the downstream processing engine according to a given order, and wherein transferring the image on a block basis transfers the image as a sequence of image blocks according to the given order.

40. The computer program product of claim 38, wherein the code for causing at least one computer to transfer the image on a block basis causes the at least one computer to transfer the image to a local memory over a local interconnect coupled to the camera core, the local memory and the downstream processing engine, and to transfer the image from the local memory to the downstream processing engine over the local interconnect.

41. The computer program product of claim 40, wherein transfer of the image blocks from the local memory to the downstream processing engine includes communicating direct handshake signals between the camera core and the downstream processing engine over a direct handshaking path coupled to the camera core and to the downstream processing engine.

42. The computer program product of claim 41, wherein communicating the direct handshake signals between the camera core and the downstream processing engine is based on preventing an overflow in a buffer associated with the camera core.

43. The computer program product of claim 38, wherein the code for causing at least one computer to receive the image comprises:

code for causing at least one computer to scan the image N times at an optical sensor to generate N scans;

code for causing at least one computer to transfer, for each of the N scans, at least one block of the image to at least one of the camera core and a local memory associated with the camera core; and

code for causing at least one computer to concurrently store a quantity of N of the blocks of the image in at least one of the camera core and the local memory.

44. A computer program product, having a computer-readable medium comprising:

code for causing at least one computer to scan an optical sensor to obtain one scan of the image;

code for causing at least one computer to extract an image block of the one scan of the image, according to a given N-block division;

code for causing at least one computer to store the extracted image block in at least one of a camera core of a local memory associated with the camera core;

code for causing at least one computer to transfer the extracted image block from at least one of the camera core or the local memory to a downstream processing engine; and

code for causing at least one computer to repeat the scan, extract, store, and transfer N times to provide the image to the downstream processing engine.

45. The computer program product of claim 44, wherein the code for causing at least one computer to store the extracted image block in at least one of the camera core or the local memory includes:

code for causing at least one computer to store the extracted image block in the camera core; and

code for causing at least one computer to transfer the extracted image block from the camera core to the local memory.

46. An image processor, comprising:

a camera core having means for receiving an image and for concurrently storing at least a portion of the image; and

means for transferring the image on a block basis from the camera core to a downstream processing engine, wherein the transferring includes communicating a handshaking signal between the camera core and the downstream processing engine.

47. The image processor of claim 46, further comprising means for detecting a fill level associated with the means for storing an image, wherein communicating the handshake signals between the camera core and the downstream processing engine is based on said detecting.

48. The image processor of claim 46, wherein the image processor is integrated in at least one semiconductor die.

49. The image processor of claim 46, further comprising a device, selected from a group consisting of a set top box, music player, video player, entertainment unit, navigation device, communications device, personal digital assistant (PDA), fixed location data unit, and a computer, into which the image processor is integrated.

50. A method of image processing, comprising:

step of receiving an image at a camera core; and

step of transferring the image on a block basis from the camera core to a downstream processing engine, wherein the transferring includes communicating a handshaking signal between the camera core and the downstream processing engine over a direct handshaking path coupled to the camera core and to the downstream processing engine.

51. A method of transferring an image to a processing engine, comprising:

step of scanning an optical sensor to obtain one scan of the image,

step of extracting an image block of the one scan of the image according to a given N-block division of the one scan;

step of storing the extracted image block in at least one of a camera core or a local memory associated with the camera core;

step of transferring the extracted image block from at least one of the camera core or the local memory to a downstream processing engine; and

repeating, N times, the step of scanning, the step of extracting, the step of storing, and the step of transferring to provide the image to the downstream processing engine.

52. The method of claim 51, wherein transferring the extracted image block from at least one of the camera core or the local memory to the downstream processing engine includes communicating a handshaking signal between the camera core and the downstream processing engine.