Security camera system and image processing apparatus

• US 10,540,565 B2
• Filed: 04/17/2018
• Issued: 01/21/2020
• Est. Priority Date: 05/24/2017
• Status: Active Grant

• Current Assignee: Renesas Electronics Corporation
• Original Assignee: Renesas Electronics Corporation
• Inventors: Niikura, Kentarou
• Primary Examiner(s): Matt, Marnie A
• Application Number:
  US15/954,670
• Publication Number:
  US 20180341827A1
• Time in Patent Office: 644 Days
• Field of Search: 348143
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G08B 13/19667 Details realated to data compression, encryption or encoding, e.g. resolution modes for reducing data volume to lower transmission bandwidth or memory requirements
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

A security camera system capable of reducing an amount of information without degrading useful information is provided. The security camera system includes a pixel array, an image signal processor that performs various types of image processing on raw data including each pixel value from the pixel array, and a pre-stage processing unit arranged between the pixel array and the image signal processor. The pre-stage processing unit includes an important area extraction circuit that receives raw data from the pixel array with respect to each frame and divides an effective pixel area of the frame into an important area and an unimportant area, and transmits the raw data of the important area to the image signal processor.

Claims

1. A security camera system comprising:

a pixel array included by a plurality of imaging elements;

an image signal processor that performs various types of image processing on raw data including each pixel value from the pixel array; and

an image processing unit arranged between the pixel array and the image signal processor,

wherein the image processing unit comprises an important area extraction circuit that receives the raw data from the pixel array with respect to each frame and divides an effective pixel area of the frame into an important area and an unimportant area, and transmits the raw data of the important area to the image signal processor;

wherein the image processing unit further comprises a data synthesis circuit that combines predetermined additional information and the raw data from the pixel array and transmits the resulting data to the image signal processor, and

wherein, based on distinction information from the important area extraction circuit, the data synthesis circuit transmits the raw data of the important area to the image signal processor during a period corresponding to the important area in the transmission period of the frame, and transmits the predetermined additional information to the image signal processor instead of the raw data of the unimportant area during a period corresponding to the unimportant area.

2. The security camera system according to claim 1,

wherein the image processing unit further comprises a blanking period information generation circuit that generates coordinate information of the important area as information about a blanking period in the transmission period of the frame,

wherein the data synthesis circuit transmits the coordinate information of the important area from the blanking period information generation circuit to the image signal processor during the blanking period, and

wherein the image signal processor obtains the raw data of the important area and the predetermined additional information separately by recognizing the coordinate information of the important area during the blanking period.

3. The security camera system according to claim 2,

wherein the data synthesis circuit transmits the coordinate information of the important area to the image signal processor during a horizontal blanking period located at the top of the horizontal coordinate at which the important area is located.

4. The security camera system according to claim 1,

wherein the image processing unit further comprises an additional information generation circuit that generates the predetermined additional information, and

wherein the additional information generation circuit performs predetermined image analysis on the raw data of the important area based on the distinction information from the important area extraction circuit, and generates the analysis result as the predetermined additional information.

5. The security camera system according to claim 1, further comprising:

a predetermined sensor that is provided in the pixel array or separately from the pixel array and outputs sensor data in parallel with the raw data with respect to each frame from the pixel array with respect to each frame,

wherein the image processing unit further comprises an additional information generation circuit that receives the sensor data and generates information based on the sensor data as the predetermined additional information.

6. The security camera system according to claim 1,

wherein the important area extraction circuit distinguishes between the important area and the unimportant area based on comparison between the each pixel value of the raw data of a previous frame and the each pixel value of the raw data of a current frame.

7. The security camera system according to claim 6,

wherein the important area extraction circuit manages the effective pixel area by dividing it into a plurality of blocks, and determines whether the block is the important area or the unimportant area by comparing either an integrated value or an average value of the each pixel value included in each of the blocks between the last time and this time.

8. The security camera system according to claim 1, further comprising:

a predetermined sensor that is provided in the pixel array or separately from the pixel array and outputs sensor data in parallel with the raw data with respect to each frame from the pixel array,

wherein the important area extraction circuit distinguishes between the important area and the unimportant area based on the sensor data.

9. The security camera system according to claim 1,

wherein the pixel array and the image processing unit are mounted on a single semiconductor chip.

10. A security camera system comprising:

a pixel array included by a plurality of imaging elements;

an image signal processor that performs various types of image processing on raw data including each pixel value from the pixel array; and

an image processing unit provided at a later stage of the image signal processor,

wherein the image processing unit comprises an important area extraction circuit that receives image data from the image signal processor with respect to each frame, and divides an image area with respect to each frame into an important area and an unimportant area.

11. An image processing apparatus arranged between a pixel array included by a plurality of imaging elements and an image signal processor that performs various types of image processing on raw data including each pixel value from the pixel array,

wherein the image processing apparatus comprises an important area extraction circuit that receives the raw data from the pixel array with respect to each frame and divides an effective pixel area of the frame into an important area and an unimportant area and transmits the raw data of the important area to the image signal processor.

12. The image processing apparatus according to claim 11, further comprising:

a data synthesis circuit that combines predetermined additional information and the raw data from the pixel array and transmits the resulting data to the image signal processor,

wherein, based on distinction information from the important area extraction circuit, the data synthesis circuit transmits the raw data of the important area to the image signal processor during a period corresponding to the important area in the transmission period of the frame, and transmits the predetermined additional information to the image signal processor instead of the raw data of the unimportant area during a period corresponding to the unimportant area.

13. The image processing apparatus according to claim 12, further comprising:

a blanking period information generation circuit that generates coordinate information of the important area as information about the blanking period in the transmission period of the frame,

wherein the data synthesis circuit transmits the coordinate information of the important area from the blanking period information generation circuit to the image signal processor during the blanking period.

14. The image processing apparatus according to claim 13,

wherein the data synthesis circuit transmits the coordinate information of the important area to the image signal processor during a horizontal blanking period located at the top of the horizontal coordinate at which the important area is located.

15. The image processing apparatus according to claim 12, further comprising:

an additional information generation circuit that generates the predetermined additional information,

wherein the additional information generation circuit performs predetermined image analysis on the raw data of the important area based on the distinction information from the important area extraction circuit, and generates the analysis result as the predetermined additional information.

16. The image processing apparatus according to claim 12, further comprising:

an additional information generation circuit that receives sensor data from a predetermined sensor provided in the pixel array or separately from the pixel array in parallel with the raw data with respect to each frame from the pixel array, and generates information based on the sensor data as the predetermined additional information.

17. The image processing apparatus according to claim 11,

wherein the important area extraction circuit distinguishes between the important area and the unimportant area based on comparison between the each pixel value of the raw data of a previous frame and the each pixel value of the raw data of a current frame.

18. The image processing apparatus according to claim 17,

wherein the important area extraction circuit manages the effective pixel area by dividing it into a plurality of blocks, and determines whether the block is the important area or the unimportant area by comparing either an integrated value or an average value of the each pixel value included in each of the blocks between the last time and this time.

19. The image processing apparatus according to claim 11,

wherein the important area extraction circuit receives sensor data from a predetermined sensor provided in the pixel array or separately from the pixel array in parallel with the raw data with respect to each frame from the pixel array, and distinguishes between the important area and the unimportant area based on the sensor data.