Image processing device and image processing method

• US 10,453,166 B2
• Filed: 11/13/2017
• Issued: 10/22/2019
• Est. Priority Date: 06/12/2015
• Status: Active Grant

• Current Assignee: Olympus Corporation
• Original Assignee: Olympus Corporation
• Inventors: Nakazono, Keisuke, Ueno, Akira
• Primary Examiner(s): Buttram, Todd
• Application Number:
  US15/810,481
• Publication Number:
  US 20180068413A1
• Time in Patent Office: 708 Days
• CPC Class Codes: G06F 13/1673 using buffers
• CPC Class Codes: G06F 13/28 using burst mode transfer, e.g. direct memory access DMA, cycle steal G06F13/32 takes precedence
• CPC Class Codes: G06F 13/38 Information transfer, e.g. on bus G06F13/14 takes precedence
• CPC Class Codes: G06F 13/4022 using switching circuits, e.g. switching matrix, connection or expansion network G06F13/4009 takes precedence
• CPC Class Codes: G06F 15/173 using an interconnection network, e.g. matrix, shuffle, pyramid, star, snowflake
• CPC Class Codes: G06F 9/38 Concurrent instruction execution, e.g. pipeline or look ahead
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 1/60 Memory management
• CPC Class Codes: G06T 3/4007 based on interpolation, e.g. bilinear interpolation image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403
• CPC Class Codes: H04N 25/60 Noise processing, e.g. detecting, correcting, reducing or removing noise
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

There is provided an image processing device includes an image processing section including a pipeline in which a plurality of processing modules is connected in series, and a system control section performing setting the pipeline processing to be performed by the image processing section. Each of the processing modules includes a data buffer temporarily store the data, on arithmetic section configured to perform an arithmetic operation of the process according to a setting of processing content by the system control section, and a control section configured to select a data transfer path within the processing module and control an operation of the data buffer according to a setting of a data transfer path by the system control section. The image processing section further includes a connection switching section configured to switch a connection between the processing modules according to a setting of a pipeline configuration by the system control section.

Claims

1. An image processing device that includes a memory, an image processing section and a system control section, the memory and the image processing section being connected to a data bus, the image processing section including a pipeline in which a plurality of processing modules is connected in series, each processing modules being configured to perform a predetermined process on input data, the image processing section performing pipeline processing by the processing modules sequentially performing the process, the system control section performing setting the pipeline processing to be performed by the image processing section,

wherein each of the processing modules includes:

a data buffer configured to temporarily store the data in unit of processing,

an arithmetic section configured to perform an arithmetic operation of the process in accordance with a setting of processing content by the system control section, and

a control section configured to select a path within the processing module by which the data is transferred to the arithmetic section and control an operation of the data buffer in accordance with a setting of a data transfer path by the system control section,

wherein the image processing section further includes;

a connection switching section configured to switch a connection between the processing modules constituting the pipeline in accordance with a setting of a pipeline configuration by the system control section,

wherein at least one processing module among the plurality of processing modules is a second processing module including a second arithmetic section which is the arithmetic section for exclusively performing arithmetic operations of a plurality of predetermined processes, and

wherein the system control section performs the setting of the transfer path and the setting of the pipeline configuration on the basis of contents of the process to be performed on the data by the second processing module and a storage capacity of the data buffer provided in the processing module connected adjacent to the second processing module.

2. The image processing device according to claim 1,

wherein the system control section sets the second arithmetic section and the arithmetic section provided in the adjacently connected processing module as a single arithmetic section if remaining storage capacity in the data buffer is present when the second processing module performs the arithmetic operation of the process according to the setting of the processing content,

wherein the system control section sets the transfer path of the adjacently connected processing module to a path along which the data bypasses the data buffer and transferred to the arithmetic section without being stored in the data buffer, and causes an operation of the data buffer provided in the adjacently connected processing module to stop if the remaining storage capacity is a capacity available instead of the storage capacity of the data buffer provided in the adjacently connected processing module, and

wherein the system control section sets the transfer path of the adjacently connected processing module to a path along which the data is read and transferred to the arithmetic section after being temporarily stored in the data buffer if the remaining storage capacity is a capacity unavailable instead of the storage capacity of the data buffer provided in the adjacently connected processing module.

3. The image processing device according to claim 2,

wherein the system control section calculates the remaining storage capacity of the data buffer provided in the second processing module on the basis of the entire storage capacity of the data buffer and the storage capacity of the data buffer used when the second processing module performs the arithmetic operation of the process according to the setting of the processing content.

4. The image processing device according to claim 1,

wherein the processing module and the second processing module further include a selector configured to select either a path along which the data read from the data buffer is transferred or a path along which the data is transferred by bypassing the data buffer, as a path along which the data is transferred.

5. The image processing device according to claim 1,

wherein the data is pixel data corresponding to each pixel included in an image,

wherein the second arithmetic section is an arithmetic section for exclusively performing arithmetic operations of a plurality of processes according to an image interpolation process using a function of converting coordinates of each pixel included in the image, and

wherein the system control section calculates the remaining storage capacity of the data buffer provided in the second processing module on the basis of the entire storage capacity of the data buffer and the storage capacity of the data buffer used when the second processing module performs the arithmetic operation of the image interpolation process according to a setting value of the image interpolation process.

6. The image processing device according to claim 5,

wherein the image interpolation process includes at least a resizing process for changing a size of the image and a distortion correction process of correcting distortion of distortion aberration included in the image.

7. The image processing device according to claim 1,

wherein the processing module connected adjacent to the second processing module is connected to a stage subsequent to the second processing module in the pipeline configuration.

8. An image processing method in an image processing device that includes a memory, an image processing section and a system control section, the memory and the image processing section being connected to a data bus, the image processing section including a pipeline in which a plurality of processing modules is connected in series, each processing modules being configured to perform a predetermined process on input data, the image processing section performing pipeline processing by the processing modules sequentially performing the process, and the system control section performing setting the pipeline processing to be performed by the image processing section, the image processing method comprising:

an arithmetic step, by an arithmetic section of the processing modules, performing an arithmetic operation of the process in according to a setting of processing content by the system control section;

a control step, by a control section of the processing modules, selecting a path within the processing module by which the data is transferred to the arithmetic section and controlling an operation of a data buffer configured to temporarily store the data in unit of processing in accordance with a setting of a data transfer path by the system control section;

a connection switching step, by the image processing section, switching a connection between the processing modules constituting the pipeline in accordance with a setting of a pipeline configuration by the system control section;

wherein at least one processing module among the plurality of processing modules is a second processing module including a second arithmetic section which is the arithmetic section for exclusively performing arithmetic operations of a plurality of predetermined processes, and

wherein the setting of the transfer path and the setting of the pipeline configuration is performed by the system control section on the basis of contents of the process to be performed on the data by the second processing module and a storage capacity of the data buffer provided in the processing module connected adjacent to the second processing module.