Dynamically reconfigurable signal processing circuit, pattern recognition apparatus, and image processing apparatus

• US 20060228027A1
• Filed: 06/09/2006
• Published: 10/12/2006
• Est. Priority Date: 03/28/2001
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Matsugu, Masakazu, Nomura, Osamu, Mori, Katsuhiko
• Granted Patent: US 7,512,271 B2
• Time in Patent Office: Days
• US Class Current: 382/181
• CPC Class Codes: G06N 3/049 Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/94 Hardware or software architectures specially adapted for image or video understanding
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships

Abstract

A plurality of signal processing functions are achieved with the same arithmetic processing circuit by controlling wiring arrangements or signal modulation in accordance with a predetermined arrangement control signal that is output based on circuit arrangement information read from a circuit arrangement information storage unit. Hierarchical parallel processing is realized with small-scale circuit configuration. Further, detection of a predetermined feature and integration of the detection results can be efficiently performed.

Claims

1-34. -34. (canceled)

35. A signal processing circuit comprising:

an arithmetic processing circuit;

circuit arrangement information storage means for storing circuit arrangement information; and

circuit arrangement control means for outputting a predetermined arrangement control signal to said arithmetic processing circuit on the basis of the circuit arrangement information read out from said circuit arrangement information storage means,

wherein said arithmetic processing circuit executes a plurality of different signal processing functions by reconfiguring a circuit on the basis of the predetermined arrangement control signal, said arithmetic processing circuit comprising a plurality of arithmetic processing blocks, each of said arithmetic processing blocks comprising an adaptive processing block capable of changing a processing function including signal modulation for the input signal on the basis of a control signal contained in the arrangement control signal and a plurality of switch blocks arranged between signal lines for connecting said adaptive processing block and another arithmetic processing block to connect/disconnect the signal lines on the basis of the control signal contained in the arrangement control signal,

wherein said arithmetic processing circuit realizes the different signal processing functions by controlling said switch blocks and said adaptive processing blocks in accordance with the arrangement control signal from said arrangement control means,

wherein said arrangement control means controls said switch blocks and said adaptive processing block by the arrangement control signal at a timing based on arithmetic results from said plurality of arithmetic processing blocks,

wherein said arrangement control means outputs the arrangement control signal at a timing after an elapse of a predetermined time from when a sum value of the arithmetic results from said adaptive processing block, which are input to said arrangement control means, exceeds a predetermined value, and

wherein when said adaptive processing block has a refractory period, and the refractory period is larger than a difference in input signal arrival time at some or all adaptive processing blocks belonging to a single layer, the predetermined time is not less than the refractory period.

36. A signal processing circuit comprising:

an arithmetic processing circuit;

circuit arrangement information storage means for storing circuit arrangement information; and

circuit arrangement control means for outputting a predetermined arrangement control signal to said arithmetic processing circuit on the basis of the circuit arrangement information read out from said circuit arrangement information storage means,

wherein said arithmetic processing circuit executes a plurality of different signal processing functions by reconfiguring a circuit on the basis of the predetermined arrangement control signal, said arithmetic processing circuit comprising a plurality of arithmetic processing blocks, each of said arithmetic processing blocks comprising an adaptive processing block capable of changing a processing function including signal modulation for the input signal on the basis of a control signal contained in the arrangement control signal and a plurality of switch blocks arranged between signal lines for connecting said adaptive processing block and another arithmetic processing block to connect/disconnect the signal lines on the basis of the control signal contained in the arrangement control signal,

wherein said arithmetic processing circuit realizes the different signal processing functions by controlling said switch blocks and said adaptive processing blocks in accordance with the arrangement control signal from said arrangement control means,

wherein said arrangement control means controls said switch blocks and said adaptive processing block by the arrangement control signal at a timing based on arithmetic results from said plurality of arithmetic processing blocks, and

wherein said arrangement control means controls an arithmetic value of weighted addition processing or weighted integration processing for a predetermined adaptive processing block at a timing based on the arithmetic result.

37. The circuit according to claim 36, wherein control of the arithmetic value is resetting an integrated value to 0.

38. (canceled)

39. A signal processing circuit comprising:

an arithmetic processing circuit;

circuit arrangement information storage means for storing circuit arrangement information; and

circuit arrangement control means for outputting a predetermined arrangement control signal to said arithmetic processing circuit on the basis of the circuit arrangement information read out from said circuit arrangement information storage means,

wherein said arithmetic processing circuit executes a plurality of different signal processing functions by reconfiguring a circuit on the basis of the predetermined arrangement control signal,

wherein said circuit further comprises a signal input circuit for time divisionally inputting signals from a plurality of signal sources,

said arrangement control means determines connection between the plurality of signal sources and said arithmetic processing circuit to realize an interconnection structure,

said signal input circuit inputs the signals from the plurality of signal sources to said arithmetic processing circuit through the connection determined by said arrangement control means, and

said arithmetic circuit regenerates some or all of the input signals and executes predetermined signal processing for the regenerated signals.

40. The circuit according to claim 39, wherein

the signals from the plurality of signal sources form a pulse signal sequence, and

said arithmetic circuit executes time integration with a predetermined weight for the regenerated pulse signal.

41. (canceled)

42. A pattern recognition apparatus comprising:

input means for inputting pattern data; and

a signal processing circuit comprising;

an arithmetic processing circuit;

circuit arrangement information storage means for storing circuit arrangement information; and

circuit arrangement control means for outputting a predetermined arrangement control signal to said arithmetic processing circuit on the basis of the circuit arrangement information read out from said circuit arrangement information storage means,

wherein said arithmetic processing circuit executes a plurality of different signal processing functions by reconfiguring a circuit on the basis of the predetermined arrangement control signal,

said arithmetic processing circuit comprises a plurality of arithmetic processing blocks, each of said arithmetic processing blocks comprises an adaptive processing block capable of changing a processing function including signal modulation for the input signal on the basis of a control signal contained in the arrangement control signal, and a plurality of switch blocks arranged between signal lines for connecting said adaptive processing block and another arithmetic processing block to connect/disconnect the signal lines on the basis of the control signal contained in the arrangement control signal, and

said arithmetic processing circuit realizes the different signal processing functions by controlling said switch blocks and said adaptive processing blocks in accordance with the arrangement control signal from said arrangement control means, and

wherein a pulse signal corresponding to a feature of the pattern data is modulated in said arithmetic processing block in accordance with a detection category, thereby hierarchically detecting lower to higher order features and executing pattern recognition of the pattern data.

43-45. -45. (canceled)

46. A pattern recognition apparatus comprising:

time-division data input means for time serially inputting pattern data having a predetermined size as part of input data a plurality of number of times;

position information input means for inputting position information of the pattern data on the input data;

feature detection means, having a processing circuit, for detecting predetermined medium or higher order features related to a predetermined category from the pattern data;

time serial integration processing means for time serially integrating outputs from said feature detection means on the basis of the position information and the feature category to generate feature detection map information; and

determination means for outputting position information of the higher order feature in the input data and the category information on the basis of an output from said time serial integration processing means,

wherein said processing circuit comprises;

an arithmetic processing circuit;

circuit arrangement information storage means for storing circuit arrangement information; and

circuit arrangement control means for outputting a predetermined arrangement control signal to said arithmetic processing circuit on the basis of the circuit arrangement information read out from said circuit arrangement information storage means, and

said arithmetic processing circuit executes a plurality of different signal processing functions by reconfiguring a circuit on the basis of the predetermined arrangement control signal, and

wherein said time division data input means scans and inputs data in a block region having a predetermined size from the input data, and

further comprising scan control means for updating a scanning position of said data input means on the basis of a predetermined matching value with a higher order pattern to be detected, which is obtained by said integration means.

47. The apparatus according to claim 46, wherein

the apparatus further comprises size update means for updating the size of the block, and

said determination means executes determination on the basis of integration processing results corresponding to different block sizes.

48. The apparatus according to claim 46, wherein said detection means comprises an arithmetic element for detecting predetermined geometrical features having different sizes in a local region in the block.

49. A pattern recognition apparatus comprising:

time-division data input means for time serially inputting pattern data having a predetermined size as part of input data a plurality of number of times;

position information input means for inputting position information of the pattern data on the input data;

feature detection means, having a processing circuit, for detecting predetermined medium or higher order features related to a predetermined category from the pattern data;

time serial integration processing means for time serially integrating outputs from said feature detection means on the basis of the position information and the feature category to generate feature detection map information; and

determination means for outputting position information of the higher order feature in the input data and the category information on the basis of an output from said time serial integration processing means,

wherein said processing circuit comprises;

an arithmetic processing circuit;

circuit arrangement information storage means for storing circuit arrangement information; and

circuit arrangement control means for outputting a predetermined arrangement control signal to said arithmetic processing circuit on the basis of the circuit arrangement information read out from said circuit arrangement information storage means, and

said arithmetic processing circuit executes a plurality of different signal processing functions by reconfiguring a circuit on the basis of the predetermined arrangement control signal,

wherein said circuit arrangement control means time serially updates the arithmetic characteristic of said arithmetic processing circuit.

50. The apparatus according to claim 49, wherein said detection means extracts a predetermined local feature at each position in the data.

51. The apparatus according to claim 50, wherein said integration means stores a detection result of the local feature in predetermined temporal storage means together with position information of the local feature.

52. The apparatus according to claim 49, wherein said detection means comprises parallel processing means formed by arraying a plurality of arithmetic elements in parallel and connecting said arithmetic elements by predetermined connection means.

53. The apparatus according to claim 49, wherein said arithmetic processing circuit comprises a plurality of feature detection layers and feature integration layers which are alternately cascaded.

54. The apparatus according to claim 49, wherein

said integration means integrates outputs from said detection means for the pattern at a plurality of scanning positions, and

said determination means outputs position information of a pattern of a predetermined category present in the input data together with the category.

55. The apparatus according to claim 49, wherein

said integration means integrates a plurality of features detected at the plurality of scanning positions on the basis of the position information, and

said determination means determines a presence/absence of a higher order feature composed of the said plurality of features.

56. The apparatus according to claim 49, wherein said control means controls the arithmetic characteristic of said detection means so as to detect features corresponding to a plurality of different categories or sizes for the input pattern data.

57. The apparatus according to claim 49, wherein the detection map information contains information related to a presence position of the feature and at least one of a type and detection level of the feature.

58. The apparatus according to claim 49, wherein

said arithmetic processing circuit further comprises

arrayed elements capable of changing an interconnection structure there-between on the basis of the circuit arrangement information,

a storage element for supplying predetermined distributed weight data accompanying the interconnection structure, and

control means for updating and controlling the circuit arrangement information and distributed weight data,

said feature detection means has a receptive field structure unique to the interconnection structure and the distributed weight data, and

said integration processing means integrates a plurality of outputs from said feature detection means, which are obtained by updating the receptive field structure.

59. The apparatus according to claim 58, wherein

said control means time serially supplies the circuit arrangement information or distributed weight data to said feature detection means as pipeline data, and

said feature detection means time serially outputs information related to the presence/absence of detection or a detection level of a predetermined feature.

60. An image processing apparatus for controlling input operation of an image to be processed, on the basis of an output signal corresponding to a result of processing of a signal representing the image to be processed using a pattern recognition apparatus comprising:

time-division data input means for time serially inputting pattern data having a predetermined size as part of input data a plurality of number of times;

position information input means for inputting position information of the pattern data on the input data;

feature detection means, having a processing circuit, for detecting predetermined medium or higher order features related to a predetermined category from the pattern data;

time serial integration processing means for time serially integrating outputs from said feature detection means on the basis of the position information and the feature category to generate feature detection map information; and

determination means for outputting position information of the higher order feature in the input data and the category information on the basis of an output from said time serial integration processing means,

wherein said processing circuit comprises;

an arithmetic processing circuit;

circuit arrangement information storage means for storing circuit arrangement information; and

circuit arrangement control means for outputting a predetermined arrangement control signal to said arithmetic processing circuit on the basis of the circuit arrangement information read out from said circuit arrangement information storage means, and

said arithmetic processing circuit executes a plurality of different signal processing functions by reconfiguring a circuit on the basis of the predetermined arrangement control signal.

61. An image processing apparatus comprising:

image input means; and

image processing means for time serially executing a plurality of spatial filter arithmetic operations for single image data input from said image input means in correspondence with a plurality of different spatial filter characteristics.

62. The apparatus according to claim 61, wherein

said image input means comprises

a pixel array having a photoelectric conversion characteristic, and

a sensor region including read means for reading image data from said pixel array, and

said image processing means executes the spatial filter arithmetic operations for single image data read from said sensor region.

63. The apparatus according to claim 61, wherein said image input means and said image processing means are formed on a single semiconductor chip.

64. The apparatus according to claim 61, wherein

said image processing means comprises

a spatial filter circuit capable of changing a selectivity related to a predetermined geometrical feature, and

a control circuit for controlling a characteristic of said spatial filter circuit, and

said control circuit applies a finite number of different spatial filter characteristics that are set in advance to the input image data and time serially extracts a plurality of spatial filter outputs related to the input image data.

65. The apparatus according to claim 61, wherein

said image processing means comprises

a sensor circuit having a transfer circuit for transferring a pixel signal to be read from a pixel array,

a control circuit, and

a spatial filter circuit for time serially executing spatial filter processing with predetermined different characteristics,

said control circuit outputting a read control pulse signal of the pixel signal to said sensor circuit and outputting a spatial filter characteristic control signal to said spatial filter circuit on the basis of the read control pulse signal, and

said spatial filter circuit time serially executing predetermined different spatial filter processing operations for the pixel signal in correspondence with the pixel signal output that is output from said transfer circuit.

66. The apparatus according to claim 61, wherein

said image processing means comprises

a sensor circuit having a transfer circuit for transferring a pixel signal to be read from a pixel array,

a control circuit, and

a spatial filter array circuit formed by arranging a demultiplexer circuit and a spatial filter element circuit for time serially executing spatial filter processing with predetermined different characteristics,

said control circuit outputting a read control pulse signal of the pixel signal to said sensor circuit and outputting a spatial filter characteristic control signal to said spatial filter array circuit on the basis of the read control pulse signal,

said demultiplexer circuit separately outputting a time serial signal from said sensor circuit to different signal lines connected to said spatial filter element circuits on the basis of the read control pulse signal, and

said spatial filter element circuit time serially executing predetermined different spatial filter processing operations for the pixel signal in correspondence with the pixel signal output that is output from said transfer circuit.

67. The apparatus according to claim 61, wherein

said image processing means comprises

a transfer circuit for transferring a pixel signal of a sensor pixel,

a control circuit, and

a spatial filter circuit for time serially executing spatial filter processing with predetermined different characteristics,

said control circuit outputting a read pulse signal to said transfer circuit in synchronism with an output timing signal of said spatial filter circuit.

68. The apparatus according to claim 67, wherein said spatial filter circuit time serially executes predetermined local spatial filtering at a plurality of scales in a scale space.

69. The apparatus according to claim 68, wherein the local spatial filtering is predetermined multiplex scale spatial filter processing for giving a Gabor filter or an output approximate to a Gabor filter output, in which filtering operations with different scale selectivities and different direction selectivities are time serially executed.

70. The apparatus according to claim 61, wherein

said image processing means comprises

a sensor circuit having a transfer circuit for transferring a pixel signal to be read out from a pixel array,

a control circuit,

a spatial filter circuit for time serially executing spatial filter processing with predetermined different characteristics, and

a block memory for temporarily storing signals of a predetermined sensor pixel group,

said control circuit outputting a read control pulse signal of the pixel signal to said sensor circuit and said block memory and outputting a spatial filter characteristic control signal to said spatial filter circuit in synchronism with the read control pulse signal, and

said spatial filter circuit reading out, from said block memory, image data at an address corresponding to the pixel signal output that is output from said transfer circuit and time serially executing predetermined spatial filter processing for the image data in correspondence with the pixel signal.

71. The apparatus according to claim 61, further comprising

photographing means for photographing the input image data,

detection means for detecting a position and size of a pattern of a specific category from the input image data on the basis of a processing result from said d image processing means, and

control means for controlling said photographing means on the basis of a detection result from said detection means.