Dynamically reconfigurable signal processing circuit, pattern recognition apparatus, and image processing apparatus

• US 20020181799A1
• Filed: 03/26/2002
• Published: 12/05/2002
• Est. Priority Date: 03/28/2001
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Matsugu, Masakazu, Nomura, Osamu, Mori, Katsuhiko
• Granted Patent: US 7,088,860 B2
• Time in Patent Office: Days
• US Class Current: 382/260
• CPC Class Codes: G06N 3/049 Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/94 Hardware or software architectures specially adapted for image or video understanding
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships

Abstract

A plurality of signal processing functions are achieved with the same arithmetic processing circuit by controlling wiring arrangements or signal modulation in accordance with a predetermined arrangement control signal that is output based on circuit arrangement information read from a circuit arrangement information storage unit. Hierarchical parallel processing is realized with small-scale circuit configuration. Further, detection of a predetermined feature and integration of the detection results can be efficiently performed.

Claims

1. A signal processing circuit comprising:

an arithmetic processing circuit;

circuit arrangement information storage means for storing circuit arrangement information; and

circuit arrangement control means for outputting a predetermined arrangement control signal to said arithmetic processing circuit on the basis of the circuit arrangement information read out from said circuit arrangement information storage means,

wherein said arithmetic processing circuit executes a plurality of different signal processing functions by reconstructing a circuit on the basis of the predetermined arrangement control signal.

2. The circuit according to claim 1, wherein

said arithmetic processing circuit comprises

a plurality of switch block means each including a plurality of switch elements and a plurality of signal lines,

analog processing block means for executing predetermined signal modulation for an input signal, and

a signal line for connecting said switch block means or said analog processing block means, and

said circuit arrangement control means controls an ON/OFF pattern of operations of the plurality of switch elements or a signal modulation amount at said analog processing block means, thereby causing said arithmetic processing circuit to execute the plurality of different signal processing functions.

3. The circuit according to claim 2, wherein

said analog processing block means comprises a plurality of first and second type analog processing block means which are connected to said plurality of switch block means by predetermined signal lines,

said first type analog processing block means applies predetermined modulation to a signal from said second type analog processing block means, and

said second type analog processing block means outputs a predetermined signal by integrating signals from said plurality of first type analog processing block means.

4. The circuit according to claim 3, wherein

said first type analog processing block means has input signal modulating means, and

said second type analog processing block means executes weighted time integration of the input signal.

5. The circuit according to claim 2, wherein said arithmetic processing circuit further comprises logic processing block means connected to said plurality of switch block means and said analog processing block means by predetermined signal lines.

6. The circuit according to claim 5, wherein said logic processing block means has at least one AND circuit for a plurality of input signals.

7. The circuit according to claim 1, wherein said switch block means receives signals from said plurality of analog processing block means through a predetermined signal line.

8. The circuit according to claim 1, wherein said analog processing block means has input signal modulation means and a branch circuit for an output signal.

9. The circuit according to claim 1, wherein said analog processing block means executes, for an input signal, predetermined weighted addition processing or weighted integration processing at different timings.

10. The circuit according to claim 1, wherein said analog processing block means has a plurality of input signal modulation circuits capable of setting different modulation factors.

11. The circuit according to claim 10, wherein said input signal modulation circuit delays a pulse signal or modulates a phase or a width of the pulse signal.

12. The circuit according to claim 1, wherein said switch block means outputs a signal input from one of the plurality of signal lines to at least one of the remaining signal lines.

13. The circuit according to claim 12, wherein said switch block means is formed by arranging a plurality of signal lines in a single direction.

14. The circuit according to claim 1, wherein

said circuit further comprises storage means for holding intermediate output data from said arithmetic processing circuit, and

said circuit arrangement control means outputs the predetermined arrangement control signal to said arithmetic processing circuit on the basis of the circuit arrangement information read out from said circuit arrangement information storage means and the intermediate output data held by said storage means.

15. A pattern recognition apparatus having the circuit of claim 1, comprising:

input means for inputting pattern data,

wherein said arithmetic processing circuit comprises output means for detecting, for part or whole of the input pattern, a plurality of predetermined feature categories at a plurality of predetermined positions and outputting a predetermined recognition result.

16. The circuit according to claim 1, wherein

said arithmetic processing circuit comprises a plurality of arithmetic processing blocks, each of said arithmetic processing blocks comprises an adaptive processing block capable of changing a processing function including signal modulation for the input signal on the basis of a control signal contained in the arrangement control signal, and a plurality of switch blocks arranged between signal lines for connecting said adaptive processing block and another arithmetic processing block to connect/disconnect the signal lines on the basis of the control signal contained in the arrangement control signal, and

said arithmetic processing circuit realizes the different signal processing functions by controlling said switch blocks and said adaptive processing blocks in accordance with the arrangement control signal from said arrangement control means.

17. The circuit according to claim 16, wherein in said adaptive processing block, a signal modulation amount for the input signal is set in accordance with an arithmetic processing block as an output destination.

18. The circuit according to claim 16, wherein said arrangement control means outputs the arrangement control signal in synchronism with a predetermined clock signal.

19. The circuit according to claim 16, wherein said arithmetic processing circuit comprises signal regeneration means for regenerating some or all input signals.

20. The circuit according to claim 19, wherein

the input signal is a pulse signal sequence, and

said adaptive processing block executes time integration with predetermined weights for the pulse signals regenerated by said regeneration means.

21. The circuit according to claim 16, wherein said switch block receives signals processed by said plurality of arithmetic processing blocks through the signal lines.

22. The circuit according to claim 16, wherein said adaptive processing block executes, for an input signal, predetermined weighted addition processing or weighted integration processing at different timings.

23. The circuit according to claim 16, wherein said adaptive processing block has a plurality of input signal modulation circuits capable of setting different modulation factors.

24. The circuit according to claim 23, wherein said input signal modulation circuit delays a pulse signal or modulates a phase or a width of the pulse signal.

25. The circuit according to claim 16, wherein said switch block includes a plurality of switch elements for independently disconnecting/connecting the plurality of signal lines, each of said switch elements operating in accordance with a control signal.

26. The circuit according to claim 16, wherein said switch block is arranged for a plurality of signal lines that run in a single direction.

27. The circuit according to claim 16, wherein said arithmetic processing block has a hexagonal shape and has six switch blocks corresponding to sides and one adaptive processing block.

28. The circuit according to claim 16, wherein at least one of said switch blocks in one arithmetic processing block is shared as a switch block in another arithmetic processing block.

29. The circuit according to claim 16, wherein

said circuit further comprises holding means for holding output data from said plurality of arithmetic processing blocks as intermediate data, and

said arrangement control means generates the circuit arrangement control signal on the basis of the circuit arrangement information and the intermediate data held by said holding means to set the processing function of said arithmetic processing circuit and provides the intermediate data to said arithmetic processing circuit as an input signal.

30. The circuit according to claim 16, wherein said arrangement control means controls said switch blocks and said adaptive processing block by the arrangement control signal at a timing based on arithmetic results from said plurality of arithmetic processing blocks.

31. The circuit according to claim 30, further comprising signal separation means for making correspondence between an arithmetic result and an adaptive processing block that has executed arithmetic processing on a path of a signal line for inputting the arithmetic result from said arithmetic processing block to said arrangement control means.

32. The circuit according to claim 30, wherein said arrangement control means outputs the arrangement control signal at a timing after an elapse of a predetermined time from when a sum value of the arithmetic results from said adaptive processing block, which are input to said arrangement control means, exceeds a predetermined value.

33. The circuit according to claim 32, wherein the predetermined time is not less than a time when some or all adaptive processing blocks belonging to a single layer end arithmetic processing.

34. The circuit according to claim 32, wherein the predetermined time is not less than a difference in input signal arrival time at some or all adaptive processing blocks belonging to a single layer.

35. The circuit according to claim 32, wherein when said adaptive processing block has a refractory period, and the refractory period is larger than a difference in input signal arrival time at some or all adaptive processing blocks belonging to a single layer, the predetermined time is not less than the refractory period.

36. The circuit according to claim 30, wherein said arrangement control means controls an arithmetic value of weighted addition processing or weighted integration processing for a predetermined adaptive processing block at a timing based on the arithmetic result.

37. The circuit according to claim 36, wherein control of the arithmetic value is resetting an integrated value to 0.

38. The circuit according to claim 30, wherein said arithmetic processing circuit further comprises a signal regeneration circuit for an output signal from said arithmetic processing block.

39. The circuit according to claim 1, wherein

said circuit further comprises a signal input circuit for time-divisionally inputting signals from a plurality of signal sources,

said arrangement control means determines connection between the plurality of signal sources and said arithmetic processing circuit to realize an interconnection structure,

said signal input circuit inputs the signals from the plurality of signal sources to said arithmetic processing circuit through the connection determined by said arrangement control means, and

said arithmetic circuit regenerates some or all of the input signals and executes predetermined signal processing for the regenerated signals.

40. The circuit according to claim 39, wherein

the signals from the plurality of signal sources form a pulse signal sequence, and

said arithmetic circuit executes time integration with a predetermined weight for the regenerated pulse signal.

41. The circuit according to claim 39, wherein said arrangement control means receives an arithmetic result from said arithmetic processing circuit and changes the connection at a timing based on the arithmetic result.

42. A pattern recognition apparatus comprising:

input means for inputting pattern data; and

the signal processing circuit of claim 16,

wherein a pulse signal corresponding to a feature of the pattern data is modulated in said arithmetic processing block in accordance with a detection category, thereby hierarchically detecting lower- to higher-order features and executing pattern recognition of the pattern data.

43. A pattern recognition apparatus comprising:

time-division data input means for time-serially inputting pattern data having a predetermined size as part of input data a plurality of number of times;

position information input means for inputting position information of the pattern data on the input data;

feature detection means, having the said processing circuit of claim 1, for detecting predetermined medium- or higher-order features related to a predetermined category from the pattern data;

time-serial integration processing means for time-serially integrating outputs from said feature detection means on the basis of the position information and the feature category to generate feature detection map information; and

determination means for outputting position information of the higher-order feature in the input data and the category information on the basis of an output from said time-serial integration processing means.

44. The apparatus according to claim 43, wherein said detection means or said time-serial integration processing means further comprises storage means for holding a processing result.

45. The apparatus according to claim 43, wherein said time-division data input means scans and inputs data in a block region having a predetermined size from the input data.

46. The apparatus according to claim 45, further comprising scan control means for updating a scanning position of said data input means on the basis of a predetermined matching value with a higher-order pattern to be detected, which is obtained by said integration means.

47. The apparatus according to claim 45, wherein

the apparatus further comprises size update means for updating the size of the block, and

said determination means executes determination on the basis of integration processing results corresponding to different block sizes.

48. The apparatus according to claim 45, wherein said detection means comprises an arithmetic element for detecting predetermined geometrical features having different sizes in a local region in the block.

49. The apparatus according to claim 43, wherein said circuit arrangement control means time-serially updates the arithmetic characteristic of said arithmetic processing circuit.

50. The apparatus according to claim 43, wherein said detection means extracts a predetermined local feature at each position in the data.

51. The apparatus according to claim 50, wherein said integration means stores a detection result of the local feature in predetermined temporal storage means together with position information of the local feature.

52. The apparatus according to claim 43, wherein said detection means comprises parallel processing means formed by arraying a plurality of arithmetic elements in parallel and connecting said arithmetic elements by predetermined connection means.

53. The apparatus according to claim 43, wherein said arithmetic processing circuit comprises a plurality of feature detection layers and feature integration layers which are alternately cascaded.

54. The apparatus according to claim 43, wherein

said integration means integrates outputs from said detection means for the pattern at a plurality of scanning positions, and

said determination means outputs position information of a pattern of a predetermined category present in the input data together with the category.

55. The apparatus according to claim 43, wherein

said integration means integrates a plurality of features detected at the plurality of scanning positions on the basis of the position information, and

said determination means determines a presence/absence of a higher-order feature composed of the said plurality of features.

56. The apparatus according to claim 43, wherein said control means controls the arithmetic characteristic of said detection means so as to detect features corresponding to a plurality of different categories or sizes for the input pattern data.

57. The apparatus according to claim 43, wherein the detection map information contains information related to a presence position of the feature and at least one of a type and detection level of the feature.

58. The apparatus according to claim 43, wherein

said arithmetic processing circuit further comprises

arrayed elements capable of changing an interconnection structure there-between on the basis of the circuit arrangement information,

a storage element for supplying predetermined distributed weight data accompanying the interconnection structure, and

control means for updating and controlling the circuit arrangement information and distributed weight data,

said feature detection means has a receptive field structure unique to the interconnection structure and the distributed weight data, and

said integration processing means integrates a plurality of outputs from said feature detection means, which are obtained by updating the receptive field structure.

59. The apparatus according to claim 58, wherein

said control means time-serially supplies the circuit arrangement information or distributed weight data to said feature detection means as pipeline data, and

said feature detection means time-serially outputs information related to the presence/absence of detection or a detection level of a predetermined feature.

60. An image processing apparatus for controlling input operation of an image to be processed, on the basis of an output signal corresponding to a result of processing of a signal representing the image to be processed using the pattern recognition apparatus of claim 43.

61. An image processing apparatus comprising:

image input means; and

image processing means for time-serially executing a plurality of spatial filter arithmetic operations for single image data input from said image input means in correspondence with a plurality of different spatial filter characteristics.

62. The apparatus according to claim 61, wherein

said image input means comprises

a pixel array having a photoelectric conversion characteristic, and

a sensor region including read means for reading image data from said pixel array, and

said image processing means executes the spatial filter arithmetic operations for single image data read from said sensor region.

63. The apparatus according to claim 61, wherein said image input means and said image processing means are formed on a single semiconductor chip.

64. The apparatus according to claim 61, wherein

said image processing means comprises

a spatial filter circuit capable of changing a selectivity related to a predetermined geometrical feature, and

a control circuit for controlling a characteristic of said spatial filter circuit, and

said control circuit applies a finite number of different spatial filter characteristics that are set in advance to the input image data and time-serially extracts a plurality of spatial filter outputs related to the input image data.

65. The apparatus according to claim 61, wherein

said image processing means comprises

a sensor circuit having a transfer circuit for transferring a pixel signal to be read from a pixel array,

a control circuit, and

a spatial filter circuit for time-serially executing spatial filter processing with predetermined different characteristics,

said control circuit outputting a read control pulse signal of the pixel signal to said sensor circuit and outputting a spatial filter characteristic control signal to said spatial filter circuit on the basis of the read control pulse signal, and

said spatial filter circuit time-serially executing predetermined different spatial filter processing operations for the pixel signal in correspondence with the pixel signal output that is output from said transfer circuit.

66. The apparatus according to claim 61, wherein

said image processing means comprises

a sensor circuit having a transfer circuit for transferring a pixel signal to be read from a pixel array,

a control circuit, and

a spatial filter array circuit formed by arranging a demultiplexer circuit and a spatial filter element circuit for time-serially executing spatial filter processing with predetermined different characteristics,

said control circuit outputting a read control pulse signal of the pixel signal to said sensor circuit and outputting a spatial filter characteristic control signal to said spatial filter array circuit on the basis of the read control pulse signal,

said demultiplexer circuit separately outputting a time-serial signal from said sensor circuit to different signal lines connected to said spatial filter element circuits on the basis of the read control pulse signal, and

said spatial filter element circuit time-serially executing predetermined different spatial filter processing operations for the pixel signal in correspondence with the pixel signal output that is output from said transfer circuit.

67. The apparatus according to claim 61, wherein

said image processing means comprises

a transfer circuit for transferring a pixel signal of a sensor pixel,

a control circuit, and

a spatial filter circuit for time-serially executing spatial filter processing with predetermined different characteristics,

said control circuit outputting a read pulse signal to said transfer circuit in synchronism with an output timing signal of said spatial filter circuit.

68. The apparatus according to claim 67, wherein said spatial filter circuit time-serially executes predetermined local spatial filtering at a plurality of scales in a scale space.

69. The apparatus according to claim 68, wherein the local spatial filtering is predetermined multiplex scale spatial filter processing for giving a Gabor filter or an output approximate to a Gabor filter output, in which filtering operations with different scale selectivities and different direction selectivities are time-serially executed.

70. The apparatus according to claim 61, wherein

said image processing means comprises

a sensor circuit having a transfer circuit for transferring a pixel signal to be read out from a pixel array,

a control circuit,

a spatial filter circuit for time-serially executing spatial filter processing with predetermined different characteristics, and

a block memory for temporarily storing signals of a predetermined sensor pixel group,

said control circuit outputting a read control pulse signal of the pixel signal to said sensor circuit and said block memory and outputting a spatial filter characteristic control signal to said spatial filter circuit in synchronism with the read control pulse signal, and

said spatial filter circuit reading out, from said block memory, image data at an address corresponding to the pixel signal output that is output from said transfer circuit and time-serially executing predetermined spatial filter processing for the image data in correspondence with the pixel signal.

71. The apparatus according to claim 61, further comprising

photographing means for photographing the input image data,

detection means for detecting a position and size of a pattern of a specific category from the input image data on the basis of a processing result from said d image processing means, and

control means for controlling said photographing means on the basis of a detection result from said detection means.