Methods and apparatus for transmitting signals through network elements for classification

• US 20060112035A1
• Filed: 09/30/2004
• Published: 05/25/2006
• Est. Priority Date: 09/30/2004
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: International Business Machines Corporation
• Inventors: Peck, Charles Clyde III, Rao, Ravishankar, Cecchi, Guillermo Alberto, Kozloski, James Robert
• Granted Patent: US 7,287,015 B2
• Time in Patent Office: Days
• US Class Current: 706/20
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]

Abstract

The present invention provides techniques for transmitting at least one signal through an element of a classification system. One or more input signals are received at the element. One or more functional components are extracted from the one or more input signals, and one or more membership components are extracted from the one or more input signals. An output signal is generated from the element comprising a functional component and a membership component that correspond to one or more functional components and membership components from one or more input signals.

Claims

1. A method of transmitting at least one signal through an element of a classification system comprising the steps of:

receiving one or more input signals at the element;

extracting one or more functional components from the one or more input signals;

extracting one or more membership components from the one or more input signals; and

generating an output signal from the element comprising a functional component and a membership component that correspond to one or more of the functional components and membership components from the one or more input signals.

2. The method of claim 1, further comprising the step of storing in the element a first internal value representing a functional component of the element and a second internal value representing a membership component of the element.

3. The method of claim 2, wherein, in the step of generating an output signal, the functional component and the membership component of the output signal correspond to the first internal value and the second internal value, respectively.

4. The method of claim 2, wherein the step of generating an output signal further comprises the steps of:

comparing the functional and membership components of the one or more input signals to the first internal value and the second internal value; and

selecting a subset of the one or more input signals based on the comparison, wherein the subset comprises one or more functional and membership components to which the functional component and membership component of the output signal correspond.

5. The method of claim 1, wherein the step of generating an output signal further comprises the step of:

selecting an arbitrary subset of the one or more input signals comprising one or more functional components and membership components to which the functional component and membership component of the output signal correspond.

6. The method of claim 1, wherein the step of generating an output signal further comprises the step of modifying the output signal through a learning mechanism corresponding to a subset of the one or more input signals.

7. The method of claim 1, wherein the step of generating an output signal further comprises the step of computing a non-linear function of a weighted sum of a subset of the one or more functional components of the one or more input signals in generating the functional component of the output signal.

8. The method of claim 7, wherein the step of computing a non-linear function further comprises the step of applying a scalar value to each of the one or more functional components of the one or more input signals.

9. The method of claim 7, wherein the step of computing a non-linear function further comprises the step of applying a product of a scalar value and a computed value to each of the one or more functional components of the one or more input signals.

10. The method of claim 7, wherein the step of computing a non-linear function further comprises the step of modifying a scalar portion of at least one weight applied in the weighted sum.

11. The method of claim 10, wherein the step of modifying a scalar portion of at least one weight further comprises the step of adjusting one or more subsets of weights through at least one learning mechanism.

12. The method of claim 11, wherein, in the step of adjusting one or more subsets of weights, the at least one learning mechanism is dependent on learning signals corresponding to one or more input signals, internal values of the element, and the output signal.

13. The method of claim 11, wherein, in the step of adjusting one or more subsets of weights, at least two subsets of input signals contribute to the step of generating an output signal, wherein at least one subset of input signals comprises fixed weights and at least one other subset of input signals comprises adjustable weights.

14. The method of claim 13, wherein, in the step of adjusting one or more subsets of weights, a learning signal controlling the at least one learning mechanism of the adjustable weights is computed from a weighted sum of input signals from the at least one subset of input signals comprising fixed weights.

15. The method of claim 7, wherein, in the step of computing a non-linear function, the non-linear function comprises a sigmoid.

16. The method of claim 7, wherein, in the step of computing a nonlinear function, after weight adjustments a vector formed by the weights has a predetermined length.

17. The method of claim 1, wherein, in the step of generating an output signal, the membership component of the output signal comprises a reference time.

18. The method of claim 17, wherein, in the step of generating an output signal, the reference time corresponds to a classifier element internal value and reference times of the one or more input signals.

19. The method of claim 18, wherein the step of generating an output signal further comprises the step of computing a weighted average of the differences between the reference time of the classifier element and the reference time of each input signal, wherein the weight is a function of the functional component and the reference time difference of the corresponding input signal.

20. The method of claim 1, wherein, in the step of generating an output signal, each membership component of the one or more input signals are represented in a plurality of dimensions.

21. The method of claim 20, wherein a dimensional space formed by a plurality of membership components is a hyper-sphere.

22. The method of claim 1, wherein, in the step of generating an output signal, the membership component is represented by a circular variable.

23. The method of claim 1, wherein, in the step of generating an output signal, the membership component is represented by a phase.

24. The method of claim 1, wherein, in the step of receiving one or more input signals, the one or more input signals comprise at least one feedforward signal and at least one feedback signal.

25. The method of claim 24, wherein, in the step of generating an output signal, the functional component of the output signal corresponds to the one or more functional components of the at least one feedforward signal.

26. The method of claim 24, wherein, in the step of generating an output signal, the membership component of the output signal corresponds to the one or more membership components of the at least one feedforward signal and the at least one feedback signal.

27. A method of transmitting signals in a network of elements comprising the steps of:

communicating signals from each of one or more source elements to at least one destination element in the network of elements, wherein each signal comprises a functional component and a membership component;

generating a new signal at each destination element comprising a functional component and a membership component that correspond to one or more functional components and membership components communicated from one or more respective source elements; and

repeating the steps of communicating signals and generating a new signal until one or more final destination elements are reached, wherein a given source element in a next subsequent iteration comprises the destination element in a current iteration, and a communicated signal in the next subsequent iteration comprises the new signal in the current iteration.

28. The method of claim 27, wherein the step of generating a new signal comprises the step of comparing the membership component of the at least one destination element with the membership component of the one or more source elements when the connections between the at least one destination element and the one or more source elements comprise at least two connection types that produce different effects on the functional and membership components of the at least one destination element.

29. The method of claim 27, wherein, in the step of communicating signals, the one or more source elements and at least one destination element each comprise a sub-network of elements extracting different features from an input network.

30. The method of claim 27, wherein the step of repeating the steps of communicating signals further comprises the step of communicating signals through a hierarchy of sub-networks where lower sub-networks extract local features, and higher sub-networks extract global features.

31. The method of claim 27, wherein the step of repeating the steps of communicating signals further comprises the steps of:

activating a decision-making sub-network by the presence of one or more global features; and

comparing functional and membership components of higher and lower sub-networks, such that only those elements in the lower sub-networks with a high functional component and a membership component similar to the membership component of one or more global features in the higher sub-network are selected to be acted upon.

32. Apparatus for transmitting at least one signal through an element of a classification system, comprising:

a memory; and

at least one processor coupled to the memory and operative to;

(i) receive one or more input signals at the element;

(ii) extract one or more functional components from the one or more input signals;

(iii) extract one or more membership components from the one or more input signals; and

(iv) generate an output signal from the element comprising a functional component and a membership component that correspond to one or more functional components and membership components from the one or more input signals.

33. The apparatus of claim 32, further comprising the operation of storing in the element a first internal value representing a functional component of the element and a second internal value representing a membership component of the element.

34. The apparatus of claim 33, wherein, in the operation of generating an output signal, the functional component and the membership component of the output signal correspond to the first internal value and the second internal value, respectively.

35. The apparatus of claim 33, wherein the operation of generating an output signal further comprises the operations of:

comparing the functional and membership components of the one or more input signals to the first internal value and the second internal value; and

selecting a subset of the one or more input signals based on the comparison, wherein the subset comprises one or more functional and membership components to which the functional component and membership component of the output signal correspond.

36. The apparatus of claim 32, wherein the operation of generating an output signal further comprises the operation of:

selecting an arbitrary subset of the one or more input signals comprising one or more functional components and membership components to which the functional component and membership component of the output signal correspond.

37. The apparatus of claim 32, wherein the operation of generating an output signal further comprises the operation of modifying the output signal through a learning mechanism corresponding to a subset of the one or more input signals.

38. The apparatus of claim 32, wherein the operation of generating an output signal further comprises the operation of computing a non-linear function of a weighted sum of a subset of the one or more functional components of the one or more input signals in generating the functional component of the output signal.

39. The apparatus of claim 32, wherein, in the operation of receiving one or more input signals, the one or more input signals comprise at least one feedforward signal and at least one feedback signal.

40. The apparatus of claim 39, wherein, in the operation of generating an output signal, the functional component of the output signal corresponds to the one or more functional components of the at least one feedforward signal.

41. The apparatus of claim 39, wherein, in the operation of generating an output signal, the membership component of the output signal corresponds to the one or more membership components of the at least one feedforward signal and the at least one feedback signal.

42. An article of manufacture for transmitting at least one signal through a element of a classification system, comprising a machine readable medium containing one or more programs which when executed implement the steps of:

receiving one or more input signals at the element;

extracting one or more functional components from the one or more input signals;

extracting one or more membership components from the one or more input signals; and

generating an output signal from the element comprising a functional component and a membership component that correspond to one or more of the functional components and membership components from the one or more input signals.