Method and apparatus for determining image affine flow using artifical neural system with simple cells and lie germs

• US 5,535,302 A
• Filed: 12/01/1994
• Issued: 07/09/1996
• Est. Priority Date: 12/01/1994
• Status: Expired due to Fees

• Current Assignee: Tien-Ren Tsao
• Original Assignee: Tien-Ren Tsao
• Inventors: Tsao, Tien-Ren
• Primary Examiner(s): Bayerl, Raymond J.
• Assistant Examiner(s): KATBAB, ABDOLLAH
• Application Number:
  US08/352,140
• Time in Patent Office: 586 Days
• Field of Search: 395/21, 395/24, 395/22, 395/23, 382/107, 382/158, 382/156, 382/157, 382/103, 382/291, 348/699
• US Class Current: 706/20
• CPC Class Codes: G06N 3/065 Analogue means
• CPC Class Codes: G06T 7/269 using gradient-based methods
• CPC Class Codes: G06V 10/449 Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters

Abstract

A method and apparatus is provided to determine image affine flow from time-varying imagery. The novel artificial neural computational system of a cortical hypercolumn comprising a plurality of specific orientation (SO) columns and a least square error fitting circuit is based on a Lie group model of cortical visual motion processing. Time-varying imagery, comprising intensity imagery and time-derivative imagery is provided to a plurality of specific orientation (SO) columns comprising simple cells and Lie germs. The cortical representation of image time derivative and affine Lie derivatives are extracted from responses of simple cells and Lie germs, respectively. The temporal derivative and affine Lie-derivative information obtained from each specific orientation (SO) columns is applied to least square error fitting analog circuit having a three layer multiplicative neural architecture to determine image affine flow components in accordance with an error minimization gradient dynamical system technique.

Claims

1. An artificial neural system for extracting local image affine flow information from visual stimuli comprising:

a) time differentiation means for receiving time-varying image intensity signal and providing image time-derivative signal;

b) hypercolumn (HC) reference frame means coupled to said time differentiation means for receiving said image time-derivative signal and providing HC-time-derivative signal;

c) hypercolumn (HC) Lie differentiation means for receiving image intensity signal and providing six HC-Lie-derivative signals;

d) least square error fitting means coupled to said HC-reference frame means and said HC-Lie differentiation means for receiving HC-time-derivative signal and six HC-Lie-derivative signals and determining local affine flow.

2. The system of claim 1, wherein said HC-reference frame means comprising a plurality of specific orientation (SO) simple cell means.

3. The system of claim 2, wherein said SO-simple cell means comprising permanent memory means for preserving an array of fixed weights, and weighted summing means for receiving input pattern signal and producing SO-response signal.

4. The system of claim 3, wherein said array of fixed weights is constructed according to a particular rapid descent analytical function describing the receptive field of said SO-simple cell.

5. The system of claim 1, wherein said HC-Lie differentiation means comprising six Lie-derivative operator means, for receiving said image intensity signal and producing HC-Lie-derivative signal.

6. The system of claim 5, wherein said Lie-derivative-operator means further comprising a plurality of specific orientation (SO) Lie germ means.

7. The system of claim 6, wherein said SO-Lie germ means of said Lie-derivative-operator means comprising permanent memory means for preserving an array of fixed weights, and weighted summing means for receiving input pattern signal and producing SO-Lie germ response signal.

8. The system of claim 7, wherein said array of fixed weights is constructed according to function analytically derived by applying the Hilbert space conjugate of the infinitesimal generator corresponding to said Lie-derivative-operator to said analytical function describing the receptive field of said SO-simple cell.

9. The system of claim 1, wherein said least square error fitting means comprising six affine parameter extraction means, each corresponding to one of six affine parameters of the two dimensional affine Lie group.

10. The system of claim 9, wherein said affine parameter extraction means comprising affine parameter predict means, affine parameter increment means, and affine parameter adjusting means.

11. The system of claim 10, wherein said affine parameter predicting means comprising temporal Lie-derivative-memory means coupled with Lie-derivative-operator means for receiving and temporally preserving six HC-Lie-derivative signal, inner-producting means coupled with temporal time-derivative-memory means and temporal Lie-derivative-memory means for receiving said HC-time-derivative signal and said HC-Lie-derivative signal and producing affine parameter predict signal.

12. The system of claim 10, wherein said affine parameter increment means comprising vector-projecting means coupled with said vector weighted summing means and said temporal Lie-derivative-memory means for receiving vector weighted sum signal and HC-Lie-derivative signal and producing vector-projection signal, subtracting means coupled with said affine parameter predicting means and said vector-projecting means for receiving temporal affine parameter signal and vector-projection signal and producing increment signal.

13. The system of claim 10, wherein said parameter adjust means coupled with parameter predict means and parameter increment means for receiving predict affine parameter signal and increment signal and producing incremently adjusted affine parameter signal.

14. The system of claim 9 additionally comprising a temporal time-derivative-memory means coupled with HC-reference frame means for receiving and temporally preserving HC-time-derivative signal.

15. The system of claim 14 additionally comprising vector weighted summing means coupled with temporal Lie-derivative-memory means and affine parameter adjusting means for receiving six HC-Lie-derivative signals and six adjusted parameter signals and producing vector weighted sum signal.

16. In an artificial neural system including time differentiator means for receiving time-varying intensity image signal and providing time-derivative image signal, HC-reference frame means for receiving time-derivative image signal and providing HC-time-derivative signal, HC-Lie differentiation means for receiving intensity image signal and providing HC-Lie derivative signals, least squares error fitting means for receiving said HC-time-derivative signal and HC-Lie-derivative signals and providing affine flow signal, the method for determining the affine flow of the time-varying image comprising the steps of:

differentiating said time-varying intensity image by time-differentiation means;

extracting HC-time-derivative Ω

_t from said time-derivative image by using SO-simple cells in said HC-reference frame to obtain each said SO-simple cell response signals;

extracting HC-Lie-derivatives Ω

₁, Ω

₂, Ω

₃, Ω

₄, Ω

₅, Ω

₆ SO-Lie-germ response signals from said intensity image by using said SO-Lie germs of said Lie derivative operators to obtain each said SO-Lie-germ response signals;

applying said HC-time-derivative and said six HC-Lie-derivatives to respective least squares error fitting circuit to explicitly determine six affine flow component parameters ρ

₁, ρ

₂, ρ

₃, ρ

₄, ρ

₅, and ρ

₆.

17. The method of claim 16, wherein said least square error fitting circuit are configured to perform least squares error fit analysis to said HC-time-derivative Ω

_t and linear combination of said HC-Lie-derivatives for the following error function;

##EQU14## where ρ

=(ρ

₁, ρ

₂, ρ

₃, ρ

₄, ρ

₅, ρ

₆).

18. The method of claim 17, further comprising steps of determining predicted affine parameter by inner products of said HC-time derivative with said HC-Lie-derivatives, respectively:

space="preserve" listing-type="equation">ρ

.sub.i =Ω

.sub.i ·

Ω

.sub.t, i=1, . . . , 6.

19. The method of claim 18, further comprising the steps of operating said least square error fitting circuit to determine affine parameter increments Δ

ρ

_i, i=1, . . . , 6;

generating linear combination of HC-Lie derivatives using said predicted affine flow component parameters as coefficients ##EQU15## projecting said linear combination of HC-Lie derivatives to each HC-Lie-derivative ##EQU16## subtracting Ω

_t ·

Ω

_i from ##EQU17## i=1, . . . , 6, and taking the difference as affine parameter increments ##EQU18##

20. The method of claim 19, further comprising steps of using the said increments as correcting force to incremently adjust each of the six affine flow component parameters in accordance with error minimization gradient dynamical system technique to determine said six affine flow component parameters as signals along said parameter adjust means.