Real-time optical multiple object recognition and tracking system and method

• US 4,924,507 A
• Filed: 02/11/1988
• Issued: 05/08/1990
• Est. Priority Date: 02/11/1988
• Status: Expired due to Fees

• Current Assignee: United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration
• Original Assignee: United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration
• Inventors: Chao, Tien-Hsin, Liu, Hua-Kuang
• Primary Examiner(s): Boudreau, Leo H.
• Assistant Examiner(s): Mancuso, Joseph
• Application Number:
  US07/154,718
• Time in Patent Office: 817 Days
• Field of Search: 382/31, 382/30, 382/42, 382/48, 350/3.6, 350/3.66, 350/3.67, 350/3.72, 350/3.83, 350/3.84, 350/3.85, 350/3.75, 356/347, 365/125
• US Class Current: 382/211
• CPC Class Codes: G06V 10/88 Image or video recognition using optical means, e.g. reference filters, holographic masks, frequency domain filters or spatial domain filters

Abstract

System for optically recognizing and tracking a plurality of objects within a field of vision. Laser (46) produces a coherent beam (48). Beam splitter (24) splits the beam into object (26) and reference (28) beams. Beam expanders (50) and collimators (52) transform the beams (26, 28) into coherent collimated light beams (26'"'"', 28'"'"'). A two-dimensional SLM (54), disposed in the object beam (26'"'"'), modulates the object beam with optical information as a function of signals from a first camera (16) which develops X and Y signals reflecting the contents of its field of vision. A hololens (38), positioned in the object beam (26'"'"') subsequent to the modulator (54), focuses the object beam at a plurality of focal points (42). A planar transparency-forming film (32), disposed with the focal points on an exposable surface, forms a multiple position interference filter (62) upon exposure of the surface and development processing of the film (32). A reflector (53) directing the reference beam (28'"'"') onto the film (32), exposes the surface, with images focused by the hololens (38), to form interference patterns on the surface. There is apparatus (16'"'"', 64) for sensing and indicating light passage through respective ones of the positions of the filter (62), whereby recognition of objects corresponding to respective ones of the positions of the filter (62) is affected. For tracking, apparatus (64) focuses light passing through the filter (62) onto a matrix of CCD'"'"'s in a second camera (16'"'"') to form a two-dimensional display of the recognized objects.

Claims

1. Apparatus for optically simultaneously recognizing a plurality of objects within a field of vision comprising:

(a) a source of a coherent collimated light beam;

(b) beam splitter means for splitting said light beam into an object beam and a reference beam;

(c) two-dimensional spatial light modulation means disposed in said object beam for modulating said object beam with optical information as a function of signals connected to X and Y inputs thereof;

(d) television camera means for developing X and Y signals reflecting the contents of its field of vision, said X and Y signals being connected to said X and Y inputs of said two-dimensional spatial light modulation means;

(e) hololens means disposed in said object beam subsequent to said two-dimensional spatial light modulation means for focusing said object beam at a plurality of focal points;

(f) planar film means disposed with said focal points on an exposable surface thereof for forming a multiple position interference filter upon exposure of said exposable surface and subsequent development processing of said film means;

(g) means for directing said reference beam onto said exposable surface of said planar film means to expose said exposable surface in combination with images focused by said hololens means forming interference patterns on said exposable surface; and

,

(h) means for simultaneously sensing and indicating light passage through respective ones of the positions of said multiple position interference filter following development processing of said exposable surface whereby simultaneous recognition of objects corresponding to respective ones of said positions of said multiple position interference filter is affected.

2. The apparatus of claim 1 wherein:

said means for simultaneously sensing and indicating light passage through respective ones of the positions of said multiple position interference filter includes means for indicating light passage only if light passing therethrough is above a pre-established threshold level corresponding to object recognition.

3. The method of optically simultaneously recognizing individual pre-defined ones of a plurality of objects within a field of vision comprising the steps of:

(a) pre-defining the objects to be recognized by the steps of,

(a1) providing a controlled source of a coherent collimated light beam and means for splitting the light beam into an object beam and a reference beam;

(a2) disposing a two-dimensional spatial light modulator in the path of the object beam to modulate the object beam with optical information as a function of signals connected to X and Y inputs thereof;

(a3) providing a television camera to develop X and Y signals reflecting the contents of its field of vision;

(a4) connecting the X and Y signals to the X and Y inputs of the two-dimensional spatial light modulator;

(a5) disposing a hololens in the path of the object beam subsequent to the two-dimensional spatial light modulator to focus the object beam at a plurality of focal points in a plane;

(a6) disposing unexposed transparency-forming film with an exposable surface thereof in the plane of the focal points;

(a7) providing means for directing the reference beam onto the exposable surface of the film to expose the exposable surface in combination with images focused by the hololens forming interference patterns on the exposable surface;

(a8) placing an object to be recognized within the field of vision of the television camera;

(a9) covering the exposable surface of the film except for one focal point location thereof;

(a10) turning on the coherent collimated light beam to expose the exposable surface of the film at the one exposed focal point location;

(a11) repeating steps (a8) through (a10) with different objects and focal point locations to expose points of a multiple position interference filter;

(a12) development processing the film to form the multiple position interference filter; and

thereafter,

(b) subsequently simultaneously recognizing the objects by the steps of,

(b1) disposing the multiple position interference filter with the points thereof in the plane of the focal points;

(b2) operatively positioning means for simultaneously sensing and indicating light passage through respective ones of the positions of the multiple position interference filter;

(b3) directing the field of vision of the television camera on an area containing objects to be recognized;

(b4) turning on the coherent collimated light beam to create only the object beam with the contents of the field of vision of the television camera dynamically impressed on the object beam by the two-dimensional spatial light modulator; and

,

(b5) monitoring the simultaneous sensing and indicating means for indications of light passage through ones of the positions of the multiple position interference filter whereby simultaneous recognition of objects corresponding to respective ones of the pre-defined positions of the multiple position interference filter is affected.

4. The method of claim 3 wherein:

said step of monitoring the simultaneous sensing and indicating means for indications of light passage through ones of the positions of the multiple position interference filter includes the step of indicating light passage only if light passing through a position is above a pre-established threshold level corresponding to object recognition.

5. The method of claim 4 and additionally comprising the step for tracking the objects of:

focusing light passing through positions above said pre-established threshold level onto a charge coupled device to create a two-dimensional image of the field of view of the television camera and the recognized objects therein whereby movement of the objects can be seen and visually tracked.

6. Apparatus for optically simultaneously recognizing a plurality of objects within a field of vision comprising:

(a) a laser producing a coherent laser beam;

(b) beam expanders and collimators positioned to transform said object beam and said reference beam into coherent collimated light beams;

(c) a beam splitter disposed to split said laser beam into an object beam and a reference beam;

(d) a two-dimensional spatial light modulator disposed in said object beam to modulate said object beam with optical information as a function of signals connected to X and Y inputs thereof;

(e) a first television camera for developing X and Y signals reflecting the contents of its field of vision, said X and Y signals being connected to said X and Y inputs of said two-dimensional spatial light modulator;

(f) a hololens positioned in said object beam subsequent to said two-dimensional spatial light modulator to focus said object beam at a plurality of focal points;

(g) a planar transparency-forming film disposed with said focal points on an exposable surface thereof to form a multiple position interference filter upon exposure of said exposable surface and subsequent development processing of said film;

(h) means for directing said reference beam onto said exposable surface of said film to expose said exposable surface in combination with images focused by said hololens to form interference patterns on said exposable surface; and

,

(i) means for simultaneously sensing and indicating light passage through respective ones of the positions of said multiple position interference filter following development processing of said exposable surface whereby simultaneous recognition of objects corresponding to respective ones of said positions of said multiple position interference filter is affected.

7. The apparatus of claim 6 wherein:

said means for simultaneously sensing and indicating light passage through respective ones of the positions of said multiple position interference filter includes a second television camera containing a matrix of charge coupled devices and means for focusing light passing through said interference filter onto said matrix.

8. The apparatus of claim 7 wherein:

said means for simultaneously sensing and indicating light passage through respective ones of the positions of said multiple position interference filter employs outputs of the charge coupled devices to indicate light passage at each position of the interference filter and includes means for indicating light passage only if light passing through a position is above a pre-established threshold level corresponding to object recognition.

9. The method of optically recognizing individual pre-defined ones of a plurality of possible orientations of an object within a field of vision without the necessity of including a step for sequentially comparing to a series of pre-defined position definitions comprising the steps of:

(a) pre-defining the orientations of the object to be recognized by the steps of,

(a1) providing a controlled source of a coherent collimated light beam and means for splitting the light beam into an object beam and a reference beam;

(a2) disposing a two-dimensional spatial light modulator in the path of the object beam to modulate the object beam with optical information as a function of signals connected to X and Y inputs thereof;

(a3) providing a television camera to develop X and Y signals reflecting the contents of its field of vision;

(a4) connecting the X and Y signals to the X and Y inputs of the two-dimensional spatial light modulator;

(a5) disposing a hololens in the path of the object beam subsequent to the two-dimensional spatial light modulator to focus the object beam at a plurality of focal points in a plane;

(a6) disposing unexposed transparency-forming film with an exposable surface thereof in the plane of the focal points;

(a7) providing means for directing the reference beam onto the exposable surface of the film to expose the exposable surface in combination with images focused by the hololens forming interference patterns on the exposable surface;

(a8) placing the object within the field of vision of the television camera in a first orientation to be recognized;

(a9) covering the exposable surface of the film except for one focal point location thereof;

(a10) turning on the coherent collimated light beam to expose the exposable surface of the film at the one exposed focal point location;

(a11) repeating steps (a8) through (a10) with different orientations and focal point locations to expose points of a multiple position interference filter;

(a12) development processing the film to form the multiple position interference filter; and

thereafter,

(b) subsequently recognizing the orientation of the object by the steps of,

(b1) disposing the multiple position interference filter with the points thereof in the plane of the focal points;

(b2) operatively positioning means for sensing and indicating light passage through respective ones of the positions of the multiple position interference filter;

(b3) directing the field of vision of the television camera on an area containing the object;

(b4) turning on the coherent collimated light beam to create only the object beam with the contents of the field of vision of the television camera dynamically impressed on the object beam by the two-dimensional spatial light modulator; and

(b5) simultaneously monitoring said sensing and indicating means for indications of light passage through ones of the positions of the multiple position interference filter whereby instant recognition of the orientation of the object corresponding to respective ones of the pre-defined positions of the multiple position interference filter is affected.

10. The method of claim 9 wherein:

said step of simultaneously monitoring said sensing and indicating means for indications of light passage through ones of the positions of the multiple position interference filter includes the step of indicating light passage only if light passing through a position is above a pre-established threshold level corresponding to object orientation recognition.

11. The method of claim 10 and additionally comprising the step for tracking the object of:

focusing light passing through a position above said pre-established threshold level onto a charge coupled device to create a two-dimensional image of the field of view of the television camera and the object therein whereby movement of the object can be seen and visually tracked.

12. Apparatus for optically simultaneously recognizing a plurality of objects within a field of vision comprising:

(a) a laser producing a coherent laser beam;

(b) a beam splitter disposed to split said laser beam into an object beam and a reference beam;

(c) beam expanders and collimators positioned to transform said object beam and said reference beam into coherent collimated light beams;

(d) a two-dimensional spatial light modulator disposed in said object beam to modulate said object beam with optical information as a function of signals connected to X and Y inputs thereof;

(e) a first television camera for developing X and Y signals reflecting the contents of its field of vision, said X and Y signals being connected to said X and Y inputs of said two-dimensional spatial light modulator;

(f) a hololens positioned in said object beam subsequent to said two-dimensional spatial light modulator to focus said object beam at a plurality of focal points;

(g) a planar transparency-forming film disposed with said focal points on an exposable surface thereof to form a multiple position interference filter upon exposure of said exposable surface and subsequent development processing of said film;

(h) means for directing said reference beam onto said exposable surface of said film to expose said exposable surface in combination with images focused by said hololens to form interference patterns on said exposable surface; and

,

(i) means for simultaneously sensing indicating light passage through respective ones of the positions of said multiple position interference filter following development processing of said exposable surface whereby simultaneously recognition of objects corresponding to respective ones of said positions of said multiple position interference filter is affected.

13. The apparatus of claim 12 wherein:

said means for simultaneously sensing and indicating light passage through respective ones of the positions of said multiple position interference filter includes a second television camera containing a matrix of charge coupled devices and means for focusing light passing through said interference filter onto said matrix.

14. The apparatus of claim 13 wherein:

said means for simultaneously sensing and indicating light passage through respective ones of the positions of said multiple position interference filter employs outputs of the charge coupled devices to indicate light passage at each position of the interference filter and includes means for indicating light passage only if light passing through a position is above a pre-established threshold level corresponding to object recognition.

15. In Vander Lugt filter type apparatus for optically recognizing an object within a field of vision including a source of a coherent light beam;

a beam splitter disposed to split the light beam into an object beam and a reference beam;

beam expanders and collimators positioned to transform the object beam and the reference beam into coherent collimated light beams;

a transparency-forming film disposed form an interference filter upon exposure of said exposable surface and subsequent development processing of the film;

means for directing the reference beam onto the exposable surface of the film to expose the exposable surface in combination with an image focused to form an interference pattern on the exposable surface; and

, means for sensing and indicating light passage through the interference pattern position of the interference filter following development processing of the exposable surface whereby recognition of an object corresponding to the one used to create the interference filter is affected, the improvement to allow simultaneous recognition of multiple objects within the field of vision comprising;

(a) two-dimensional spatial light modulator disposed in the object beam to modulate the object beam with optical information as a function of signals connected to X and Y inputs thereof;

(b) a first television camera for developing X and Y signals reflecting the contents of its field of vision, said X and Y signals being connected to said X and Y inputs of said two-dimensional spatial light modulator;

(c) a hololens positioned in the object beam subsequent to said two-dimensional spatial light modulator to focus the object beam at a plurality of focal points;

(d) means for positioning the transparency-forming film with said focal points on an exposable surface thereof to form a multiple position interference filter upon exposure of said exposable surface and subsequent development processing of said film;

(e) the means for directing said reference beam onto said exposable surface of said film including means to expose said exposable surface in combination with multiple images focused by said hololens to form a plurality of interference patterns on said exposable surface; and

,

(f) the means for sensing and indicating light passage including means for simultaneously sensing and indicating light passage through respective ones of the positions of said multiple position interference filter following development processing of said exposable surface whereby simultaneous recognition of objects corresponding to respective ones of said positions of said multiple position interference filter is affected.

16. The improvement of claim 15 wherein:

said means for simultaneously sensing and indicating light passage through respective ones of the positions of said multiple position interference filter includes a second television camera containing a matrix of charge coupled devices and means for focusing light passing through said interference filter onto said matrix whereby light passage through individual positions of said interference filter can be determined by the presence of light at corresponding positions of said matrix.

17. The apparatus of claim 16 wherein:

said means for simultaneously sensing indicating light passage through respective ones of the positions of said multiple position interference filter employs outputs of the charge coupled devices to indicate light passage at each position of the interference filter and includes means for indicating light passage only if light passing through a position is above a pre-established threshold level corresponding to object recognition.

18. The improvement of claim 15 wherein:

said means for simultaneously sensing and indicating light passage through respective ones of the positions of said multiple position interference filter includes a second television camera containing a matrix of charge coupled devices and means for focusing light passing through at least one position of said interference filter onto said matrix to create a two-dimensional image of the field of view of the first television camera and the objects therein for use in visually tracking movement of the objects.