Image recording apparatus and method using light fields to track position and orientation

• US 6,342,917 B1
• Filed: 01/16/1998
• Issued: 01/29/2002
• Est. Priority Date: 01/16/1998
• Status: Expired due to Term

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Amenta, Annamaria B.
• Primary Examiner(s): HO, TUAN V
• Application Number:
  US09/007,776
• Time in Patent Office: 1,474 Days
• Field of Search: H04/N.5225, 348/207, 348/222, 348/232, 348/272, 348/169, 348/239, 348/294, 348/311, 348/231, 348/218, 348/36, 348/37, 348/38, 348/39, 382/154, 382/106, 382/141
• US Class Current: 348/207.1
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

The invention provides a simple method and apparatus for tracking image recording device motion using a light field. The invention locates the image recording device'"'"'s position and orientation in each frame very precisely by checking the radiance seen along lines captured in previous frames. The invention provides an interactive system that provides the operator with feedback, to capture a sequence of frames that sufficiently cover the light field, to provide sufficient data for reconstruction of three-dimensional structures.

Claims

1. An image recording apparatus comprising:

an image pickup device that acquires images of the surrounding environment, the images being a sequence of frames;

a frame storage that stores the sequence of frames acquired by the image pickup device; and

a processor that determines a position and an orientation of the image pickup device based on the sequence of frames, wherein

the frames comprise a plurality of lines and the processor determines the position of the image pickup device by correlating a radiance along at least one of the plurality of lines in a current frame with the radiance along corresponding ones of the plurality of lines of at least one previous'"'"'frame in the sequence of frames.

2. The image recording apparatus of claim 1, wherein the processor determines an initial light field from the sequence of frames stored in the frame storage.

3. The image recording apparatus of claim 2, wherein the processor correlates a radiance along at least one of the plurality of lines in a current frame with the radiance along corresponding ones of the plurality of lines of the initial light field to determine the position and the orientation of the image pickup device.

4. The image recording apparatus of claim 3, wherein, after the position and orientation of the image pickup device is determined, the processor adds the current frame to the initial light field.

5. The image recording apparatus of claim 4, wherein the processor adds the current frame to the initial light, field during times when the image pickup device is not acquiring frames.

6. The image recording apparatus of claim 1, wherein said image pickup device is a hand-held camera.

7. The image recording apparatus of claim 1, further comprising an interactive display that displays a current frame through the image pickup device and that provides instructions and information to an operator of the image recording apparatus.

8. The image recording apparatus of claim 7, wherein the interactive display provides instructions for positioning the image pickup device.

9. The image recording apparatus of claim 7, wherein the interactive display provides information about a path currently traversed by the operator.

10. The image recording apparatus of claim 7, wherein the image pickup, the frame storage, the processor and the interactive display are contained in a single housing.

11. The image recording apparatus of claim 7, wherein the image pickup, the frame storage, the processor and the interactive display are portions of a distributed computer system.

12. The image recording apparatus of claim 7, further comprising an audio output device that outputs audio information designating a path currently traversed by the operator.

13. The image recording apparatus of claim 1, further comprising an audio output device that outputs audio instructions for positioning the image pickup device.

14. The image recording apparatus of claim 1, further comprising an audio output device that outputs audio instructions and information to an operator of the image recording apparatus.

15. A method of determining a position and an orientation of an image recording apparatus, comprising:

recording a sequence of image frames;

determining the position and the orientation of the image recording apparatus for each frame of the sequence of frames;

providing registration feedback to an image recording device operator in real time;

recording a new image frame;

determining the position and orientation of the image recording apparatus for the new image frame based on the position and orientation of the image recording apparatus for frames in the sequence of image frames.

16. The method of claim 15, wherein determining the position and the orientation of the image recording apparatus for each frame of the sequence of frames comprises determining the position and the orientation of the image recording apparatus relative to fiducial points.

17. The method of claim 15, wherein determining the position and the orientation of the image recording apparatus for the new image frame comprises locating at least one line coincident in at least one image frame of the sequence of image frames and in the new image frame, the at least one located line having a radiance that is substantially identical in the at least one image frame of the sequence of image frames and in the new image frame.

18. The method of claim 15, further comprising determining an initial light field from the sequence of image frames.

19. The image recording apparatus of claim 18, wherein the step of determining the position and orientation of the image recording apparatus for a new image frame includes correlating a radiance along at least one line in the new image frame with the radiance along a corresponding line of the initial light field.

20. The image recording apparatus of claim 19, wherein, after the position and orientation of the image recording apparatus is determined, the new image frame is added to the initial light field.

21. The image recording apparatus of claim 20, wherein the new image frame is added to the initial light field during times when the image recording apparatus is not recording image frames.