Gantry observation feedback controller

• US 9,160,904 B1
• Filed: 09/12/2012
• Issued: 10/13/2015
• Est. Priority Date: 09/12/2012
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Chang, Samuel Henry, Zhu, Yuhua, Krishnasamy, Prasanna Venkatesh
• Primary Examiner(s): Velez, Roberto
• Assistant Examiner(s): COLEMAN, STEPHEN P
• Application Number:
  US13/612,113
• Time in Patent Office: 1,126 Days
• Field of Search: 348/207.99, 356/139.03
• US Class Current: 1/1
• CPC Class Codes: F16M 11/10 around a horizontal axis F16M11/12 takes precedence
• CPC Class Codes: F16M 11/12 in more than one direction
• CPC Class Codes: F16M 11/18 with mechanism for moving the apparatus relatively to the stand
• CPC Class Codes: F16M 11/2014 around a vertical axis F16M11/2035 takes precedence
• CPC Class Codes: F16M 11/425 along guiding means
• CPC Class Codes: F16M 13/02 for supporting on, or attaching to, an object, e.g. tree, gate, window-frame, cycle
• CPC Class Codes: F16M 13/027 Ceiling supports
• CPC Class Codes: G06F 3/00 Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 2207/30208 Marker matrix
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: H04N 13/20 Image signal generators
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 9/3141 Constructional details thereof details not peculiar to the presence of an electronic spatial light modulator G03B21/14
• CPC Class Codes: H04N 9/3194 including sensor feedback

Abstract

A calibration module may acquire calibration data for calibrating rotations of augmented reality functional node. The calibration module scans a target, records pose data for the augmented reality functional node, and captures images of the scanned target. From these acquired images, calibration may be determined. Once determined, the calibration may be used to determine a rotator control model.

Claims

1. A system comprising:

a platform having at least two rotational degrees of freedom and carrying an image sensor and a light projector, the image sensor having an associated field of view and configured to capture one or more images, the light projector having an associated optical axis and configured to illuminate at least a portion of a target located in the field of view of the image sensor, wherein the field of view of the image sensor and the optical axis of the light projector move in accordance with rotations of the platform;

one or more processors; and

one or more non-transitory computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform acts comprising;

positioning the platform in multiple positions relative to the target to illuminate a sequence of target locations by the light projector;

capturing, by the image sensor, at least one image of the target, the at least one image including information regarding the sequence of target locations on the target that were illuminated by the light projector;

for individual positions of the multiple positions, determining pose data for the platform, the pose data being indicative of rotational values for the two rotational degrees of freedom of the platform while the platform is at the respective position;

determining a correlation between the sequence of target locations and the pose data for the multiple positions; and

using the correlation to control rotations of the platform about the two rotational degrees of freedom.

2. The system of claim 1, the acts further comprising:

illuminating the sequence of target locations with light emitted from the light projector; and

for each position of the platform and target location of the sequence, recording corresponding pose data for the platform.

3. The system of claim 1, the acts further comprising:

identifying an illuminated target location from the at least one image;

identifying an aiming location on the target corresponding to the illuminated target location based at least in part on the pose data of the platform;

determining a disparity between the aiming location and the illuminated target location.

4. The system of claim 3, wherein the illuminated target location is a first illuminated target location, the aiming location is a first aiming location, the disparity is a first disparity, and the acts further comprise:

determining whether the first disparity is greater than a threshold amount;

responsive to the first disparity being greater than the threshold amount;

identifying a second illuminated target location from the at least one image;

identifying a second aiming location on the target corresponding to the second illuminated target location based at least in part on the pose data of the platform; and

determining a second disparity between the second aiming location and the second illuminated target location.

5. The system of claim 1, wherein the platform is a gantry platform having a base, a first member rotatably coupled to the base and a second member rotatably coupled to the first member, wherein the first member rotates about a first axis corresponding to a first degree of freedom of that at least two degrees of freedom of the platform, and wherein the second member rotates about a second axis corresponding to a second degree of freedom of that at least two degrees of freedom of the platform.

6. The system of claim 5, further comprising:

a pan motor that drives the first member in a yaw motion; and

a tilt motor that drives the second member in a pitch motion.

7. The system of claim 1, the acts further comprising:

identifying a plurality of illuminated target locations from the at least one image;

for each illuminated target location of the plurality of target locations, identifying a corresponding aiming location on the target based at least in part on the pose data of the platform; and

for each illuminated target location of the plurality of target locations, determining a disparity between the aiming location and the illuminated target location.

8. The system of claim 7, wherein the pose data for the platform comprises a plurality of input vectors, each input vector corresponding to an aiming target location, and wherein correlating the sequence of target locations with the pose data comprises determining a best fit for the plurality of input vectors and the disparities between the plurality of illuminated target locations and the corresponding aiming target locations.

9. The system of claim 7, wherein the correlation is a first correlation, the sequence of target locations on the target is a first sequence of target locations, and the pose data is first pose data, and the acts further comprise:

positioning the platform in multiple additional positions relative to the target to illuminate a second sequence of target locations by the light projector;

acquiring at least one additional image, captured by the image sensor, of the target, the at least one additional image including information regarding the second sequence of target locations on the target;

for each of the multiple additional positions, determining pose data for the platform;

determining a second correlation between the second sequence of target locations and the pose data for the multiple additional positions, and

using the second correlation to control rotations of the platform about the two degrees of freedom.

10. A method comprising:

providing control signals to position a platform in multiple positions, wherein the platform has at least two rotational axes and carries an image sensor having an associated field of view of a target and a light projector having an associated optical axis, and wherein the field of view of the image sensor and the optical axis of the light projector move in accordance with rotations of the platform;

capturing, by the image sensor, at least one image of the target, the at least one image including information regarding a sequence of target locations on the target that were illuminated by the light projector at the multiple positions;

for individual positions of the multiple positions, acquiring pose data for the platform comprising rotational values for the two rotational degrees of freedom of the platform while the platform is positioned at the respective position;

determining a correlation between the sequence of target locations and the pose data for the multiple positions; and

using the correlation to control rotations of the platform about the two rotational degrees of freedom.

11. The method of claim 10, further comprising:

illuminating the sequence of target locations with light emitted from the light projector; and

for each position of the platform and target location of the sequence, recording corresponding pose data for the platform.

12. The method of claim 10, wherein the correlating the sequence of target locations with the pose data comprises:

identifying an illuminated target location from the at least one image;

identifying an aiming location on the target corresponding to the illuminated target location based at least in part on the pose data of the platform;

determining a disparity between the aiming location and the illuminated target location.

13. The method of claim 12, wherein the illuminated target location is a first illuminated target location, the aiming location is a first aiming location, the disparity is a first disparity, and further comprising:

determining whether the first disparity is greater than a threshold amount;

responsive to the first disparity being greater than the threshold amount;

identifying a second illuminated target location from the at least one image;

identifying a second aiming location on the target corresponding to the second illuminated target location based at least in part on the pose data of the platform;

determining a second disparity between the second aiming location and the second illuminated target location; and

correlating the sequence of target locations with the pose data based at least in part on the second disparity.

14. The method of claim 10, wherein the correlating the sequence of target locations with the pose data comprises:

identifying a plurality of illuminated target locations from the at least one image;

for each illuminated target location of the plurality of target locations, identifying a corresponding aiming location on the target based at least in part on the pose data of the platform;

for each illuminated target location of the plurality of target locations, determining a disparity between the aiming location and the illuminated target location.

15. The method of claim 14, wherein the pose data for the platform comprises a plurality of input vectors, each input vector corresponding to an aiming target location, and wherein the correlating the sequence of target locations with the pose data comprises determining a best fit for the plurality of input vectors and the disparities between the plurality of illuminated target locations and the corresponding aiming target locations.

16. The method of claim 14, wherein the correlation is a first correlation, the sequence of target locations on the target is a first sequence of target locations, and further comprising:

positioning the platform in multiple additional positions relative to the target to illuminate a second sequence of target locations by the light projector;

acquiring at least one additional image, captured by the image sensor, of the target, the at least one additional image including information regarding a second sequence of target locations on the target;

for each of the multiple additional positions, determining pose data for the platform;

determining a second correlation between the second sequence of target locations and the second pose data for the multiple additional positions; and

using the second correlation to control rotations of the platform about the two degrees of freedom.

17. One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform acts comprising:

causing a platform to be positioned in multiple positions relative to a target to illuminate a sequence of target locations by a light projector;

capturing, by an image sensor having an associated field of view, at least one image of the target, the at least one image including information regarding a sequence of target locations on the target that were illuminated by the light projector having an associated optical axis, wherein the platform has at least two rotational degrees of freedom and carries the image sensor and the light projector, and wherein the field of view of the image sensor and the optical axis of the light projector move in accordance with rotations of the platform;

for individual positions of the multiple positions, determining pose data for the platform comprising rotational values for the two rotational degrees of freedom of the platform while the platform is rotated about at least one of the two rotational degrees of freedom to illuminate a target location of the sequence of target locations;

determining a correlation between the sequence of target locations and the pose data for the multiple positions; and

using the correlation to control rotations of the platform about the two rotational degrees of freedom.

18. The one or more non-transitory computer-readable media of claim 17, the acts further comprising:

illuminating the sequence of target locations with light emitted from the light projector; and

for each position of the platform and target location of the sequence, recording corresponding pose data for the platform.

19. The one or more non-transitory computer-readable media of claim 17, the acts further comprising:

identifying an illuminated target location from the at least one image;

identifying an aiming location on the target corresponding to the illuminated target location based at least in part on the pose data of the platform; and

determining a disparity between the aiming location and the illuminated target location, wherein the correlation is based at least in part on the disparity.

20. The one or more non-transitory computer-readable media of claim 19, wherein the illuminated target location is a first illuminated target location, the aiming location is a first aiming location, the disparity is a first disparity, and the acts further comprise:

determining whether the first disparity is greater than a threshold amount;

responsive to the first disparity being greater than the threshold amount;

identifying a second illuminated target location from the at least one image;

identifying a second aiming location on the target corresponding to the second illuminated target location based at least in part on the pose data of the platform;

determining a second disparity between the second aiming location and the second illuminated target location; and

calibrating rotations of the platform based at least in part on the second disparity.

21. The one or more non-transitory computer-readable media of claim 17, the acts further comprising:

identifying a plurality of illuminated target locations from the at least one image;

for each illuminated target location of the plurality of target locations, identifying a corresponding aiming location on the target based at least in part on the pose data of the platform;

for each illuminated target location of the plurality of target locations, determining a disparity between the aiming location and the illuminated target location; and

calibrating rotations of the platform based at least in part on the disparities between the plurality of illuminated target locations and the corresponding aiming target locations.

22. The one or more non-transitory computer-readable media of claim 21, wherein the pose data for the platform comprises a plurality of input vectors, each input vector corresponding to an aiming target location, and wherein the correlating the sequence of target locations with the pose data comprises determining a best fit for the plurality of input vectors and the disparities between the plurality of illuminated target locations and the corresponding aiming target locations.

23. The one or more non-transitory computer-readable media of claim 21, wherein the correlation is a first correlation, the sequence of target locations on the target is a first sequence of target locations, and the pose data is first pose data, and the acts further comprise:

positioning the platform in multiple additional positions relative to the target to illuminate a second sequence of target locations by the light projector;

acquiring at least one additional image, captured by the image sensor, of the target, the at least one additional image including information regarding the second sequence of target locations on the target;

for each of the multiple additional positions, determining pose data for the platform;

determining a second correlation between the second sequence of target locations and the second pose data for the multiple additional positions; and

using the second correlation to control rotations of the platform about the two degrees of freedom.