3D MAPPING WITH FLEXIBLE CAMERA RIG

• US 20150316767A1
• Filed: 05/01/2014
• Published: 11/05/2015
• Est. Priority Date: 05/01/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Ebstyne, Michael John, Schaffalitzky, Frederik, Steedly, Drew, Klein, Georg, Eade, Ethan, Grabner, Michael
• Granted Patent: US 9,759,918 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G06T 7/85 Stereo camera calibration

Abstract

Embodiments related to mapping an environment of a machine-vision system are disclosed. For example, one disclosed method includes acquiring image data resolving one or more reference features of an environment and computing a parameter value based on the image data, wherein the parameter value is responsive to physical deformation of the machine-vision system.

Claims

1. In a machine-vision system, a method to map an environment, the method comprising:

acquiring image data resolving one or more reference features of an environment; and

computing a parameter value based on the image data, the parameter value responsive to physical deformation of the machine-vision system.

2. The method of claim 1, further comprising assigning coordinates to one or more subject features in the environment based on the parameter value.

3. The method of claim 1, wherein the machine-vision system is a wearable system, the method further comprising assigning coordinates to one or more of the machine-vision system and a wearer thereof based on the parameter value.

4. The method of claim 1, wherein the machine-vision system is coupled to a near-eye display projector, the method further comprising assigning coordinates to a virtual display frame of the near-eye display projector based on the parameter value.

5. The method of claim 1, wherein the parameter value is one of a plurality of parameter values responsive to physical deformation of the machine vision system, which are computed based on the image data and used to assign coordinates to the one or more subject features.

6. The method of claim 5, wherein the parameter value is responsive to a twist of the machine-vision system about an axis of the machine-vision system.

7. The method of claim 5, wherein the parameter value is responsive to a bend of the machine vision system about a fixed point on the machine-vision system.

8. The method of claim 1, wherein the deformation reorients a field of view of one or more cameras of the machine-vision system.

9. The method of claim 1, wherein computing the parameter value includes locating the one or more reference features in concurrent image frames acquired by the machine-vision system.

10. The method of claim 1, wherein computing the parameter value includes locating the one or more reference features in image frames acquired a different times by the machine-vision system.

11. The method of claim 1, further comprising receiving output from an inertial measurement unit in the machine-vision system to determine an orientation of the machine-vision system.

17. The machine-vision system of claim 11, further comprising an inertial measurement unit responsive to an orientation of the machine-vision system and/or a change in position of the machine-vision system.

12. A machine-vision system comprising:

one or more cameras to be positioned on a wearer'"'"'s head and configured to acquire image data resolving one or more reference features of an environment;

a logic machine operatively coupled to an instruction-storage machine and configured to compute a parameter value based on the image data, the parameter value responsive to physical deformation of the machine-vision system, the logic machine further configured to assign coordinates to one or more subject features in the environment based on the parameter value.

13. The machine-vision system of claim 12, wherein the one or more cameras include cameras with overlapping fields of view.

14. The machine-vision system of claim 12, wherein the one or more cameras include cameras oriented for stereoscopic depth imaging.

15. The machine-vision system of claim 12, wherein the one or more cameras include a camera configured for time-of-flight depth sensing.

16. The machine-vision system of claim 12, further comprising a flexible, wearable rig to which the one or more cameras are fixed, and near-eye display componentry also fixed to the rig.

18. In a head-wearable machine-vision system, a method to map an environment, the method comprising:

acquiring image data resolving one or more reference features of an environment, the image data collected by one or more cameras arranged on a flexible rig;

computing a plurality of parameter values based on the image data, each parameter value responsive to an extent of deformation of the rig in a corresponding deformation mode; and

assigning coordinates to one or more subject features of the environment based on each of the parameter values;

presenting a virtual display image in display componentry coupled to the machine-vision system based coordinates assigned to the display componentry.

19. The method of claim 18, wherein computing the plurality of parameter values includes computing at regular intervals during acquisition of the image data.

20. The method of claim 18, wherein one or more of the parameter values is responsive to flexing of the rig in real time, and the assigned coordinates of the subject features are invariant to flexing of the rig in real time.