Methods for enhancing performance and data acquired from three-dimensional image systems

• US 6,512,838 B1
• Filed: 10/05/2000
• Issued: 01/28/2003
• Est. Priority Date: 09/22/1999
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Canesta Incorporated (Microsoft Corporation)
• Inventors: Bamji, Cyrus, Rafii, Abbas, Torunoglu, Iihami, Sze, Cheng-Feng
• Primary Examiner(s): Mancuso, Joseph
• Assistant Examiner(s): MILLER, MARTIN E
• Application Number:
  US09/684,368
• Time in Patent Office: 845 Days
• Field of Search: 356/5.01, 356/622, 356/623, 382/106, 382/276, 382/166, 382/293, 345/168, 345/173
• US Class Current: 382/106
• CPC Class Codes: G01C 3/08 Use of electric radiation detectors
• CPC Class Codes: G01S 17/10 using transmission of interrupted, pulse-modulated waves determination of distance by phase measurements G01S17/32
• CPC Class Codes: G01S 17/14 wherein a voltage or current pulse is initiated and terminated in accordance with the pulse transmission and echo reception respectively, e.g. using counters
• CPC Class Codes: G01S 17/46 Indirect determination of position data
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 7/4802 using analysis of echo signal for target characterisation; Target signature; Target cross-section
• CPC Class Codes: G01S 7/486 Receivers
• CPC Class Codes: G01S 7/497 Means for monitoring or calibrating
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1632 External expansion units, e.g. docking stations
• CPC Class Codes: G06F 1/1673 Arrangements for projecting a virtual keyboard
• CPC Class Codes: G06F 3/0221 Arrangements for reducing keyboard size for transport or storage, e.g. foldable keyboards, keyboards with collapsible keys G06F3/0216 takes precedence; for keyboards integrated in a laptop computer G06F1/1666
• CPC Class Codes: G06F 3/0426 tracking fingers with respect to a virtual keyboard projected or printed on the surface virtual keyboards on touch screens G06F3/04886
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06V 30/228 of three-dimensional handwriting, e.g. writing in the air
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: H04N 25/46 by combining or binning pixels
• CPC Class Codes: H04N 3/155 Control of the image-sensor operation, e.g. image processing within the image-sensor

Abstract

A three-dimension distance time-of-flight system is disclosed in which distance values are acquired by a plurality of sensors independently from each other. For use with this and similar systems, Z-distance accuracy and resolution are enhanced using various techniques including over-sampling acquired sensor data and forming running averages, or forming moving averages. Acquired data may be rejected if it fails to meet criteria associated with distance, luminosity, velocity, or estimated shape information reported by neighboring sensors. A sub-target having at least one pre-calibrated reflectance zone is used to improve system measurement accuracy. Elliptical error is corrected for using a disclosed method, and reversible mapping of Z-values into RGB is provided.

Claims

1. For use with a system that acquires distance and brightness measurements using energy transmitted from an emitter at a first location on a plane, said energy reflecting at least in part from a target object and being detected by independent sensors defining a sensor array on said plane but spaced-apart from said first location, a method of improving actual (x,y,z) measurement by correcting elliptical error of a system including said energy transmitter and sensor array, the method comprising the following steps:

(a) defining a spherical coordinate for said sensors in said sensor array, and constructing a look-up table containing spherical coordinates for each of said sensors, where a spherical coordinate for sensor P_ij is given by (p_ij,−

a_ij,−

b_ij);

(b) defining a spatial coordinate of said emitter from a spherical coordinate system defined in step (a) such that said spatial coordinate of said emitter has a same origin as said spherical coordinate system, where said spatial coordinate of said emitter is (C_x, C_y, C_z) and defining a constant k as square of distance between said emitter and said origin of said spherical coordinate system;

(c) For each sensor P_ij, constructing a look-up table of a constant h_ij where h_ij is calculated as follows;

 

h_ij=2(p_ij+C_x cos(a_ij)sin(b_ij)+C_y sin(a_ij)sin(b_ij)+C_zcos(b_ij))

where (p_ij, −

a_ij, −

b_ij) is the spherical coordinate of said sensor P_ij defined at step (a), and where (C_x, C_y, C_z) is the spatial Cartesian coordinate of said emitter defined at step (b);

(d) identifying sensors {circumflex over (P)}_ij that actually detect energy reflected from said target object;

(e) using measured rountrip distance d_ij from said emitter to sensors {circumflex over (P)}_ij identified at step (d) to calculate distance r_ij from an imaging portion of said target object and said sensor {circumflex over (P)}_ij according to r_ij=((d_ij−

p_ij)²−

k)/(2d_ij−

h_ij), using values of k determined at step (b), values of h_ij determined in step (c), and {circumflex over (p)}_ij determined in step (a); and

(f) calculating actual Cartesian coordinate (A_x, A_y, A_z) of imaging portion of said target objection according to A_x=r_ij cos(â

_ij)sin({circumflex over (b)}_ij), A_x=r_ij sin(â

_ij)sin({circumflex over (b)}_ij), and A_x=r_ij cos({circumflex over (b)}_ij), where r_ij is obtained in step (e) and â

_ij, {circumflex over (b)}_ij of sensor {circumflex over (P)}_ij are obtained in step (a).

2. A computer-readable storage medium whereon is located a computer program that causes a computer sub-system having at least a processor unit to improve actual (x,y,z) measurements by correcting elliptical error in a system that uses energy transmitted from an emitter at a first location on a plane, said energy reflecting at least in part from a target object and being detected by independent sensors defining a sensor array on said plane but spaced-apart from said first location, the computer program upon execution carrying out the following steps:

(a) defining a spherical coordinate for said sensors in said sensor array, and constructing a look-up table containing spherical coordinates for each of said sensors, where a spherical coordinate for sensor P_ij is given by (p_ij,−

a_ij,−

b_ij);

(b) defining a spatial coordinate of said emitter from a spherical coordinate system defined in step (a) such that said spatial coordinate of said emitter has a same origin as said spherical coordinate system, where said spatial coordinate of said emitter is (C_x, C_y, C_z) and defining a constant k as square of distance between said emitter and said origin of said spherical coordinate system;

(c) For each sensor P_ij, constructing a look-up table of a constant h_ij where h_ij is calculated as follows;