Determining connectedness and offset of 3D objects relative to an interactive surface
First Claim
1. A method of detecting a three-dimensional object adjacent to a user-interactive side of a surface that optically diffuses light, as a function of an infrared light transmitted toward the three-dimensional object from an opposite side of the surface and reflected back through the surface from the three-dimensional object to be received by a light sensor disposed on the opposite side of the surface, comprising the steps of:
- (a) creating a first pixilated image representing the intensity of the infrared light reflected from the three-dimensional object and received by the light sensor;
(b) creating a first binarized image from the first pixilated image by filtering out pixels of the first pixilated image that do not have an intensity exceeding a first threshold value, the first binarized image representing a first planar distance of the three-dimensional object from the interactive side of the surface;
(c) creating a second binarized image from the first pixilated image by filtering out pixels of the first pixilated image that do not have an intensity exceeding a second threshold value, the second binarized image having substantially equal area and coordinate locations as the first binarized image, the second binarized image representing a second planar distance from the interactive side of the surface;
(d) detecting a first connected component in the first binarized image, the first connected component representing a first set of pixels that have an intensity exceeding the first threshold value and are immediately adjacent to each other, without an intervening region of pixels that do not have an intensity exceeding the first threshold value;
(e) determining a first bounding area and a first coordinate location of the first connected component in the first binarized image;
(f) detecting a second connected component in the second binarized image, the second connected component representing a second set of pixels that have an intensity exceeding the second threshold value and are immediately adjacent to each other, without an intervening region of pixels that do not have an intensity exceeding the second threshold value;
(g) determining a second bounding area and a second coordinate location of the second connected component in the second binarized image that has substantially equal area and coordinate locations as the first binarized image; and
(h) determining that both the first connected component and the second connected component correspond to the three-dimensional object adjacent to the interactive side of the surface, if one of the first connected component and the second connected component is disposed substantially within the bounding area of the other of the first connected component and the second connected component.
2 Assignments
0 Petitions
Accused Products
Abstract
A position of a three-dimensional (3D) object relative to a display surface of an interactive display system is detected based upon the intensity of infrared (IR) light reflected from the object and received by an IR video camera disposed under the display surface. As the object approaches the display surface, a “hover” connected component is defined by pixels in the image produced by the IR video camera that have an intensity greater than a predefined hover threshold and are immediately adjacent to another pixel also having an intensity greater than the hover threshold. When the object contacts the display surface, a “touch” connected component is defined by pixels in the image having an intensity greater than a touch threshold, which is greater than the hover threshold. Connected components determined for an object at different heights above the surface are associated with a common label if their bounding areas overlap.
-
Citations
20 Claims
-
1. A method of detecting a three-dimensional object adjacent to a user-interactive side of a surface that optically diffuses light, as a function of an infrared light transmitted toward the three-dimensional object from an opposite side of the surface and reflected back through the surface from the three-dimensional object to be received by a light sensor disposed on the opposite side of the surface, comprising the steps of:
-
(a) creating a first pixilated image representing the intensity of the infrared light reflected from the three-dimensional object and received by the light sensor;
(b) creating a first binarized image from the first pixilated image by filtering out pixels of the first pixilated image that do not have an intensity exceeding a first threshold value, the first binarized image representing a first planar distance of the three-dimensional object from the interactive side of the surface;
(c) creating a second binarized image from the first pixilated image by filtering out pixels of the first pixilated image that do not have an intensity exceeding a second threshold value, the second binarized image having substantially equal area and coordinate locations as the first binarized image, the second binarized image representing a second planar distance from the interactive side of the surface;
(d) detecting a first connected component in the first binarized image, the first connected component representing a first set of pixels that have an intensity exceeding the first threshold value and are immediately adjacent to each other, without an intervening region of pixels that do not have an intensity exceeding the first threshold value;
(e) determining a first bounding area and a first coordinate location of the first connected component in the first binarized image;
(f) detecting a second connected component in the second binarized image, the second connected component representing a second set of pixels that have an intensity exceeding the second threshold value and are immediately adjacent to each other, without an intervening region of pixels that do not have an intensity exceeding the second threshold value;
(g) determining a second bounding area and a second coordinate location of the second connected component in the second binarized image that has substantially equal area and coordinate locations as the first binarized image; and
(h) determining that both the first connected component and the second connected component correspond to the three-dimensional object adjacent to the interactive side of the surface, if one of the first connected component and the second connected component is disposed substantially within the bounding area of the other of the first connected component and the second connected component. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. A system for detecting a relative position of a three-dimensional object, the system comprising:
-
(a) a surface that diffuses light and has;
(i) an interactive side adjacent to which the three-dimensional object can be manipulated; and
(ii) an opposite side that is opposite the interactive side;
(b) a light source spaced away the opposite side of the surface, the light source emitting an infrared light that is transmitted through the surface to the interactive side of the surface;
(c) a light sensor disposed on the opposite side of the surface so as to sense infrared light reflected back from the three-dimensional object, through the surface;
(d) a processor in communication with the light sensor; and
(e) a memory in communication with the processor, the memory storing data and machine instructions that when executed by the processor cause a plurality of functions to be carried out, including;
(i) creating a first pixilated image representing the intensity of the infrared light reflected from the three-dimensional object and received by the light sensor;
(ii) creating a first binarized image from the first pixilated image by filtering out pixels of the first pixilated image that do not have an intensity exceeding a first threshold value, the first binarized image representing a first planar distance of the three-dimensional object from the interactive side of the surface;
(iii) creating a second binarized image from the first pixilated image by filtering out pixels of the first pixilated image that do not have an intensity exceeding a second threshold value, the second binarized image having substantially equal area and coordinate locations as the first binarized image, the second binarized image representing a second planar distance from the interactive side of the surface;
(iv) detecting a first connected component in the first binarized image, the first connected component representing a first set of pixels that have an intensity exceeding the first threshold value and are immediately adjacent to each other, without an intervening region of pixels that do not have an intensity exceeding the first threshold value;
(v) determining a first bounding area and a first coordinate location of the first connected component in the first binarized image;
(vi) detecting a second connected component in the second binarized image, the second connected component representing a second set of pixels that have an intensity exceeding the second threshold value and are immediately adjacent to each other, without an intervening region of pixels that do not have an intensity exceeding the second threshold value;
(vii) determining a second bounding area and a second coordinate location of the second connected component in the second binarized image that has substantially equal area and coordinate locations as the first binarized image; and
(viii) determining that both the first connected component and the second connected component correspond to the three-dimensional object adjacent to the interactive side of the surface, if one of the first connected component and the second connected components is disposed substantially within the bounding area of the other of the first connected component and the second connected component. - View Dependent Claims (12, 13, 14, 15, 16, 17, 18, 19, 20)
-
Specification