Optical flow-based manipulation of graphical objects
First Claim
1. A method of manipulating a graphical object presented on a display surface in response to a movement of one or more physical objects that are adjacent to the display surface, comprising the steps of:
- (a) creating a first image of the display surface in which one or more patches are included that represent one or more portions of the physical object at a first time index;
(b) creating a second image of the display surface in which the one or more patches are included and which represent the one or more portions of the physical object at a second time index occurring after the first time index;
(c) determining flow vectors describing a two-dimensional motion for at least some points in the one or more patches at the first time index and the second time index, the optical flow vectors defining an optical flow for the one or more portions of the one or more physical objects; and
(d) manipulating the graphical object by applying the optical flow to the graphical object presented on the display surface.
2 Assignments
0 Petitions
Accused Products
Abstract
Motion of one or more physical objects relative to a display surface of a display system is detected and an optical flow determined from the motion is used to manipulate a graphical object presented on the display surface. The one or more physical objects are detected in response to light reflected from the object(s) and received by a video camera. The optical flow is determined from the video camera image by identifying flow vectors for points in one or more patches included in the image that correspond to the physical objects. A proximity of a physical object to the display surface can be determined based on an intensity of light reflected from the physical object(s), or using a touch sensor such as a capacitance, pressure, or electromagnetic sensor or the like. Based on the optical flow, the graphical object can be translated, rotated, and/or scaled in size.
125 Citations
20 Claims
-
1. A method of manipulating a graphical object presented on a display surface in response to a movement of one or more physical objects that are adjacent to the display surface, comprising the steps of:
-
(a) creating a first image of the display surface in which one or more patches are included that represent one or more portions of the physical object at a first time index;
(b) creating a second image of the display surface in which the one or more patches are included and which represent the one or more portions of the physical object at a second time index occurring after the first time index;
(c) determining flow vectors describing a two-dimensional motion for at least some points in the one or more patches at the first time index and the second time index, the optical flow vectors defining an optical flow for the one or more portions of the one or more physical objects; and
(d) manipulating the graphical object by applying the optical flow to the graphical object presented on the display surface. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. Apparatus for detecting, applying, and displaying manipulation of a graphical object in response to movement of a physical object, comprising:
-
(a) a display surface adjacent to which the physical object can be positioned and moved and on which the graphical image is presented;
(b) a light source that emits light for illuminating a physical object disposed adjacent to the display surface;
(c) a light sensor disposed so as to sense light reflected back from the physical object, through the display surface;
(d) a processor in communication with the light sensor; and
(e) a memory in communication with the processor, the memory storing data and machine instructions that when executed by the processor cause a plurality of functions to be carried out, including;
(i) using the light sensor, creating a first image of the display surface in which one or more patches are included that represent one or more portions of the physical object at a first time index;
(ii) using the light sensor, creating a second image of the display surface in which the one or more patches are included and which represent the one or more portions of the physical object at a second time index occurring after the first time index;
(iii) determining flow vectors describing a two-dimensional motion for at least some points in the one or more patches at the first time index and the second time index, the optical flow vectors defining an optical flow for the one or more portions of the one or more physical objects; and
(iv) manipulating the graphical object by applying the optical flow to the graphical object presented on the display surface. - View Dependent Claims (11, 12, 13, 14, 15, 16)
-
-
17. A method for controlling a graphical object on a user interface based on an input derived from optical flow information for a physical object disposed adjacent to a display surface, comprising the steps of:
-
(a) executing computer-readable code to display a graphical object and to produce a first image of the display surface corresponding to a first time index, the first image including a patch representing a portion of a physical object adjacent to the display surface at the first time index;
(b) executing computer-readable code to produce a second image corresponding to a second time index, the second image including the patch at the second time index;
(d) executing computer-readable code to determine an optical flow describing a two-dimensional motion of points comprising the patch when steps (a) and (b) are determined to have occurred; and
(e) executing computer-readable code to apply the optical flow to manipulate the graphical object on the user interface. - View Dependent Claims (18, 19, 20)
-
Specification