Motion compensation in an interactive display system
First Claim
1. A method of operating a computer system including a handheld human interface device, comprising:
- generating visual payload image frame data;
combining, with the visual payload image frame data, at least one positioning target pattern corresponding to complementary intensity variances at one or more selected pixel locations of the visual payload image frame data, applied in first and second consecutive frames;
displaying the combined visual payload image frame data and positioning target pattern on a display;
capturing, at the handheld human interface device, image data representative of at least a portion of the display including the positioning target, from each of the first and second frames;
detecting relative motion of the device between the times at which image data is captured from the first and second frames, based on measurements acquired from one or more motion sensors in the device;
aligning the image data of the first and second frames based on the detected relative motion;
subtracting the aligned image data from the first and second frames from one another to recover the positioning target pattern as viewed at the device; and
determining a location of the display at which the device is pointing at the time of one of the first and second frames, responsive to the recovered positioning target pattern.
1 Assignment
0 Petitions
Accused Products
Abstract
An interactive display system including a wireless pointing device, and positioning circuitry capable of determining absolute and relative positions of the display at which the pointing device is aimed. The pointing device captures images displayed by the computer, including one or more human-imperceptible positioning targets. The positioning targets are presented as patterned modulation of the intensity (e.g., variation in pixel intensity) in a display frame of the visual payload, followed by the opposite modulation in a successive frame. At least two captured image frames are subtracted from one another to recover the positioning target in the captured visual data and to remove the displayed image payload. Motion of the pointing device between the two frames is detected by relative motion sensors, and used to align the positions of the positioning targets in the captured images for those frames to improve the fidelity of the recovered positioning target following the subtraction.
-
Citations
20 Claims
-
1. A method of operating a computer system including a handheld human interface device, comprising:
-
generating visual payload image frame data; combining, with the visual payload image frame data, at least one positioning target pattern corresponding to complementary intensity variances at one or more selected pixel locations of the visual payload image frame data, applied in first and second consecutive frames; displaying the combined visual payload image frame data and positioning target pattern on a display; capturing, at the handheld human interface device, image data representative of at least a portion of the display including the positioning target, from each of the first and second frames; detecting relative motion of the device between the times at which image data is captured from the first and second frames, based on measurements acquired from one or more motion sensors in the device; aligning the image data of the first and second frames based on the detected relative motion; subtracting the aligned image data from the first and second frames from one another to recover the positioning target pattern as viewed at the device; and determining a location of the display at which the device is pointing at the time of one of the first and second frames, responsive to the recovered positioning target pattern. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. An interactive display system, comprising:
-
a computer for generating display image data to be displayed on a display; graphics output circuitry for generating graphics output signals corresponding to the display image data in a format suitable for display; a pointing device, comprising; a hand-held housing; an image sensor disposed in the housing; one or more motion sensors; and image capture circuitry for capturing image data obtained by the image sensor; and positioning circuitry for determining a location at the display at which the pointing device is aimed by performing a plurality of operations comprising; generating visual payload image frame data; combining, with the visual payload image frame data, at least one positioning target pattern corresponding to complementary intensity variances at one or more selected pixel locations of the visual payload image frame data, applied in first and second consecutive frames; displaying the combined visual payload image frame data and positioning target pattern on a display; capturing, at the handheld human interface device, image data representative of at least a portion of the display including the positioning target, from each of the first and second frames; detecting relative motion of the device between the times at which image data is captured from the first and second frames, based on measurements acquired from one or more motion sensors in the device; aligning the image data of the first and second frames based on the detected relative motion; subtracting the aligned image data from the first and second frames from one another to recover the positioning target pattern as viewed at the device; and determining a location of the display at which the device is pointing at the time of one of the first and second frames, responsive to the recovered positioning target pattern. - View Dependent Claims (12, 13, 14, 15, 16, 17, 18, 19, 20)
-
Specification