Visual pairing in an interactive display system

• US 8,446,364 B2
• Filed: 02/27/2012
• Issued: 05/21/2013
• Est. Priority Date: 03/04/2011
• Status: Expired due to Fees

• Current Assignee: Interphase Corporation
• Original Assignee: Interphase Corporation
• Inventors: Solomon, Yoram
• Primary Examiner(s): Amadiz, Rodney
• Application Number:
  US13/406,143
• Publication Number:
  US 20120223883A1
• Time in Patent Office: 449 Days
• Field of Search: 345156-184, 178 1801- 1907, 713150-171
• US Class Current: 345/156
• CPC Class Codes: G06F 21/83 input devices, e.g. keyboards, mice or controllers thereof
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0383 Signal control means within the pointing device

Abstract

An interactive display system including a wireless pointing device including a camera or other image capture system. Pairing of the pointing device is performed by a computerized display system displaying a visual pairing code at its display, for capture by the pointing device. The pairing code may be displayed in a form that is human-readable, human-perceptible but not human-readable, or human-imperceptible. In response to detecting the pairing code, the pointing device transmits a wireless signal to the computerized display system including the pairing code. If the pairing code received by the display system matches that originally displayed, the computerized display system authorizes the pointing device as a source of control signals in the graphical user interface manner. Various additional handshaking approaches in combination with the visual pairing procedure are disclosed.

Claims

1. A method of pairing a remote pointing device with a display system for interactively controlling the operation of the display system, comprising the steps of:

displaying pairing code information at a graphics display;

capturing, at the pointing device, image data representative of at least a portion of the graphics display, the captured image data including the pairing code information;

transmitting, from the pointing device, a wireless pairing signal corresponding to the captured pairing code information;

receiving the wireless pairing signal at the display system;

responsive to the received wireless pairing signal matching the pairing code information displayed at the graphics display, authorizing wireless control signals transmitted by the pointing device to control the operation of the display system; and

then, responsive to wireless control signals from the pointing device, operating the display system to display visual information at the graphics display.

2. The method of claim 1, further comprising:

transmitting a wireless pairing request signal from the pointing device;

wherein the displaying step is performed by the display system responsive to receiving the wireless pairing request signal.

3. The method of claim 1, further comprising:

prior to the step of displaying pairing code information, transmitting a wireless beacon signal from the display system;

wherein the capturing step is performed by the pointing device responsive to receiving the wireless beacon signal.

4. The method of claim 1, further comprising:

prior to the step of displaying pairing code information, transmitting a wireless beacon signal from the display system; and

responsive to receiving the wireless beacon signal, transmitting a wireless pairing request signal from the pointing device;

wherein the displaying step is performed by the display system responsive to receiving the wireless pairing request signal.

5. The method of claim 1, further comprising:

not later than the step of displaying pairing code information, displaying a coded image indicating intent of the display system to pair with a pointing device.

6. The method of claim 5, further comprising:

capturing, at the pointing device, image data representative of at least a portion of the graphics display, the captured image data including the coded image indicating intent to pair;

responsive to capturing the coded image indicating intent by the display system to pair, transmitting a wireless pairing request signal from the pointing device;

wherein the step of displaying pairing code information is performed by the display system responsive to receiving the wireless pairing request signal.

7. The method of claim 1, wherein the wireless pairing signal further includes an identifier of the pointing device.

8. The method of claim 1, wherein the step of displaying pairing code information comprises:

generating visual payload image frame data;

combining, with the visual payload image frame data, at least one pairing code pattern corresponding to complementary intensity variances at one or more selected pixel locations of the visual payload image frame data, applied in successive frames; and

displaying the combined visual payload image frame data and pairing code pattern on the graphics display.

9. The method of claim 1, further comprising:

generating visual payload image frame data;

combining, with the visual payload image frame data, at least one positioning target pattern;

displaying the combined visual payload image frame data and positioning target pattern on the graphics display;

at the pointing device, capturing image data representative of at least a portion of the display including the positioning target;

determining a pointed-to location of the display responsive to the recovered positioning target pattern,

generating a wireless control signal from the pointing device corresponding to the pointed-to location of the display, responsive to receiving a user input at the pointing device.

10. A method of pairing a remote pointing device with a display system for interactively controlling the operation of the display system, comprising the steps of:

displaying pairing code information at a graphics display;

transmitting, from the display system, a wireless beacon signal including the pairing code information;

capturing, at the pointing device, image data representative of at least a portion of the graphics display, the captured image data including the pairing code information;

responsive to the pairing code information in the wireless beacon signal received at the pointing device matching the pairing code information included in the captured image data, transmitting a wireless pairing signal from the pointing device;

receiving the wireless pairing signal at the display system;

authorizing wireless control signals transmitted by the pointing device to control the operation of the display system; and

then, responsive to wireless control signals from the pointing device, operating the display system to display visual information at the graphics display.

11. The method of claim 10, wherein the steps of displaying pairing code information and transmitting the wireless beacon signal are performed repeatedly by the display system.

12. The method of claim 10, further comprising:

transmitting, from the pointing device, a wireless pairing request signal;

wherein the steps of displaying pairing code information and transmitting the wireless beacon signal are performed by the display system responsive to receiving the wireless pairing request signal.

13. The method of claim 10, further comprising:

receiving, at the display system, wireless beacon signals including pairing code information from other display systems;

then, at the display system, deriving a pairing code that differs from the pairing code information received from the other display systems;

wherein the step of transmitting the wireless beacon signal includes pairing code information corresponding to the derived pairing code.

14. The method of claim 10, wherein the wireless pairing signal further includes an identifier of the pointing device.

15. The method of claim 10, wherein the step of displaying pairing code information comprises:

generating visual payload image frame data;

combining, with the visual payload image frame data, at least one pairing code pattern corresponding to complementary intensity variances at one or more selected pixel locations of the visual payload image frame data, applied in successive frames; and

displaying the combined visual payload image frame data and pairing code pattern on the graphics display.

16. The method of claim 10, further comprising:

generating visual payload image frame data;

combining, with the visual payload image frame data, at least one positioning target pattern;

displaying the combined visual payload image frame data and positioning target pattern on the graphics display;

at the pointing device, capturing image data representative of at least a portion of the display including the positioning target;

determining a pointed-to location of the display responsive to the recovered positioning target pattern,

generating a wireless control signal from the pointing device corresponding to the pointed-to location of the display, responsive to receiving a user input at the pointing device.

17. An interactive display system, comprising:

a computerized display system, comprising;

a display;

a computer for generating image data to be displayed on the display, the image data including visual payload data and a pairing code; and

a receiver, for receiving wireless signals; and

a pointing device, comprising;

a housing;

a camera disposed in the housing;

image capture circuitry for capturing image data obtained by the camera;

pairing circuitry, coupled to the image capture circuitry for detecting pairing code information detected in the captured image data; and

a transmitter, coupled to the pairing circuitry, for transmitting wireless signals including pairing code information detected in the captured image data;

wherein the computer is programmed to respond to control signals transmitted by the pointing device responsive to receiving wireless signals from the pointing device including pairing code information matching the pairing code in the displayed image data.

18. The system of claim 17, wherein the transmitter of the pointing device is also for transmitting a wireless pairing request signal;

and wherein the computer is programmed to generate image data including the visual pairing code responsive to receiving the wireless pairing request signal.

19. The system of claim 18, wherein the computer is also for generating image data including a coded image indicating intent of the display system to pair with a pointing device,

and wherein the transmitter of the pointing device transmits the wireless pairing request signal responsive to the pairing circuitry detecting the coded image indicating intent to pair in the captured image data.

20. The system of claim 17, wherein the display system further comprises:

a transmitter for transmitting a wireless beacon signal;

wherein the pointing device further comprises;

a receiver for receiving wireless signals;

and wherein the transmitter of the pointing device transmits the wireless pairing request signal responsive to the receiver of the pointing device receiving the wireless beacon signal.

21. The system of claim 17, wherein the transmitter is also for transmitting an identifier of the pointing device in the transmitted wireless signal including pairing code information.

22. The system of claim 17, wherein the computer generates the pairing code in the form of a non-human-readable pattern.

23. The system of claim 22, wherein the computer generates the pairing code in the form of patterned modulation of light intensity of alternating polarity in alternating frames;

and wherein the pairing circuitry is for subtracting successive frames of captured image data to detect the pairing code.

24. The system of claim 17, wherein the computerized display system further comprises:

target generating circuitry for generating at least one positioning target pattern, the at least one positioning target pattern corresponding to complementary intensity variances to be applied to a portion of the payload image data in successive frames; and

graphics output circuitry for combining the visual payload data with the at least one positioning target pattern into graphics output signals in a format suitable for display;

and further comprising;

positioning circuitry for subtracting successive frames captured by the pointing device from a display coupled to the computer to recover image data including the target pattern, and for determining a location of the display at which the pointing device is aimed.

25. An interactive display system, comprising:

a computerized display system, comprising;

a display;

a computer for generating image data to be displayed on the display, the image data including visual payload data and pairing code information; and

wireless circuitry for transmitting and receiving wireless signals, at least some of the transmitted wireless signals including pairing code information; and

a pointing device, comprising;

a housing;

a camera disposed in the housing;

image capture circuitry for capturing image data obtained by the camera;

wireless circuitry for receiving and transmitting wireless signals;

pairing circuitry for comparing pairing code information detected in the captured image data with pairing code information detected in wireless signals received by the wireless circuitry of the pointing device; and

wherein the computer is programmed to respond to control signals transmitted by the pointing device, responsive to receiving a wireless signal from the pointing device indicating that the pairing circuitry has detected a match between the pairing code information in the displayed image data and pairing code information transmitted by the wireless circuitry of the display system.

26. The system of claim 25, wherein the wireless circuitry of the display system receives wireless signals including pairing codes transmitted by other interactive display systems;

wherein the computer is programmed to derive a pairing code that differs from the pairing codes in the wireless signals transmitted by other interactive display systems.