User input system for immersive interaction

• US 10,001,833 B2
• Filed: 10/12/2016
• Issued: 06/19/2018
• Est. Priority Date: 08/14/2012
• Status: Active Grant

• Current Assignee: Position Imaging, Inc.
• Original Assignee: Position Imaging, Inc.
• Inventors: Hill, Edward L.
• Primary Examiner(s): POLO, GUSTAVO D
• Application Number:
  US15/291,336
• Publication Number:
  US 20170031432A1
• Time in Patent Office: 615 Days
• Field of Search: 345156
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/23 for interfacing with the game device, e.g. specific interfaces between game controller and console
• CPC Class Codes: A63F 2300/1025 details of the interface with the game device, e.g. USB version detection
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 1/1698 the I/O peripheral being a sending/receiving arrangement to establish a cordless communication link, e.g. radio or infrared link, integrated cellular phone details of antennas disposed inside a computer H01Q1/2266
• CPC Class Codes: G06F 3/01 Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Abstract

A system and method for providing immersive interaction between users and interactive software programs comprise receiving radio frequency (RF) signals, transmitted by an RF-transmitting device, at three or more spatially separated antennas. Time of arrival information is acquired from the RF signals. A position of the RF-transmitting device is computed from the time of arrival information acquired from the RF signals. The position of the RF-transmitting device is registered within a virtual environment produced by an interactive software program (which can execute on a mobile device). The virtual environment, as affected by the registered position of the RF-transmitting device, is transmitted to a display apparatus for display.

Claims

1. A user input system comprising:

an input interface connected to three or more spatially separated antennas to acquire therefrom radio signals transmitted by an RF-transmitting device operated by a user;

a first interface in wireless communication with a mobile device executing at least a portion of an interactive software program on the mobile device that, when executed, displays a virtual environment on a display screen;

a second interface in communication with the display screen; and

a processor programmed to calculate a position of the RF-transmitting device from timing information determined from the radio signals received through the input interface and to register the position of the RF-transmitting device within the interactive software program as user input by which the user interacts with and navigates the virtual environment displayed by the interactive software program on the display screen,

wherein the first interface is configured to receive the virtual environment, as affected by the registered position of the RF-transmitting device, from the mobile device, and wherein the second interface is configured to transmit the virtual environment, as affected by the registered position of the RF-transmitting device, to a device having the display screen after the virtual environment, as affected by the registered position of the RF-transmitting device, has been received from the mobile device by the first interface.

2. The user input system of claim 1, wherein the processor is further programmed to run at least a portion of the interactive software program.

3. The user input system of claim 1, wherein the mobile device comprises the RF-transmitting device whose position is calculated and registered.

4. The user input system of claim 1, wherein the processor is further programmed to determine an orientation of the mobile device from inertial data included in the radio signals transmitted by the RF-transmitting device.

5. The user input system of claim 1, wherein the input interface is in wireless communication with the three or more spatially separated antennas.

6. The user input system of claim 1, wherein the second interface comprises a USB connection to the device having the display screen through which the user input system receives power.

7. The user input system of claim 1, wherein the second interface comprises a High Definition Multimedia Interface (HDMI) connection to the device having the display screen through which the user input system sends the virtual environment as affected by the registered position of the RF-transmitting device.

8. A method for providing immersive interaction between users and interactive software programs, the method comprising:

acquiring timing information associated with radio signals transmitted by an RF-transmitting device operated by a user and received by at least three spatially separated antennas;

computing a position of the RF-transmitting device from the timing information associated with the radio signals received by the at least three antennas;

executing at least a portion of an interactive software program on a mobile device to provide a virtual environment;

registering the computed position of the RF-transmitting device within the virtual environment of the interactive software program as user input by which the user interacts with and navigates the virtual environment produced by the interactive software program;

receiving a signal representing the virtual environment, as affected by the registered position of the RF-transmitting device, over a wireless communication link from the mobile device executing the interactive software program; and

transmitting the virtual environment, as affected by the registered position of the RF-transmitting device, to a device having a display screen for display of the virtual environment and the RF transmitting device within the virtual environment after receiving the virtual environment, as affected by the registered position of the RF-transmitting device, from the mobile device.

9. The method of claim 8, wherein transmitting the virtual environment, as affected by the registered position of the RF-transmitting device, to the device with the display screen occurs over an HDMI connection.

10. The method of claim 8, further comprising determining an orientation of the mobile device that comprises the RF-transmitting device from inertial data included in the radio signals transmitted by the RF-transmitting device.

11. A display device comprising:

a unit with a display screen;

three or more spatially separated antennas coupled fixedly at different locations to the unit, the three or more antennas receiving radio signals sent from an RF-transmitting device operated by a user to interact with an interactive software program executing at least in part on a mobile device that produces a virtual environment; and

a user input system in communication with the antennas to acquire therefrom timing information associated with the radio signals sent from the RF-transmitting device and received by each of the antennas, the user input system including a processor programmed to calculate a position of the RF-transmitting device from the timing information associated with the radio signals received by each of the antennas and to register the position of the RF-transmitting device within the interactive software program as user input by which the user interacts with and navigates within the virtual environment to be displayed on the display screen, the user input system further including a first interface in wireless communication with the interactive software program executing at least in part on the mobile device and configured to receive the virtual environment, as affected by the registered position of the RF-transmitting device, from the mobile device, and a second interface in communication with the display screen and configured to transmit the virtual environment, as affected by the registered position of the RF-transmitting device, to the display screen after the virtual environment, as affected by the registered position of the RF-transmitting device, has been received from the mobile device via the first interface.

12. The display device of claim 11, wherein the processor of the user input system is further programmed to run at least in part the interactive software program.

13. The display device of claim 11, wherein the user input system is in wireless communication with the three or more spatially separated antennas.

14. The display device of claim 11, wherein the first interface of the user input system is in wireless communication with the mobile device that comprises the RF-transmitting device whose position is calculated and registered within the interactive software program that produces the virtual environment to be displayed on the display screen.