VIRTUAL REALITY AND AUGMENTED REALITY FUNCTIONALITY FOR MOBILE DEVICES

• US 20150221135A1
• Filed: 02/05/2015
• Published: 08/06/2015
• Est. Priority Date: 02/06/2014
• Status: Active Grant

• Current Assignee: Position Imaging, Inc.
• Original Assignee: Position Imaging, Inc.
• Inventors: Hill, Edward L., Komoni, Krenar, Piotrowski, Rafal, Xiong, Yifeng
• Granted Patent: US 10,200,819 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: H04W 4/026 using orientation information, e.g. compass

Abstract

Systems and methods improve virtual reality and augmented reality functionality for mobile devices using radio frequency (RF) signals transmitted by a tracked device and received at four or more spatially separated antennae. These antennae are connected, wirelessly or through wired connections, to a base station. Through RF signal time of arrival information acquired at the antennae, the base station can continuously determine accurate position information of the tracked device, without lighting or line of sight limitations experienced by camera and other optical systems. As the position of the RF-transmitting tracked device is registered within a virtual environment produced by an interactive software program in communication with (or part of) the base station, the virtual viewpoint of the tracked device is controlled to reflect the relative position and orientation of the tracked device with respect to the virtual environment produced by the software program and displayed on a view screen.

Claims

1. A system for immersing a user of a mobile, RF signal-transmitting tracked device in a digital world produced and displayed on a view screen by an interactive software program, the system comprising:

a network of four or more spatially separated radio frequency (RF) receiver antennae at different locations, each of the RF receiver antennae receiving RF signals transmitted from the tracked device; and

a base station in communication with the network of four or more spatially separated RF receiver antennae, the base station including one or more receiver channels for processing the RF signals acquired by the four or more spatially separated RF receiver antennae and a processor programmed to dynamically compute a position and orientation of the tracked device from information conveyed by the RF signals as the tracked device moves within a physical environment,

wherein the base station sends the computed position and orientation of the tracked device to the interactive software program to adjust an image displayed on the view screen in accordance with a virtual viewpoint of the tracked device determined from the computed position and orientation.

2. The system of claim 1, wherein the processor of the base station is further programmed to run the interactive software program.

3. The system of claim 1, wherein the interactive software program is executing on the tracked device, the base station includes a RF signal transmitter, and the processor of the base station is programmed to transmit the position and orientation through the RF signal transmitter to the interactive software program running on the tracked device.

4. The system of claim 1, wherein the base station includes a RF signal transmitter, and the processor of the base station is programmed to transmit data to provide one or more of haptic and interactive feedback to the interactive software program.

5. The system of claim 1, wherein the view screen used for displaying the digital world produced by the interactive software program comprises a monitor separate from the tracked device, and the base station includes a High Definition Multimedia Interface (HDMI) connection to the monitor.

6. The system of claim 1, further comprising a computing device with a processor running the interactive software program, and wherein the base station includes an interface to the computing device by which the base station sends the position and orientation to the interactive software program.

7. The system of claim 1, wherein the interactive software program is one of a virtual reality application program or an augmented reality application program.

8. The system of claim 1, wherein the information conveyed by the RF signals includes inertial data acquired by inertial sensors of the tracked device.

9. The system of claim 1, further comprising the RF signal-transmitting tracked device a processor that executes the interactive software program and the view screen upon which the digital world is displayed in response to execution of the interactive software program.

10. A method for improving virtual reality and augmented reality interactions with a tracked device, the method comprising:

displaying, on a view screen, a software-generated environment from a three-dimensional virtual viewpoint of the tracked device;

receiving, on at least four spatially separated receiver antennae, radio-frequency (RF) signals transmitted by the tracked device;

dynamically computing a three-dimensional physical position, based on information conveyed by the RF signals received by the receiver antennae, and an orientation of the tracked device as the tracked device moves within a physical environment;

dynamically updating the three-dimensional virtual viewpoint of the tracked device based on the dynamically computed three-dimensional physical position and orientation of the tracked device; and

dynamically updating the display of the software-generated environment on the view screen in accordance with the dynamically updated three-dimensional virtual viewpoint of the tracked device.

11. The method of claim 10, wherein a processor of a base station in communication with the at least four spatially separated receiver antennae dynamically computes the three-dimensional physical position and orientation of the tracked device as the tracked device moves within a physical environment.

12. The method of claim 11, further comprising sending, by the base station, the computed position and orientation to a second processor that dynamically updates the three-dimensional virtual viewpoint of the tracked device based on the dynamically computed three-dimensional physical position and orientation of the tracked device, and dynamically updates the display of the software-generated environment on the view screen in accordance with the dynamically updated three-dimensional virtual viewpoint of the tracked device.

13. The method of claim 12, wherein the tracked device comprises the second processor.

14. The method of claim 12, wherein a computing device in communication with the base station comprises the second processor.

15. The method of claim 10, further comprising generating one of haptic and interactive feedback in response to the computed position and orientation.

16. The method of claim 10, wherein the software-generated environment is one of a virtual reality digital world or an augmented reality digital world.

17. The method of claim 10, wherein the information conveyed by the RF signals includes inertial data acquired by one or more inertial sensors of the tracked device.

18. The method of claim 10, wherein the tracked device dynamically computes the orientation of the tracked device based on inertial data acquired by one or more inertial sensors of the tracked device.

19. A method for improving virtual reality or augmented reality interactions with a tracked device, the method comprising:

dynamically tracking a three-dimensional physical position and orientation of an RF-transmitting tracked device as the tracked device moves within a physical environment using radio-frequency (RF) and inertial sensing signals transmitted by the RF-transmitting tracked device;

registering a virtual viewpoint of the RF-transmitting tracked device, determined from the physical position and orientation of the RF-transmitting tracked device, in an interactive software program; and

spatially registering the virtual viewpoint of the RF-transmitting tracked device with respect to a digital image displayed on a display screen by the interactive software program.

20. Computer program product for improving virtual reality and augmented reality interactions with a tracked device, the computer program product comprising:

a computer readable non-transitory storage medium having computer readable program code embodied therewith, the computer readable program code comprising;

computer readable program code that, if executed, displays on a view screen a software-generated environment from a three-dimensional virtual viewpoint of the tracked device;

computer readable program code that, if executed, receives, at four spatially separated receiver antennae, radio-frequency (RF) signals transmitted by the tracked device;

computer readable program code that, if executed, dynamically computes, based on information conveyed by the RF signals received by the receiver antennae, a three-dimensional physical position and orientation of the tracked device as the tracked device moves within a physical environment;

computer readable program code that, if executed, updates the three-dimensional virtual viewpoint of the tracked device based on the dynamically computed three-dimensional physical position and orientation of the tracked device; and

computer readable program code that, if executed, updates the display of the software-generated environment on the view screen in accordance with the dynamically updated three-dimensional virtual viewpoint of the tracked device.

21. A mobile device, comprising:

one or more inertial sensors measuring inertial data used to compute an orientation of the mobile device;

a radio frequency (RF) signal transmitter transmitting RF signals conveying RF data;

an RF signal receiver receiving RF signals conveying three-dimensional physical position about the mobile device computed from the RF data conveyed by the transmitted RF signals; and

a processor executing an interactive software program that displays a digital world on a view screen from a three-dimensional viewpoint of the mobile device, the interactive software program dynamically updating the virtual viewpoint of the mobile device based on the three-dimensional physical position and orientation of the mobile device, and dynamically updating the display of the software-generated environment on the view screen in accordance with the dynamically updated virtual viewpoint of the mobile device.

22. The mobile device of claim 21, wherein the processor is configured to compute the orientation of the mobile device based on the measured inertial data.

23. The mobile device of claim 21, wherein the RF signals transmitted by the RF signal transmitter include the measured inertial data, and wherein the RF signals received by the RF signal receiver include the orientation of the mobile device computed at a base station.