Wireless augmented reality communication system

• US 7,035,897 B1
• Filed: 01/14/2000
• Issued: 04/25/2006
• Est. Priority Date: 01/15/1999
• Status: Expired due to Term

• Current Assignee: Throop, LLC (Intellectual Discovery Co., Ltd.)
• Original Assignee: California Institute of Technology
• Inventors: Devereaux, Ann, Jedrey, Thomas, Agan, Martin
• Primary Examiner(s): VU, THONG H
• Application Number:
  US09/483,315
• Time in Patent Office: 2,293 Days
• Field of Search: 709/203, 709/217, 709/219, 709/204, 709/231, 709/201, 705/28, 705/7, 345/156, 345/700, 345/866, 455/405, 455/462, 455/31, 455/416, 455/550, 455/566, 600/300, 713/193, 379/433, 348/14, 348/211, 348/223, 348/14.07, 348/584, 370/465, 370/352, 370/277, 370/376, 370/354, 370/522, 710/68, 340/635, 382/232, 382/247, 382/251, 375/240, 375/347, 715/794
• US Class Current: 709/203
• CPC Class Codes: A61B 5/0015 characterised by features of the telemetry system
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G08C 17/02 using a radio link
• CPC Class Codes: H04L 65/1063 Application servers providing network services systems providing special services to telephonic subscribers H04M3/42
• CPC Class Codes: H04L 65/1069 Session establishment or de-establishment
• CPC Class Codes: H04L 67/04 specially adapted for terminals or networks with limited capabilities; specially adapted for terminal portability
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/152 Multipoint control units therefor
• CPC Class Codes: H04N 7/155 involving storage of or access to video conference sessions tracking arrangements for later retrieval of a computer conference content or participants activities H04L12/1831
• CPC Class Codes: H04W 76/10 Connection setup
• CPC Class Codes: Y04S 40/18 Network protocols supporting networked applications, e.g. including control of end-device applications over a network

Abstract

The system of the present invention is a highly integrated radio communication system with a multimedia co-processor which allows true two-way multimedia (video, audio, data) access as well as real-time biomedical monitoring in a pager-sized portable access unit. The system is integrated in a network structure including one or more general purpose nodes for providing a wireless-to-wired interface. The network architecture allows video, audio and data (including biomedical data) streams to be connected directly to external users and devices. The portable access units may also be mated to various non-personal devices such as cameras or environmental sensors for providing a method for setting up wireless sensor nets from which reported data may be accessed through the portable access unit. The reported data may alternatively be automatically logged at a remote computer for access and viewing through a portable access unit, including the user'"'"'s own.

Claims

1. A mobile access unit for use in a localized communications system, comprising:

a video input configured to receive real-time video information;

a video output configured to provide real-time video information;

a wearable display connected to the video output;

a codec connected to the video input and video output; and

a transceiver, comprising;

a transmitter connected to the codec that is configured to transmit a data stream provided by the codec over an upstream wireless communication link; and

a receiver connected to the codec that is configured to receive a data stream transmitted over a downstream wireless communication link, which includes encoded real-time video;

wherein the codec is configured to;

encode real-time video information received from the video input; and

multiplex the encoded real-time video with other data to generate the data stream provided by the codec to the transmitter; and

wherein the codec is also configured to;

demultiplex the encoded real-time video from the data stream provided to the codec by the receiver; and

decode the encoded real-time video information and provide the decoded real-time video information to the video output.

2. The mobile access unit of claim 1, wherein the wearable display includes a heads up display connected to the video output and configured to receive real-time video.

3. The mobile access unit of claim 1, further comprising a video camera connected to the video input and configured to provide a real-time video output.

4. The mobile access unit of claim 1, further comprising:

an audio input configured to receive real-time audio information;

an audio output configured to provide real-time audio information;

wherein the codec is connected to the audio input and the audio output;

wherein the codec is configured to;

encode real-time audio information received from the audio input;

multiplex encoded real-time video with at least the real-time audio encoded by the codec to generate the data stream that is provided to the transmitter; and

wherein the codec is configured to;

demultiplex encoded real-time video from the data stream provided by the receiver that also includes at least encoded real-time audio;

decode the encoded real-time audio and provide the decoded real-time audio to the audio output.

5. The mobile access unit of claim 4, further comprising a headphone set connected to the audio output and configured to receive real-time audio.

6. The mobile access unit of claim 4, further comprising a microphone connected to the audio input and configured to provide a real-time audio output.

7. The mobile access unit of claim 1, further comprising:

a user interface input configured to receive information;

wherein the codec is connected to the user interface input and is configured to encode the user interface information;

wherein the codec is configured to multiplex encoded real-time video with at least the encoded user interface information to form a data stream that is provided to the transmitter; and

wherein the encoded user interface information is capable of commanding a remote device.

8. The mobile access unit of claim 1, wherein the codec is implemented using at least one electronic device.

9. The mobile access unit of claim 1, wherein the data may be real-time data.

10. The mobile access unit of claim 1, wherein the data may be non-real-time data.

11. A communication system, comprising:

at least one mobile access unit configured to communicate in a localized area with a base station, the mobile access unit comprising;

a video input configured to receive real-time video information;

a video output configured to provide real-time video;

a wearable display connected to the video output;

a mobile access unit codec connected to the video input and the video output; and

a transceiver, comprising;

a mobile access unit transmitter connected to the mobile access unit codec that is configured to transmit a data stream generated by the codec over an upstream wireless communication link; and

a mobile access unit receiver connected to the mobile access unit codec that is configured to receive a data stream transmitted over a downstream wireless communication link, which includes encoded real-time video;

wherein the mobile access unit codec is configured to;

encode real-time video information received from the video input; and

multiplex the encoded real-time video with other data to generate the data stream provided by the mobile access unit codec to the transmitter; and

wherein the codec is also configured to;

demultiplex the encoded real-time video from the data stream provided to the codec by the receiver; and

decode the encoded real-time video information and provide the decoded real-time video information to the video output; and

a fixed base station, comprising;

memory containing a registry of mobile access units within the localized area;

a transceiver, comprising;

a base station transmitter that is configured to transmit a data stream including real-time video over the downstream wireless communication link; and

a base station receiver configured to receive a data stream transmitted over the upstream wireless communication link, which includes encoded real-time video.

12. The communications system of claim 11, further comprising:

a base station router connected to the base station transceiver;

wherein the base station router;

is configured to multiplex encoded real-time video with other data to generate the data stream provided by the base station router to the base station transmitter; and

is configured to demultiplex encoded real-time video from the data stream provided to the base station router by the base station receiver.

13. The communication system of claim 12, further comprising:

a network bridge connected to the base station router; and

wherein the base station router is configured to receive encoded real-time video from the base station receiver and route the encoded real-time video to the base station transmitter or to the network bridge.

14. The communication system of claim 13, wherein:

the mobile access units further comprise;

an audio input configured to receive real-time audio information;

wherein the mobile access unit codec is connected to the audio input;

wherein the mobile access unit codec is configured to encode real-time audio information;

wherein the mobile access unit codec is configured to multiplex encoded real-time video with at least the encoded real-time audio to generate the data stream that is provided to the transmitter; and

wherein the fixed base station router is configured to demultiplex at least encoded real-time video and real-time audio from the data stream received from the base station receiver; and

wherein the base station router is configured to route encoded real-time audio to the base station transmitter or to the network bridge.

15. The communication system of claim 14, wherein the router is configured to route encoded real-time video independent of the encoded real-time audio.

16. The communication system of claim 14, further comprising:

a device connected to the network bridge via a network;

a microphone connected to the audio input of one of the mobile access units;

wherein the microphone is configured to generate real-time audio including voice commands;

wherein the device is configured to receive encoded real-time audio information from the fixed base station via the network;

wherein the device is configured to identify voice commands; and

wherein the device is configured to respond to identified voice commands.

17. The communication system of claim 16, wherein:

the base station router is configured to route real-time audio encoded in a third audio format to the base station transmitter or to the network bridge; and

encoded real-time audio that is received by the network bridge is sent to at least one device via the network.

18. The communication system of claim 12, wherein:

the mobile access units further comprise;

a user interface input for receiving user input;

wherein the mobile access unit codec is connected to the user interface input and is configured to encode the user interface information received from the user interface input;

wherein the mobile access codec is configured to multiplex the encoded real-time video with at least the encoded user interface information to form the data stream that is provided to the mobile access unit transmitter.

19. The communication system of claim 18, wherein the base station router is configured to independently route encoded real-time video information and encoded user interface information.

20. The communication system of claim 18, further comprising:

a device connected to the network bridge via a network;

wherein the fixed base station router is configured to demultiplex encoded user interface information from the data stream provided to the base station router by the base station transceiver;

wherein the base station router is configured to route encoded user interface information received by the base station router to the base station transmitter or the network bridge;

wherein the device is configured to receive encoded user interface information from the fixed base station via the network; and

wherein the device is configured to respond to encoded user interface information.

21. The communication system of claim 12, wherein:

the base station router is configured to multiplex the encoded real-time video that is received by the base station router in a data stream generated by the first mobile access unit into a data stream that is provided to the base station transmitter; and

the base station transmitter is configured to transmit the data stream generated by the base station codec that contains at least the encoded real-time video from the data stream generated by the first mobile access unit to a second mobile access unit.

22. A mobile access unit for use in a localized communications system, comprising:

means for capturing real-time video;

means for encoding the captured real-time video;

means for multiplexing the encoded real-time video with other data to form a data stream;

means for transmitting the data stream;

means for simultaneously receiving a second data stream including encoded real-time video;

means for decoding the encoded real-time video; and

wearable means for displaying the decoded real-time video.

23. The mobile access unit of claim 22, wherein the data may be real-time data.

24. The mobile access unit of claim 22, wherein the data may be non-real-time data.