Multi-modal, geo-tempo communications systems

• US 8,880,606 B2
• Filed: 08/24/2012
• Issued: 11/04/2014
• Est. Priority Date: 09/14/2009
• Status: Active Grant

• Current Assignee: Applied Research Associates Incorporated
• Original Assignee: Applied Research Associates Incorporated
• Inventors: Aldunate, Roberto, Larson, Gregg E
• Primary Examiner(s): Lazaro, David
• Assistant Examiner(s): KOROBOV, VITALI A
• Application Number:
  US13/594,436
• Publication Number:
  US 20120327112A1
• Time in Patent Office: 802 Days
• Field of Search: 709/203, 709/204, 709/246, 706/47, 706/60, 701/400, 704/200, 707/607
• US Class Current: 709/204
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04L 67/52 specially adapted for the location of the user terminal
• CPC Class Codes: H04L 67/75 Indicating network or usage conditions on the user display

Abstract

Disclosed is a flexible, multi-modal system useful in communications among users, capable of synchronizing real world and augmented reality, wherein the system is deployed in centralized and distributed computational platforms. The system comprises input devices to generate signals representing speech, gestures, pointing direction, and location of a user, and transmit the same to a multi-modal interface. A plurality of agents and one or more databases are integrated into the system, where at least some of the agents receive signals from the multi-modal interface, translate the signals into data, compare the same to a database, generate signals representing meanings as defined by the database, and transmit the signals to the multi-modal interface. Finally, a plurality of output devices are associated with the system to receive and process signals from the multi-modal interface, some of said signals representing messages to the user to be communicated by means of an output device.

Claims

1. A flexible, multi-modal system useful in communications among users, capable of synchronizing real world and augmented reality, wherein the system is deployed in centralized and distributed computational platforms, the system comprising:

(a) a plurality of communication devices that transmit signals representing the location of a user, speech from a user, and gestures of a user, and that receive signals representing information;

(b) a plurality of agents that receive signals from one or more of said communication devices, translate said signals into system-specific data, and compare said data to a database to determine further action of the system, wherein said further action comprises (i) obtaining additional information from one of the said communication devices;

(ii) sending a message to another of said communication devices;

or (iii) controlling an object that receive signals from the system of the present invention;

wherein at least one of the agents generates signals representing meanings as defined by the database, and transmits the signals to a multi-modal interface having a modifiable default mode for types of signals received or generated thereby; and

wherein at least one of the agents is a location-monitoring agent that modifies the default mode of the multi-modal interface to another mode based upon the location changes of users, and communicates said modification to the multi-modal interface to modify the default mode of signals generated thereby; and

wherein at least one agent is located on computing hardware remote from at least one other agent; and

(c) wherein the plurality of agents further comprise;

(i) a mapping agent, said agent comprising a geographical internal space being defined by relative local coordinates;

said mapping agent receiving location information from one of said communication devices, and translating the same to system relative local coordinates; and

(ii) a collision detection agent, said agent being coupled with a database comprising known objects or structures and their location in the geographical internal space, wherein said agent receives orientation, azimuth and pitch information from the multi-modal interface or another agent, and location information from the multi-modal interface or the mapping agent, and using the location information and the direction information, determines which objects if any are intersected by a ray in the azimuth and pitch of the direction information, originating from a vector of the location information.

2. The system of claim 1,

wherein the collision detection agent also receives range information from a range finder input device, and

wherein said mapping agent stores an object in the database based upon the said direction information, location information and range information.

3. The system of claim 2, wherein object locations are organized into a search tree.

4. The system of claim 1, wherein the mapping agent and the collision detection agent receive and transmit signals comprising a tagged object.

5. The system of claim 4, wherein the system transmits the signal comprising the tagged object to the user for confirmation, adjustment or rejection, prior to transmission of the signal to other communication devices.

6. The system of claim 1, wherein said location-monitoring agent receives periodic signals representing location information of users, said signals being generated by one of said communication devices located on a user and transmitted to the multi-modal interface for further transmission to the location-monitoring agent, wherein said agent monitors location changes of users, and in association with a database generates signals for transmission to one or more output devices based upon some location changes of users.

7. The system of claim 1, further comprising a plurality of output devices that receive and process signals from the multi-modal interface, some of said signals representing messages to be communicated by means of an output device in visual, auditory, or tactile modes.