Virtual command post

• US 6,215,498 B1
• Filed: 09/10/1998
• Issued: 04/10/2001
• Est. Priority Date: 09/10/1998
• Status: Expired due to Fees

• Current Assignee: Lionhearth Technologies, Inc.
• Original Assignee: Lionhearth Technologies, Inc.
• Inventors: Morgenthaler, Mark P., Filo, Andrew S., Steiner, Glenn C.
• Primary Examiner(s): Zimmerman, Mark
• Assistant Examiner(s): SANTIAGO, ENRIQUE L
• Application Number:
  US09/150,769
• Time in Patent Office: 943 Days
• Field of Search: 345/419, 345/472, 345/473, 345/474, 345/475, 345/331, 345/156, 345/157, 345/158
• US Class Current: 345/419
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering

Abstract

A system of networked terminal apparatus for creating a three dimensional animated work environment wherein terminal users of various levels of immersion are depicted as avatars in the virtual work environment and wherein actions and information of the terminal users is input into the virtual work environment through their corresponding avatars in order to perform complimentary, independent and cooperative tasks in parallel to create simultaneous sets of solutions to problems relating to command, control, communications, cognition and intelligence. In a preferred embodiment, the virtual work environment emulates an actual military command post, with all of the instruments and familiar surroundings emulated in function if not in form. The terminal users of the environment (i.e., the commanders and staff) immerse in this environment via virtual reality displays (audio and video) and inputs (microphone, body encoders and pointing devices) connected to individual personal computers. The avatars are able to interact with one another in the virtual environment and they are able to select and manipulate functional objects displayed in the virtual work environment. Non-immersed terminal users provide raw data streams to the virtual environment. The non-immersed terminal users are displayed as seated avatars or selectable icons. The immersed users, through their avatars, may select the seated avatars or icons to display data streams prepared by the non-immersed terminal users on projection screens displayed in the virtual work environment. The displayed data may be further processed by the immersed terminal users for collaborative decision making by the avatars present in the virtual environment. The processed data may be transmitted outside the virtual work environment as orders or instructions.

Claims

1. A system comprising a plurality of networked terminal apparatus for creating a three dimensional virtual work environment wherein terminal users of various levels of immersion are depicted as avatars in the virtual work environment and wherein action information of the terminal users is input into the virtual work environment through their corresponding avatars in order to perform complimentary, independent and cooperative tasks in parallel to create simultaneous sets of solutions to problems relating to command, control, communications, cognition and intelligence, the system comprising:

a) at least one first level terminal apparatus for a first fully immersed terminal user, said first level terminal apparatus including;

i) first display means for displaying said virtual work environment and for displaying, in the virtual work environment, avatars corresponding to all terminal users of the virtual work environment and for displaying functional objects in the virtual work environment;

ii) first database for prestoring computer model information and graphics for the layout of the virtual environment and the appearance of each terminal user'"'"'s avatar in the virtual environment as well as collateral information including maps, documents and presentation materials;

iii) first program means for supporting interaction between avatars and said functional objects;

iv) first position tracking means for correlating head and hand movements of said first fully immersed terminal user to their corresponding avatar displayed in the virtual work environment;

v) first audio input/output means for communicating voice data back and forth between the first fully immersed terminal user and the virtual work environment;

vi) first control means for controlling said first display means, said first database and program means, said first position tracking means, and said first audio input/output means;

b) at least one second level terminal apparatus for a second partially immersed terminal user, said second level terminal apparatus including;

i) second display means for displaying said virtual work environment and for displaying, in the virtual work environment, avatars corresponding to all terminal users of the virtual work environment and for displaying functional objects in the virtual work environment;

ii) second database for prestoring computer model information and graphics for the layout of the virtual environment and the appearance of each terminal user'"'"'s avatar in the virtual environment as well as collateral information including maps, documents and presentation materials;

iii) second program means for supporting interaction between avatars and said functional objects;

iv) second position tracking means for correlating head and hand movements of said second partially immersed terminal user to their corresponding avatar displayed in the virtual work environment;

v) second audio input/output means for communicating voice data back and forth between the second partially immersed terminal user and the virtual work environment;

vi) second electronic data entry means for entering electronic data into the virtual work environment;

vi) second control means for controlling said second display means, said second database and program means, said second position tracking means, said second audio input/output means, and said second electronic data entry means; and

c) wherein at least one of said first or second level apparatus further includes object selection means comprising an exaggerated pointer displayed in said virtual work environment, and wherein said object selection means is operative to point to, select and manipulate functional objects displayed in said virtual work environment and is operative to point to and select one or more designated transportation sites displayed in said virtual work environment, whereby selection of a designated transportation site transports the selecting avatar to said designated transportation site and provides an appropriate change in the view perspective of the selecting avatar.

2. The system according to claim 1, which further comprises a third level terminal apparatus for a non-immersed terminal user, said third level terminal apparatus including:

a) third data entry means for entering electronic data generated by said non-immersed terminal user into said virtual work environment;

b) third audio input/out means for communicating voice data back and forth between said third non-immersed terminal user and said virtual work environment;

c) third display means for displaying said electronic data generated by said non-immersed terminal user;

d) third database and program means for providing a local database of programs and information for supporting the electronic information entry capability of said non-immersed user; and

e) third control means for controlling said third electronic data entry means, said third audio input/out means, said third display means, and said third database and program means.

3. The system according to claim 2, wherein:

a) said virtual environment includes at least one projection screen for displaying data and information of interest including bit mapped images, pictures, slide shows, movies and text; and

wherein

b) at least one of said first, second or third terminal apparatus further includes means for projecting, in real time, the electronic data generated by said non-immersed user onto said at least one projection screen for viewing by all avatars present in said virtual work environment.

4. The system according to claim 3, wherein each of said first and second level terminal apparatus include zoom lock means for zooming in on an image being projected on a projection screen in the three dimensional virtual work environment and locking said image as a full frame two dimensional image on the respective first or second display means of the respective first or second level user.

5. The system according to claim 3, wherein each of said first and second level terminal apparatus further include means for receiving and carrying on a private communication originating from outside the virtual environment.

6. The system according to claim 3, which further includes means for transforming a projection screen displayed in said virtual environment into a video phone for receiving real time video and audio data streams originating from a non-immersed user outside the virtual environment.

7. The system according to claim 3, which further include recorder means for recording audio and visual changes that occur in said virtual environment for a desired duration, said recording means further including means for recording initial preconditions of video data in said virtual environment, thereby to preclude the necessity of recording full frame bandwidth intensive video data during said desired duration.

8. The system according to claim 7, wherein recorder means permit avatars to select a disembodied viewpoint perspective of said virtual work environment in both realtime and historical time.

9. The system according to claim 3, wherein at least one of said first or second level terminal apparatus further includes means for configuring and networking groupings of users of said third level terminal apparatus so that data generated by said third level users can be combined and formatted in a desired manner.

10. The system according to claim 3, which further includes document drive means for sending and receiving information to and from non-immersed users outside of said virtual environment.

11. The system according to claim 3, wherein at least one of said first or second level terminal apparatus further includes remote control means for switching priority of display on said at least one projection screen in said virtual environment the electronic data generated by a plurality of third level terminal apparatus users.

12. The system according to claim 3, which further includes:

a) means for terrain mapping two dimensional map images onto a three dimensional model in the virtual environment; and

b) means for interactively linking two dimensional map images with said terrain mapped images of said three dimensional such that manipulation of one of said two dimensional map images or said three dimensional model causes changes in the other said two dimensional map images or said three dimensional model.

13. The system according to claim 12, wherein each of said first and second level terminal apparatus include rescaling means for rescaling their respective avatars down to the scale of said three dimensional model.

14. The system according to claim 3, wherein each of first and second level terminal apparatus include staff configuration means for configuring an appropriate amount of third level apparatus users, intelligent agents, programs, databases, and related resources that are required to perform a specific task or objective.

15. A system comprising a plurality of networked terminal apparatus for creating a three dimensional virtual work environment wherein terminal users of various levels of immersion are depicted as avatars in the virtual work environment and wherein action information of the terminal users is input into the virtual work environment through their corresponding avatars in order to perform complimentary, independent and cooperative tasks in parallel to create simultaneous sets of solutions to problems relating to command, control, communications, cognition and intelligence, the system comprising:

a) at least one first level terminal apparatus for a first fully immersed terminal user, said first level terminal apparatus including;

i) first display means for displaying said virtual work environment and for displaying, in the virtual work environment, avatars corresponding to all terminal users of the virtual work environment and for displaying functional objects in the virtual work environment;

ii) first database for prestoring computer model information and graphics for the layout of the virtual environment and the appearance of each terminal user'"'"'s avatar in the virtual environment as well as collateral information including maps, documents and presentation materials;

iii) first program means for supporting interaction between avatars and said functional objects;

iv) first position tracking means for correlating head and hand movements of said first fully immersed terminal user to their corresponding avatar displayed in the virtual work environment;

v) first audio input/output means for communicating voice data back and forth between the first fully immersed terminal user and the virtual work environment;

vi) first control means for controlling said first display means, said first database and program means, said first position tracking means, and said first audio input/output means;

b) at least one second level terminal apparatus for a second partially immersed terminal user, said second level terminal apparatus including;

i) second display means for displaying said virtual work environment and for displaying, in the virtual work environment, avatars corresponding to all terminal users of the virtual work environment and for displaying functional objects in the virtual work environment;

ii) second database for prestoring computer model information and graphics for the layout of the virtual environment and the appearance of each terminal user'"'"'s avatar in the virtual environment as well as collateral information including maps, documents and presentation materials;

iii) second program means for supporting interaction between avatars and said functional objects;

iv) second position tracking means for correlating head and hand movements of said second partially immersed terminal user to their corresponding avatar displayed in the virtual work environment;

v) second audio input/output means for communicating voice data back and forth between the second partially immersed terminal user and the virtual work environment;

vi) second electronic data entry means for entering electronic data into the virtual work environment;

vi) second control means for controlling said second display means, said second database and program means, said second position tracking means, said second audio input/output means, and said second electronic data entry means;

c) avatar selectable, non-human, intelligent agents displayed as seated avatars in said virtual work environment; and

d) wherein at least one of said first or second level apparatus further includes object selection means comprising an exaggerated pointer displayed in said virtual work environment, and wherein said object selection means is operative to point to, select and manipulate functional objects displayed in said virtual work environment.

16. The system according to claim 15, which further comprises a third level terminal apparatus for a non-immersed terminal user, said third level terminal apparatus including:

a) third data entry means for entering electronic data generated by said non-immersed terminal user into said virtual work environment;

b) third audio input/out means for communicating voice data back and forth between said third non-immersed terminal user and said virtual work environment;

c) third display means for displaying said electronic data generated by said non-immersed terminal user;

d) third database and program means for providing a local database of programs and information for supporting the electronic information entry capability of said non-immersed user; and

e) third control means for controlling said third electronic data entry means, said third audio input/out means, said third display means, and said third database and program means.

17. The system according to claim 16, wherein:

a) said virtual environment includes at least one projection screen for displaying data and information of interest including bit mapped images, pictures, slide shows, movies and text; and

wherein

b) at least one of said first, second or third terminal apparatus further includes means for projecting, in real time, the electronic data generated by said non-immersed user onto said at least one projection screen for viewing by all avatars present in said virtual work environment.

18. The system according to claim 17, wherein each of said first and second level terminal apparatus include zoom lock means for zooming in on an image being projected on a projection screen in the three dimensional virtual work environment and locking said image as a full frame two dimensional image on the respective first or second display means of the respective first or second level user.

19. The system according to claim 17, wherein each of said first and second level terminal apparatus further include means for receiving and carrying on a private communication originating from outside the virtual environment.

20. The system according to claim 17, which further includes means for transforming a projection screen displayed in said virtual environment into a video phone for receiving real time video and audio data streams originating from a non-immersed user outside the virtual environment.

21. The system according to claim 17, which further include recorder means for recording audio and visual changes that occur in said virtual environment for a desired duration, said recording means further including means for recording initial preconditions of video data in said virtual environment, thereby to preclude the necessity of recording full frame bandwidth intensive video data during said desired duration.

22. The system according to claim 21, wherein recorder means permit avatars to select a disembodied viewpoint perspective of said virtual work environment in both realtime and historical time.

23. The system according to claim 17, wherein at least one of said first or second level terminal apparatus further includes means for configuring and networking groupings of users of said third level terminal apparatus so that data generated by said third level users can be combined and formatted in a desired manner.

24. The system according to claim 17, which further includes document drive means for sending and receiving information to and from non-immersed users outside of said virtual environment.

25. The system according to claim 17, wherein at least one of said first or second level terminal apparatus further includes remote control means for switching priority of display on said at least one projection screen in said virtual environment the electronic data generated by a plurality of third level terminal apparatus users.

26. The system according to claim 17, which further includes:

a) means for terrain mapping two dimensional map images onto a three dimensional model in the virtual environment; and

b) means for interactively linking two dimensional map images with said terrain mapped images of said three dimensional such that manipulation of one of said two dimensional map images or said three dimensional model causes changes in the other said two dimensional map images or said three dimensional model.

27. The system according to claim 26, wherein each of said first and second level terminal apparatus include rescaling means for rescaling their respective avatars down to the scale of said three dimensional model.

28. The system according to claim 17, wherein each of said first and second level terminal apparatus include staff configuration means for configuring an appropriate amount of third level apparatus users, intelligent agents, programs, databases, and related resources that are required to perform a specific task or objective.

29. The system according to claim 15, wherein said object selection means is further operative to point to and select one or more designated transportation sites displayed in said virtual work environment, whereby selection of a designated transportation site transports the selecting avatar to said designated transportation site and provides an appropriate change in the view perspective of the selecting avatar.