Remote collaborative control and direction

• US 7,937,285 B2
• Filed: 04/12/2002
• Issued: 05/03/2011
• Est. Priority Date: 04/12/2001
• Status: Active Grant

• Current Assignee: Massachusetts Institute of Technology, Regents of the University of California (University of California)
• Original Assignee: Massachusetts Institute of Technology
• Inventors: Lee, Matthew, Spiegel, Dana, Tang, Derek, Levandowski, Anthony, Goldberg, Kenneth Y, Song, Dezhen, Pescovitz, David, Dobson, Kelly, Paulos, Eric J., Donath, Judith
• Primary Examiner(s): Jeanty; Romain
• Assistant Examiner(s): MANSFIELD, THOMAS L
• Application Number:
  US10/121,955
• Publication Number:
  US 20030079218A1
• Time in Patent Office: 3,308 Days
• Field of Search: 705/14, 705/10
• US Class Current: 705/7.31
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06Q 30/0202 Market predictions or forecasting for commercial activities
• CPC Class Codes: H04L 12/1813 for computer conferences, e.g. chat rooms instant messaging H04L51/04; protocols for multimedia communication H04L65/1101; arrangements for multi-party communication H04L65/403; telephonic conference arrangements H04M3/56; television conference systems H04N7/15
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04L 9/40 Network security protocols
• CPC Class Codes: H04N 21/2187 Live feed
• CPC Class Codes: H04N 21/252 Processing of multiple end-users' preferences to derive collaborative data
• CPC Class Codes: H04N 21/4758 for providing answers, e.g. voting
• CPC Class Codes: H04N 21/8541 involving branching, e.g. to different story endings
• CPC Class Codes: H04N 7/17318 Direct or substantially direct transmission and handling of requests

Abstract

A collaborative online tele-operation system allows an audience of many participants to simultaneously share control of a single remote actor, such that the actions of the actor are based on the ongoing collective preferences of the audience.

Claims

1. A method of facilitating remote participation in an activity monitorable by an audience comprising a plurality of members, the method comprising the steps of:

a. facilitating remote monitoring of the activity by the audience;

b. receiving at least one client implemented on a computer system comprising a computer processor, within a response period, first input commands from multiple audience members concerning progress of the action, wherein each first input command is in the form of a votel, wherein a votel is an indicator, associated with an individual audience member, that is placed by an audience member onto a visual representation of the activity environment by the individual audience member to indicate the individual audience member'"'"'s choice of input command, each votel being defined as a vector or function of the individual audience member, the time of placement of the votel onto the visual representation of the activity environment, and the coordinates of the physical location of the placed votel within the visual representation of the activity environment;

c. presenting, in a continuous, dynamic fashion, the first input commands received from all audience members to the audience;

d. accepting, at the at least one computer system-implemented client from individual audience members within the response period, revised input commands in the form of the position of a votel placed by an audience member onto a visual representation of the activity environment;

e. presenting, in a continuous, dynamic fashion, any revised input command received from any individual audience member in place of the first input command received from that audience member along with all other input commands received during the response period;

f. processing the received input commands from all audience members using an aggregator implemented on a computer system to develop a consensus command derived from the coordinates of the votels, wherein the votels are classified into clusters based on their associated coordinates and the clusters are analyzed to determine the consensus command; and

g. directing at least one future action according to the consensus command.

2. The method of claim 1, wherein the step of facilitating remote monitoring comprises displaying the activity to the audience on at least one display device.

3. The method of claim 1, wherein the step of facilitating remote monitoring comprises:

i) receiving at least one of an audio and a video signal of the activity at a base station, the activity being performed in a remote environment relative to the audience; and

ii) transmitting the at least one received signal to the audience over a network.

4. The method of claim 3, wherein the step of receiving comprises receiving the at least one signal from an apparatus associated with a tele-actor performing the activity in the remote environment.

5. The method of claim 3, wherein the step of transmitting comprises:

forwarding the at least one received signal from the base station to a server; and

distributing the at least one received signal from the server to the audience.

6. The method of claim 3, wherein the remote environment is a real physical environment.

7. The method of claim 3, wherein the remote environment is a virtual computationally generated environment.

8. The method of claim 1, wherein the step of receiving commands from the audience comprises:

i) providing a voting interface to each member of the audience;

ii) presenting the audience with at least one choice concerning progress of the activity;

iii) facilitating voting, by at least one member of the audience, for a choice concerning progress of the activity; and

iv) forwarding the at least one vote to a server.

9. The method of claim 8, wherein the step of forwarding comprises forwarding the vote through a network to the server.

10. The method of claim 8, wherein the voting interface comprises:

a question area facilitating posting of at least one question and at least one choice concerning the progress of the activity;

a chat area facilitating communication between members of the audience; and

a voting area used by the audience to indicate a response to the at least one choice presented in the question area.

11. The method of claim 10, wherein the step of voting for a choice comprises placing a vote indicator in the voting area of the voting interface.

12. The method of claim 1, wherein a plurality of commands is received, the step of processing comprising performing cluster analysis on the received commands.

13. The method of claim 1, wherein a plurality of commands is received, the step of processing comprising weighting each received command.

14. The method of claim 13, wherein the step of weighting each received command comprises weighting each command relative to the member of the audience submitting the command.

15. The method of claim 1, wherein the step of processing comprises processing at least one motion vector.

16. The method of claim 1, wherein the step of using the consensus command comprises:

i) receiving the consensus command at a base station; and

ii) forwarding the consensus command to a tele-actor performing the activity in a remote environment relative to the audience.

17. The method of claim 16, wherein the tele-actor is a mechanical device.

18. The method of claim 17, wherein the mechanical device is a robot.

19. The method of claim 16, wherein the tele-actor is a biological organism.

20. The method of claim 19, wherein the biological organism is a mammal.

21. The method of claim 20, wherein the mammal is a human.

22. The method of claim 1, further comprising the steps of:

generating, using the aggregator, predicted consensus commands during the response period from the set of current commands; and

presenting the predicted consensus commands to the audience during the response period.

23. The method of claim 1, further comprising the step of determining a degree of collaboration among the audience members based on spatial correlation of the commands received during a response period.

24. The method of claim 23, the step of determining a degree of collaboration among the audience members further comprising the steps of:

for each question i, compute a votel density ratio, c_i;

25. A system of facilitating remote participation in an activity monitorable by an audience comprising a plurality of members, the activity being performed by at least one actor in an environment remote from the audience, the system comprising:

at least one computer processor specifically adapted to provide;

at least one client associated with the audience for facilitating monitoring of the activity by the audience via a visual display, for receiving, via a user interface, within a response period, first input commands from multiple audience members concerning the progress of the action, for presenting, in a continuous, dynamic fashion on the visual display or using an auditory or visual display device, the first input commands received from all audience members to the audience, for accepting, via the user interface, revised input commands from individual audience members within the response period, and for presenting, in a continuous, dynamic fashion on the visual display or using an auditory or visual display device, any revised input command received from any individual audience member in place of the first input command received from that audience member along with all other input commands received during the response period;

a voting interface that presents a visual representation of the activity environment and accepts input commands from the user interface in the form of the position of a votel wherein a votel is an indicator, associated with an individual audience member, that is placed by each audience member directly onto the visual representation of the activity environment by the individual audience member to indicate the individual audience member'"'"'s choice of input command, each votel being defined as a vector or function of the individual audience member, the time of placement of the votel onto the visual representation of the activity environment, and the coordinates of the physical location of the placed votel within the visual representation of the activity environment; and

an aggregator in communication with the at least one client and the at least one actor for (i) processing the received input commands from all audience members concerning the progress of the action to generate a consensus command derived from the coordinates of the votels, wherein the votels are classified into clusters based on their associated coordinates and the clusters are analyzed to determine the consensus command, (ii) forwarding, via a communications interface device, the consensus command to the actor, and (iii) transmitting the activity to the client for further monitoring by the audience on the visual display.

26. The system of claim 25, wherein the aggregator comprises:

a server in communication with the client for (i) processing the commands to generate the consensus command and (ii) forwarding the activity of the actor to the client for monitoring by the audience; and

a base-station in communication the server and the actor, for receiving the consensus command from the server and transmitting the activity to the server.

27. The system of claim 25, wherein the actor is a mechanical device.

28. The system of claim 27, wherein the mechanical device is a robot.

29. The system of claim 27, wherein the mechanical device is a camera.

30. The system of claim 25, wherein the actor is a biological organism.

31. The system of claim 30, wherein the biological organism is a mammal.

32. The system of claim 31, wherein the mammal is a human.

33. The system of claim 25 further comprising an apparatus associated with the actor for transmitting the activity to the aggregator.

34. The system of claim 25, wherein the client is a computing device comprising:

a display for facilitating monitoring of the activity and voting for a choice of action concerning the progress of the activity;

a processor for processing the vote to generate a command concerning the progress of the activity; and

a network interface for (i) transmitting the command to the aggregator and (ii) receiving the activity of the actor from the aggregator.

35. The system of claim 34, wherein the display comprises:

a question area facilitating posting of at least one question and at least one choice concerning the progress of the activity;

a chat area facilitating communication between members of the audience; and

a voting area used by the audience to indicate a response to the at least one choice presented in the question area.

36. The system of claim 25, wherein the aggregator generates predicted consensus commands during the response period from the set of current commands and the client presents the predicted consensus commands to the audience.

37. The system of claim 25, wherein the aggregator determines a degree of collaboration among the audience members based on spatial correlation of the commands received during a response period.

38. The system of claim 37, wherein the degree of collaboration among the audience members is determined by:

for each question i, compute a votel density ratio, c_i;