Networked gesture collaboration system

• US 8,234,578 B2
• Filed: 07/25/2006
• Issued: 07/31/2012
• Est. Priority Date: 07/25/2006
• Status: Active Grant

• Current Assignee: Northrop Grumman Systems Corporation (Northrop Grumman Corporation)
• Original Assignee: Northrop Grumman Systems Corporatiom
• Inventors: Ferren, Bran, Nishihara, H Keith
• Primary Examiner(s): Nguyen, Chanh
• Assistant Examiner(s): WALTHALL, ALLISON N
• Application Number:
  US11/492,571
• Publication Number:
  US 20080028325A1
• Time in Patent Office: 2,198 Days
• Field of Search: 715/753, 715/754, 715/863, 348/14.01, 348/14.02, 348/586, 348/587, 353/28, 353/30, 353/37, 345/156
• US Class Current: 715/753
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display

Abstract

A system and method is provided for a collaboration input/output (I/O) system. The collaboration I/O system may comprise a display screen and a gesture image system. The gesture image system may be configured to generate image data associated with a location and an orientation of a first sensorless input object relative to a background surface. The collaboration I/O system may also comprise a transceiver. The transceiver could be configured to transmit the image data to at least one additional collaboration I/O system at at least one remote location. The transceiver can be further configured to receive image data from each of the at least one additional collaboration I/O system, such that the display screen can be configured to display the image data associated with a location and orientation of a sensorless input object associated with each of the at least one additional collaboration I/O system superimposed over a common image of visual data.

Claims

1. A collaboration input/output (I/O) system comprising:

a retroreflective display screen;

an image system comprising at least one infrared (IR) filtered camera and at least one IR light source coupled to the respective at least one IR filtered camera and being configured to obtain silhouette images of a first sensorless input object to generate image data associated with a location and an orientation of the first sensorless input object relative to the retroreflective display screen;

a transceiver configured to transmit the image data to at least one additional collaboration I/O system at least one remote location, and further configured to receive image data from each of the at least one additional collaboration I/O system, such that the display screen is configured to display the image data associated with a location and orientation of a sensorless input object associated with each of the at least one additional collaboration I/O system superimposed over a common image of visual data;

a projector configured to project both the common image of visual data and the image data of the sensorless input object associated with each of the at least one additional collaboration I/O system onto the retroreflective display screen; and

a beamsplitter configured to coaxially combine an input of the at least one IR filtered camera with an output of the respective at least one IR light source and an output of the projector.

2. The system of claim 1, wherein at least one of the first sensorless input object and the sensorless input object associated with each of the at least one additional collaboration I/O system comprises a hand associated with at least one user.

3. The system of claim 1, wherein the image data displayed on the retroreflective display screen comprises a semi-transparent silhouette image of the sensorless input object associated with each of the at least one additional collaboration I/O system superimposed on the common image of visual data, the semi-transparent silhouette image being a distinct color that corresponds to the respective one of the at least one additional collaboration I/O system.

4. The system of claim 3, wherein the semi-transparent silhouette image of at least one additional user that is co-located at the collaboration I/O system has a distinct color relative to a first user of the collaboration I/O system.

5. The system of claim 1, wherein the image system further comprises an automatic calibration unit configured to automatically calibrate the image system based on the light source coupled to the at least one IR filtered camera upon placing a predetermined pattern on the display screen.

6. The system of claim 1, wherein the image system is further configured to translate gesture inputs associated with three-dimensional motion of the first sensorless input object relative to the retroreflective display screen into device control inputs.

7. The system of claim 6, wherein the device control inputs comprises manipulation of the visual data based on respective gesture inputs that comprise movement of at least one digit of a user'"'"'s hand.

8. The system of claim 1, wherein the transceiver is communicatively coupled to one of a wired network and a wireless network.

9. The system of claim 1, wherein the transceiver receives the image data from the at least one additional collaboration I/O system via a satellite.

10. The system of claim 1, wherein the collaboration I/O system is self-contained, collapsible, and portable.

11. The system of claim 1, wherein the at least one additional collaboration I/O system comprises a plurality of additional collaboration I/O systems, and wherein the collaboration I/O system is a master site that transmits the visual data to each of the plurality of additional I/O systems for collaboration.

12. The system of claim 1, wherein the image data associated with the location and orientation of the sensorless input object associated with each of the at least one additional collaboration I/O system is annotated with a textual identifier that corresponds to a respective one of at least one additional user.

13. The system of claim 1, wherein the image data comprises a positional relationship between the visual data and the sensorless input object is digitally compressed in a run-length encoding algorithm for transmission of the image data to the at least one additional collaboration I/O system.

14. The system of claim 1, wherein the gesture inputs comprise at least one of clicking-and-dragging, zooming, panning, and rotating at least a portion of the visual data based on three-dimensional motion of the sensorless input object.

15. A method for providing collaboration of visual data between a plurality of users, each of the plurality of users being located at a remote location separate from each other, the method comprising:

illuminating a retroreflective display screen of a first user with an infrared (IR) light source and capturing first image data comprising a silhouette image of a sensorless input object of the first user with an IR filtered camera, the IR light source being coupled to the IR filtered camera and the silhouette image being based on a reflected light contrast difference between the display screen and the sensorless input object;

transmitting first image data associated with a sensorless input object of a first user of the plurality of users to each of the remaining plurality of users, the first image data comprising location and orientation data of the sensorless input object of the first user relative to the retroreflective display screen of the first user;

receiving second image data associated with a sensorless input object of each of the remaining plurality of users, the second image data comprising location and orientation data of the sensorless input object of each of the remaining plurality of users relative to the visual data;

projecting an image of the visual data and the second image data onto the display screen of the first user via a projector, the second image data being superimposed on the image of the visual data, the image of the visual data being common to the plurality of users; and

coaxially combining at a beamsplitter an input of the IR filtered camera with an output of the IR light source and an output of the projector.

16. The method of claim 15, wherein at least one of the sensorless input object of the first user and the sensorless input object of each of the remaining plurality of users comprises a hand of the respective one of the plurality of users.

17. The method of claim 15, wherein projecting the second image data comprises projecting semi-transparent silhouette images of the sensorless input objects of the remaining plurality of users superimposed on the image of the visual data, each of the semi-transparent silhouette images being a distinct color that corresponds to the respective one of the remaining plurality of users.

18. The method of claim 15, further comprising automatically calibrating the image data based on the IR light and the IR filtered camera upon placing a predetermined pattern on the display screen.

19. The method of claim 15, further comprising:

determining gesture inputs associated with three-dimensional motion of the sensorless input object of the first user relative to the display screen; and

translating the gesture inputs into device control inputs.

20. The method of claim 15, wherein transmitting the first image data comprises one of transmitting the first image data on a wired network and wirelessly transmitting the first image data.

21. The method of claim 15, wherein receiving the second image data comprises one of receiving the second image data from a wired network, receiving the second image data from a wireless network, and receiving the second image data from a satellite.

22. A networked gesture collaboration system comprising a plurality of collaboration I/O systems at each of a respective plurality of remote locations, each of the plurality of collaboration I/O systems comprising:

an image system comprising at least one IR filtered camera and at least one IR light source coupled to the respective at least one IR filtered camera and being configured to obtain silhouette images of a sensorless input object to generate image data associated with a location and an orientation of the sensorless input object relative to a retroreflective display screen, wherein the image system associated with at least one of the plurality of collaboration I/O systems comprises a first IR filtered camera and a second IR filtered camera and a first and second IR light source coupled to the respective first and second IR cameras and being configured to determine gesture inputs based on the image data to provide simulated device inputs to the respective one of the plurality of collaboration I/O systems for interaction with visual data that is transmitted from one of the plurality of collaboration I/O systems designated as a master site to each other one of the plurality of collaboration I/O systems;

a transceiver configured to transmit the image data and simulated device input data associated with the simulated device inputs to each other one of the plurality of collaboration I/O systems and further configured to receive the image data and simulated device input data associated with the simulated device inputs from each other one of the plurality of collaboration I/O systems;

an image combiner configured to combine the image data and the simulated device input data from each of the plurality of collaboration I/O systems with the visual data;

a projector configured to project images of the sensorless input object and the simulated device inputs associated with each of the respective plurality of collaboration I/O systems superimposed over the visual data onto the retroreflective display screen; and

wherein at least one of the plurality of collaboration I/O systems further comprises a beamsplitter configured to coaxially combine an input of the at least one IR filtered camera with an output of the respective at least one IR light source and an output of the projector.