Networked gesture collaboration system

• US 20080028325A1
• Filed: 07/25/2006
• Published: 01/31/2008
• Est. Priority Date: 07/25/2006
• Status: Active Grant

• Current Assignee: Northrop Grumman Systems Corporation (Northrop Grumman Corporation)
• Original Assignee: Northrop Grumman Corporation
• Inventors: Ferren, Bran, Nishihara, H. Keith
• Granted Patent: US 8,234,578 B2
• Time in Patent Office: Days
• US Class Current: 715/753
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display

Abstract

A system and method is provided for a collaboration input/output (I/O) system. The collaboration I/O system may comprise a display screen and a gesture image system. The gesture image system may be configured to generate image data associated with a location and an orientation of a first sensorless input object relative to a background surface. The collaboration I/O system may also comprise a transceiver. The transceiver could be configured to transmit the image data to at least one additional collaboration I/O system at at least one remote location. The transceiver can be further configured to receive image data from each of the at least one additional collaboration I/O system, such that the display screen can be configured to display the image data associated with a location and orientation of a sensorless input object associated with each of the at least one additional collaboration I/O system superimposed over a common image of visual data.

Claims

1. A collaboration input/output (I/O) system comprising:

a display screen;

an image system configured to generate image data associated with a location and an orientation of a first sensorless input object relative to a background surface; and

a transceiver configured to transmit the image data to at least one additional collaboration I/O system at at least one remote location, and further configured to receive image data from each of the at least one additional collaboration I/O system, such that the display screen is configured to display the image data associated with a location and orientation of a sensorless input object associated with each of the at least one additional collaboration I/O system superimposed over a common image of visual data.

2. The system of claim 1, further comprising a projector configured to project both the common image of visual data and the image data of the sensorless input object associated with each of the at least one additional collaboration I/O system.

3. The system of claim 1, wherein at least one of the first sensorless input object and the sensorless input object associated with each of the at least one additional collaboration I/O system comprises a hand associated with at least one user.

4. The system of claim 1, wherein the image data displayed on the display screen comprises a semi-transparent silhouette image of the sensorless input object associated with each of the at least one additional collaboration I/O system superimposed on the common image of visual data, the semi-transparent silhouette image being a distinct color that corresponds to the respective one of the at least one additional collaboration I/O system.

5. The system of claim 1, wherein the background surface comprises a retroreflective surface, and wherein the image system comprises a light source coupled to a camera, such that the first image data comprises a silhouette of the sensorless input object received by the camera based on a reflected light contrast difference between the background surface and the sensorless input object.

6. The system of claim 5, wherein the image system further comprises an automatic calibration unit configured to automatically calibrate the image system based on the light source and the camera upon placing a predetermined pattern on the display screen.

7. The system of claim 5, wherein the display screen is the background surface, and wherein the light source is an infrared (IR) light source coupled and the camera is an IR filtered camera.

8. The system of claim 7, further comprising a projector configured to display the common image of visual data and a beamsplitter configured to provide coaxial boresighting between the IR light source, the IR filtered camera, and the projector.

9. The system of claim 7, wherein the image system further comprises at least one additional IR light source coupled to a respective additional IR filtered camera, such that the image system is further configured to determine gesture inputs associated with three-dimensional motion of the first sensorless input object relative to the display screen, the gesture inputs being translated into device control inputs.

10. The system of claim 1, wherein the transceiver is communicatively coupled to one of a wired network and a wireless network.

11. The system of claim 1, wherein the transceiver receives the image data from the at least one additional collaboration I/O system via a satellite.

12. The system of claim 1, wherein the collaboration I/O system is self-contained, collapsible, and portable.

13. A method for providing collaboration of visual data between a plurality of users, each of the plurality of users being located at a remote location separate from each other, the method comprising:

transmitting first image data associated with a sensorless input object of a first user of the plurality of users to each of the remaining plurality of users, the first image data comprising location and orientation data of the sensorless input object of the first user relative to a display screen of the first user;

receiving second image data associated with a sensorless input object of each of the remaining plurality of users, the second image data comprising location and orientation data of the sensorless input object of each of the remaining plurality of users relative to the visual data; and

projecting an image of the visual data and the second image data onto the display screen of the first user, the second image data being superimposed on the image of the visual data, the image of the visual data being common to the plurality of users.

14. The method of claim 13, wherein at least one of the sensorless input object of the first user and the sensorless input object of each of the remaining plurality of users comprises a hand of the respective one of the plurality of users.

15. The method of claim 13, wherein projecting the second image data comprises projecting semi-transparent silhouette images of the sensorless input objects of the remaining plurality of users superimposed on the image of the visual data, each of the semi-transparent silhouette images being a distinct color that corresponds to the respective one of the remaining plurality of users.

16. The method of claim 13, wherein the display screen is retroreflective, and further comprising illuminating the retroreflective display screen of the first user with an infrared (IR) light source and capturing a silhouette image of the sensorless input object of the first user with an IR filtered camera, the IR light source being coupled to the IR filtered camera and the silhouette image being based on a reflected light contrast difference between the display screen and the sensorless input object.

17. The method of claim 16, further comprising coaxially combining at a beamsplitter an input of the IR filtered camera with an output of the IR light source and an output of a projector configured to project the image of the visual data and the second image data.

18. The method of claim 16, further comprising automatically calibrating the image data based on the IR light and the IR filtered camera upon placing a predetermined pattern on the display screen.

19. The method of claim 16, further comprising:

illuminating the display screen with at least one additional IR light source coupled to a corresponding additional IR filtered camera;

determining gesture inputs associated with three-dimensional motion of the sensorless input object of the first user relative to the display screen; and

translating the gesture inputs into device control inputs.

20. The method of claim 13, wherein transmitting the first image data comprises one of transmitting the first image data on a wired network and wirelessly transmitting the first image data.

21. The method of claim 13, wherein receiving the second image data comprises one of receiving the second image data from a wired network, receiving the second image data from a wireless network, and receiving the second image data from a satellite.

22. A system for collaborating visual data with a first user and at least one additional user located at a remote location that is separate from the first user, the system comprising:

means for generating first image data, the first image data being associated with a location and an orientation of a sensorless input object of the first user relative to a background surface;

means for transmitting the first image data to the at least one additional user and for receiving second image data, the second image data being associated with a location and an orientation of a sensorless input object of each of the at least one additional user;

means for combining the second image data with the visual data, the visual data being common to both the first user and each of the at least one additional user; and

means for displaying the second image data and the image of the visual data to the first user, such that the second image data is superimposed over the image of the visual data.

23. The system of claim 22, wherein the means for generating the first image data comprises generating first image data associated with a location and an orientation of at least one hand of the first user.

24. The system of claim 22, wherein the means for displaying the second image data comprises displaying semi-transparent silhouette images of the sensorless input object associated with each of the at least one additional user superimposed on the image of the visual data, the semi-transparent silhouette images being a distinct color that corresponds to the respective one of the at least one additional user.

25. The system of claim 22, wherein the background surface is the display screen, and wherein the means for generating first image data comprises means for generating a silhouette of the sensorless input object based on a reflected light contrast difference between the display screen and the sensorless input object.

26. The system of claim 22, further comprising means for automatically calibrating the means for generating the first image data.

27. The system of claim 22, wherein the means for generating the image data comprises means for translating gesture inputs associated with three-dimensional motion of the first sensorless input object relative to the background into device control inputs.