Method and system for facilitating wireless, full-body, real-time user interaction with digitally generated text data

• US 6,747,666 B2
• Filed: 01/26/2001
• Issued: 06/08/2004
• Est. Priority Date: 01/26/2000
• Status: Active Grant

• Current Assignee: New York University
• Original Assignee: New York University
• Inventors: Achituv, Romy, Utterback, Camille
• Primary Examiner(s): Zimmerman, Mark
• Assistant Examiner(s): Cao, Hue Dung X.
• Application Number:
  US09/771,011
• Publication Number:
  US 20010035865A1
• Time in Patent Office: 1,229 Days
• Field of Search: 345/629, 345/589, 348/42, 348/559, 348/629, 348/700, 348/47, 382/103
• US Class Current: 345/629
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042

Abstract

An interactive display system includes an electronic camera which intermittently records a participant'"'"'s image and digitally inputs the video image into a computer, creating a digitized “virtual user image”. The camera is positioned behind a large viewing screen which faces the user and records the user'"'"'s image through an opening provided within the screen. The computer further receives a text string having a fall rate value. For each digital image received by the camera, a software program determines the destination of each line of text according to its fall rate, and then uses pixel color comparison techniques to determine if a “virtual obstacle” is located at the particular destination. If not, the text is displayed at the destination so that the text appears to “fall” on the screen. If a virtual obstacle is present, a new higher destination is determined until the virtual obstacle is no longer detected. This arrangement causes the falling text displayed on the screen to be selectively “caught” by the user and otherwise manipulated in the virtual environment.

Claims

1. A method for facilitating real-time interaction between a user and a digitally represented visual environment within which the user'"'"'s moving image is integrated, said method including the use of a computer, electronic memory, a display, a video camera, and a video input device, the method comprising the steps of:

storing a first computer generated digital image in said electronic memory;

assigning a velocity of movement to said digital image, said velocity of movement including a rate of movement and a direction of movement of said digital image;

recording the image of said user using said video camera;

simultaneously displaying the image of said user and said stored first digital image onto said display, at a predetermined refresh rate;

digitally repositioning said displayed first digital image on said display according to said assigned velocity of movement;

comparing the relative position of said displayed image of said user and said displayed first digital image;

determining when said displayed first digital image and the displayed image of said user are within a predetermined distance on said display;

changing said velocity of movement of said displayed first digital image in response to determining that said displayed first digital image and the displayed image of said user are within said predetermined distance; and

simultaneously displaying said first digital image at said new velocity of movement, and the image of said user.

2. A method for facilitating real-time interaction between a user and digitally represented text data on a display within which the user image is integrated, the method comprising the steps of:

storing text data in said electronic memory;

assigning a velocity of movement to said text data, said velocity of movement including a rate of movement and a direction of movement of said text data;

recording the image of said user using a video camera;

simultaneously displaying the image of said user and said stored text data onto said display, at a predetermined refresh rate;

digitally repositioning said displayed text data on said display by said assigned velocity of movement;

comparing the relative position of said displayed image of said user and said displayed text data;

determining when said displayed text data and the displayed image of said user are within a predetermined distance on said display;

changing the velocity of movement of said displayed text data in response to determining that said displayed text data and the displayed image of said user are within said predetermined distance; and

simultaneously displaying said text data at said new velocity of movement and the image of said user.

3. The method of claim 2, wherein said display includes an upper edge and a lower edge, and said velocity of movement of said text data includes a direction towards said lower edge of the display.

4. The method of claim 2, wherein said velocity of movement of said text data includes a first rate of movement, and said new velocity of movement of said text data includes a second rate of movement.

5. The method of claim 2, wherein said new velocity of movement of said text data includes no movement.

6. The method of claim 2, wherein said assigned velocity of movement of said text data includes a first direction of movement, and said new velocity of movement of said text data includes a second direction of movement, said second direction of movement being opposite said first direction of movement.

7. The method of claim 2, wherein said text data is initially displayed on said display at a predetermined location prior to moving at said assigned velocity.

8. The method of claim 2, wherein prior to performing the step of storing text data into electronic memory, a step of receiving text data from a keyboard is performed.

9. A method for facilitating real-time interaction between a user and digitally represented text data on a display within which the user image is integrated, the method comprising the steps of:

storing text data in an electronic memory;

assigning a velocity of movement to said text data, said velocity of movement including a rate of movement and a direction of movement of said text data;

storing a threshold pixel color value in electronic memory;

recording the image of said user using a video camera;

simultaneously displaying the image of said user and said stored text data onto said display, at a predetermined refresh rate, thereby creating a combined image;

determining a destination of said text data, according to the assigned velocity, said text data destination being the point within the combination image where the text will next be displayed;

measuring the pixel color value of the displayed image at the determined text data destination;

comparing the measured pixel color value at the text data destination with said stored threshold color value;

displaying said text data at said text data destination in response to said comparing step determining that said measured pixel color value at the text data destination is less than said stored threshold color value; and

displaying said text data at a position within said combined image on said display other than said text data destination in response to determining in said comparing step that said measured pixel color value at the text data destination is greater than the stored threshold color value.

10. The method of claim 9, further comprising the step of changing the velocity of the text data in response to said comparing step determining that said measured pixel color value at the text data destination is greater than the stored threshold color value.