Placing and monitoring transparent user interface elements in a live video stream as a method for user input

• US 6,353,450 B1
• Filed: 02/16/1999
• Issued: 03/05/2002
• Est. Priority Date: 02/16/1999
• Status: Expired due to Term

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: DeLeeuw, William C.
• Primary Examiner(s): Cabeca, John
• Assistant Examiner(s): Vu, Kieu D.
• Application Number:
  US09/251,223
• Time in Patent Office: 1,113 Days
• Field of Search: 345/435, 345/328, 345/327, 345/339, 345/348, 345/344, 345/345, 345/629, 345/716, 345/719, 345/764, 345/768, 345/810, 345/632, 382/203, 382/195, 382/206, 382/190, 348/584, 348/585, 348/586, 348/588
• US Class Current: 715/768
• CPC Class Codes: G06F 2203/04804 Transparency, e.g. transparent or translucent windows
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance

Abstract

Providing input signals to a system having a display, the system being coupled to a source of video data signals, such as a video camera, is accomplished by analyzing the video data signals to detect an object in the scene over or at least partially coinciding with a location of at least one transparent user interface element, inserting at least one transparent user interface element into the captured video data signals, rendering the scene represented by the video data signals on the display such that the scene is transparently visible on the display, and generating an input signal for the system in response to detecting the object over or at least partially coinciding with the at least one transparent user interface element.

Claims

1. A method of providing input signals to an application program executing on a system having a display comprising:

capturing live video data signals in real-time by a video camera, the video data signals representing a scene in the physical world;

inserting at least one transparent user interface element into the video data signals;

rendering the video data signals such that the scene and the at least one transparent user interface element are transparently visible on the display along with non-transparent output data of the application program;

analyzing the video data signals to detect an object in the scene as rendered on the display as appearing over or at least partially coinciding with a location of the at least one transparent user interface element as rendered on the display;

generating an input signal for the application program in response to detecting the object of the scene as rendered on the display as appearing over or at least partially coinciding with the at least one transparent user interface element as rendered on the display.

2. The method of claim 1, further comprising changing the appearance of the at least one transparent user interface element on the display when the input signal is generated.

3. The method of claim 1, wherein the at least one transparent user interface element comprises at least one transparent control.

4. The method of claim 3, wherein the input signal indicates activation of the at least one transparent control.

5. The method of claim 3, wherein the at least one transparent control comprises at least one of a button, check box, radio button, menu, hyperlink, and a slider bar.

6. The method of claim 1, wherein the scene includes a user of the system and the rendered scene comprises a reflected image of the user.

7. The method of claim 1, wherein the detected object comprises at least one body part of a user of the system.

8. The method of claim 1, wherein the detected object comprises at least one prop.

9. In a system having a source of live video data and a display, an apparatus comprising:

means for capturing live video data signals in real-time by the video source, the video data signals representing a scene in the physical world;

means for inserting at least one transparent user interface element into the video data signals;

means for rendering the video data signals such that the scene and the at least one transparent user interface element are transparently visible on the display along with non-transparent output data of the application program;

means for analyzing the video data signals to detect an object in the scene as rendered on the display as appearing over or at least partially coinciding with a location of the at least one transparent user interface element as rendered on the display; and

means for generating an input signal for the application program in response to detecting the object of the scene as rendered on the display as appearing over or at least partially coinciding with the at least one transparent user interface element as rendered on the display.

10. The apparatus of claim 9, further comprising means for changing the appearance of the at least one transparent user interface element on the display when the input signal is generated.

11. In a system having a source of live video data signals and a display, the live video data signals representing a scene in the physical world captured in real-time, an apparatus comprising:

a filter to insert at least one transparent user interface element into the video data signals, to render the video data signals such that the scene and the at least one transparent user interface element are transparently visible on the display along with non-transparent output data of the application program, to analyze the video data signals to detect an object in the scene as rendered on the display as appearing over or at least partially coinciding with a location of the at least one transparent user interface element as rendered on the display, and to generate an input signal for the application program in response to detecting the object of the scene as rendered on the display as appearing over or at least partially coinciding with the at least one transparent user interface element as rendered on the display.

12. The apparatus of claim 11, further comprising video capture circuitry to capture live video data signals generated by the live video data source.

13. The apparatus of claim 11, further comprising video render circuitry to render the scene and the at least one transparent user interface element on the display such that the scene and the at least one transparent user interface element are transparently visible on the display.

14. The apparatus of claim 11, further comprising a controlling application to send commands to the filter to insert the at least one transparent user interface element and to receive the generated input signal.

15. The apparatus of claim 11, wherein the at least one transparent user interface element comprises at least one transparent control.

16. The apparatus of claim 15, wherein the input signal indicates activation of the at least one transparent control.

17. The apparatus of claim 15, wherein the at least one transparent control comprises at least one of a button, check box, radio button, menu, hyperlink, and a slider bar.

18. An article comprising a machine readable medium having a plurality of machine readable instructions, wherein when the instructions are executed by a processor, the instructions cause a system to capture in real-time live video data signals generated by a video source, the video data signals representing a scene, insert at least one transparent user interface element into the video data signals, to render the video data signals such that the scene and the at least one transparent user interface element are transparently visible on the display along with non-transparent output data of the application program, to analyze the video data signals to detect an object in the scene as rendered on the display as appearing over or at least partially coinciding with a location of the at least one transparent user interface element as rendered on the display, and to generate an input signal for the application program in response to detecting the object of the scene as rendered on the display as appearing over or at least partially coinciding with the at least one transparent user interface element as rendered on the display.

19. The article of claim 18, wherein the machine readable medium further comprises instructions to change the appearance of the at least one transparent user interface element on the display when the input signal is generated.