Interactive environment using computer vision and touchscreens

• US 20030048280A1
• Filed: 09/12/2001
• Published: 03/13/2003
• Est. Priority Date: 09/12/2001
• Status: Abandoned Application

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Russell, Ryan S.
• Application Number:
  US09/952,085
• Publication Number:
  US 20030048280A1
• Time in Patent Office: Days
• US Class Current: 345/619
• CPC Class Codes: A63F 2300/1075 using a touch screen
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

An interactive environment is created through the use of a touchscreen and a camera. A user can select an object on a display by touching the object on a touchscreen. A computer can activate a video camera in response to the touch. The video camera then inputs images of the user'"'"'s physical movements to the computer, and the computer uses software to analyze the user'"'"'s movements and apply corresponding manipulations to the object. For example, the user may select an object by touching a touchscreen near the object on a display and then rotate his or her hand to rotate the displayed object on the display.

Claims

1. An apparatus comprising:

a video camera;

a touchscreen; and

a computer coupled to the video camera and the touchscreen to manipulate an object selected with the touchscreen, based on user movements viewed by the video camera.

2. The apparatus of claim 1 further comprising a display coupled to the computer to display the object.

3. The apparatus of claim 2 wherein the touchscreen is coupled to the display.

4. The apparatus of claim 1 wherein said video camera is to be activated by a user touching the touchscreen.

5. A method comprising:

detecting a touch on a touchscreen; and

in response to said detecting, inputting viewed user movement from a video camera.

6. The method of claim 5 further comprising searching an area of a displayed environment for an object near the touch.

7. The method of claim 6 further comprising determining if a displayed object is selected, including:

if an object is near an area of the touch, selecting the object; and

if no object is near the area of the touch, not selecting any object.

8. The method of claim 6, wherein searching includes:

if multiple objects are displayed near the touch, selecting a particular one of said multiple objects nearest to the touch.

9. The method of claim 7 further comprising:

if the object has been selected, using interpretation of the viewed user movement to manipulate the selected object.

10. The method of claim 5 wherein the viewed user movement includes viewed movement of a user'"'"'s body part in a group comprising an arm, hand, finger, head, hip, leg, knee, and foot.

11. The method of claim 10 wherein the viewed user movement occurs between a static background and the camera.

12. The method of claim 10 wherein the viewed user movement represents a software recognizable motion.

13. The method of claim 5, wherein inputting viewed user movement includes using software selected from a group comprising gesture recognition software, tracking software, and video segmentation software on video frames of said user movements to determine how a selected object should be manipulated.

14. A system comprising:

a computer having a memory;

a video camera coupled to said computer;

a display coupled to said computer;

a touchscreen coupled to said computer; and

software for execution by said computer to manipulate an object displayed on the display in response to selecting the object with the touchscreen and performing

a motion by a user in view of the video camera.

15. The system of claim 14 wherein the software is selected from a group comprising gesture recognition software, tracking software, and video segmentation software.

16. The system of claim 14 wherein the video camera is located above said display.

17. The system of claim 14 wherein the video camera includes a visible light camera.

18. The system of claim 14 further comprising a static background in a view of the video camera.

19. A machine-readable medium that provides instructions, which when executed by a machine, cause said machine to perform operations comprising:

detecting a touch on a touchscreen;

identifying an object displayed near said touch on said touchscreen;

using a video camera to input video frames of a user'"'"'s movements; and

manipulating said identified object according to said video frames of said user'"'"'s movements.

20. The machine-readable medium of claim 19 wherein manipulating the identified object includes executing software selected from a group comprising gesture recognition software, tracking software, and video segmentation software.

21. The machine-readable medium of claim 19 wherein the identified object is manipulated by a user movement selected from a group comprising rotating and translating.