SYSTEM AND METHOD FOR USER INTERACTION WITH PROJECTED CONTENT

• US 20130135199A1
• Filed: 08/10/2011
• Published: 05/30/2013
• Est. Priority Date: 08/10/2010
• Status: Abandoned Application

• Current Assignee: PointGrab Ltd.
• Original Assignee: PointGrab Ltd.
• Inventors: Perski, Haim, Menadeva, Ovadya, Wohlstadter, Gil, Eilat, Eran
• Application Number:
  US13/814,756
• Publication Number:
  US 20130135199A1
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386

Abstract

A system and method for user interaction with projected content are provided. A computer generated image is projected onto a surface, the computer generated image comprising at least one symbol. The projected computer generated image is imaged to obtain a sensor image. The location of the symbol within the sensor image is detected and based on the location of the symbol in the sensor image a device may be controlled.

Claims

1-10. -10. (canceled)

11. A method for user interaction with projected content, the method comprising

projecting a computer generated image onto a surface, said computer generated image comprising at least one symbol;

imaging the projected computer generated image to obtain a sensor image;

detecting the location of the symbol within the sensor image; and

based on the location of the symbol in the sensor image, controlling a device.

12. The method according to claim 11 comprising projecting the computer generated image onto a user'"'"'s hand and calculating a distance of the user'"'"'s hand from the surface based on the location of the symbol in the sensor image.

13. The method according to claim 11 comprising transforming coordinates of the sensor image to coordinates of the computer generated image.

14. The method according to claim 12 comprising detecting a location of the user'"'"'s hand within the sensor image.

15. The method according to claim 14 comprising determining if the distance of the user'"'"'s hand from the surface is below a pre-determined distance and if the distance of the user'"'"'s hand from the surface is below a pre-determined distance, simulating a touch event on the computer generated image at the location of the hand.

16. The method according to claim 12 wherein calculating a distance of the user'"'"'s hand from the surface comprises:

calculating an expected location of the symbol in the sensor image;

detecting an actual location of the symbol in the sensor image; and

comparing the expected location to the actual location.

17. The method according to claim 11 comprising:

identifying a hand in the sensor image;

identifying the hand location within the sensor image;

translating the hand location within the sensor image to a hand location within the computer generated image; and

generating the computer generated image comprising at least one symbol located at the hand location within the computer generated image.

18. The method according to claim 17 comprising extrapolating the location of the symbol within the computer generated image.

19. The method according to claim 17 wherein the computer generated image comprises a synchronization pattern.

24. The method according to claim 11 wherein detecting the location of the symbol within the sensor image comprises:

creating a color transformation function between the projected computer generated image and the sensor image;

transforming coordinates of the projected computer generated image to coordinates of the sensor image;

transforming color space of the projected computer generated image to the color space of the sensor image, thereby obtaining a transformed image;

comparing the transformed image to the sensor image; and

determining the location of the symbol within the sensor image based on the comparison.

25. The method according to claim 24 comprising:

projecting a color calibration computer generated image;

imaging the projected color calibration image thereby obtaining a calibration sensor image;

creating a color map based on the calibration sensor image and computer generated image; and

using the color map as the color transformation function.

26. The method according to claim 24 wherein transforming coordinates of the computer generated image to coordinates of the sensor image comprises transforming corners of the computer generated image to corners of coordinates of the sensor image.

20. A method for detecting an external object on projected computer generated content, the method comprising:

creating a color transformation function between a projected computer generated image and a sensor image of the projected computer generated image;

transforming coordinates of the projected computer generated image to coordinates of the sensor image;

transforming color space of the projected computer generated image to the color space of the sensor image, thereby obtaining a transformed image;

comparing the transformed image to the sensor image; and

determining if an external object is detected in the sensor image based on the comparison.

21. The method according to claim 20 comprising:

projecting a color calibration computer generated image;

imaging the projected color calibration image thereby obtaining a calibration sensor image;

creating a color map based on the calibration sensor image and computer generated image; and

using the color map as the color transformation function.

22. The method according to claim 20 wherein transforming coordinates of the computer generated image to coordinates of the sensor image comprises transforming corners of the computer generated image to corners of coordinates of the sensor image.

23. The method according to claim 20 wherein the external object is a user'"'"'s hand.

27. The method according to claim 20 comprising:

determining a location of the external object in the sensor image; and

controlling a device based on the location of the external object.