Vision-based interactive projection system

• US 9,626,042 B2
• Filed: 04/14/2015
• Issued: 04/18/2017
• Est. Priority Date: 10/07/2011
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Shamaie, Atid
• Primary Examiner(s): Cerullo, Liliana
• Application Number:
  US14/686,630
• Publication Number:
  US 20150220213A1
• Time in Patent Office: 735 Days
• CPC Class Codes: G03B 17/54 with projector
• CPC Class Codes: G03B 21/14 Details
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06V 40/107 Static hand or arm

Abstract

Techniques are provided to improve interaction between a user and a projection system. In some embodiments, an image of a user in front of a display screen is captured. An inference can then be made as to whether a user is touching a display screen based on an analysis of shadows and/or variation of brightness (i.e., intensities) across pixels in the image. For example, it may be inferred that the object is: (1) approaching the screen when a region surrounding a top of the object is characterized by a relatively small brightness variation; (2) hovering near the screen when the brightness variation is large and the region includes a dark extremum (caused by a shadow); and (3) touching the screen when the brightness variation is large and the region includes a light extremum.

Claims

1. A projector system, comprising:

projection optics configured to project a display on a surface from a front side of the surface;

a first light source configured to emit light in a first direction towards the surface from the front side of the surface;

a second light source configured to emit light in a second direction towards the surface from the front side of the surface, wherein the light emitted in the second direction is emitted through the projection optics via a prism positioned between the second light source and a lens of the projection optics; and

an image analyzer configured to;

determine a position of an object placed in front of the surface, based at least in part on the emitted light in the first direction, wherein the position of the object is determined based on detecting one or more edges of the object in at least one image captured from the front side of the surface; and

determine whether the object is touching the surface based at least in part on the emitted light in the second direction.

2. The projector system of claim 1, wherein the first light source and the second light source are configured to emit light in an alternating fashion.

3. The projector system of claim 1, wherein the projector system is a short-throw projector.

4. The projector system of claim 1, wherein determining the position of the object comprises determining one or more pixel coordinates associated with the object.

5. The projector system of claim 1, wherein the image analyzer is further configured to determine a region of interest that includes the determined position of the object, and wherein determining whether the object is touching the surface is based at least in part on the determined region of interest.

6. The projector system of claim 5, wherein the image analyzer is further configured to analyze brightness variation within the region of interest, and wherein determining whether the object is touching the surface is based at least in part on the analyzed brightness variation.

7. A method, comprising:

projecting a display on a surface from a front side of the surface;

emitting, via a first light source, light in a first direction towards the surface from the front side of the surface;

emitting, via a second light source, light in a second direction towards the surface from the front side of the surface, wherein the light emitted in the second direction is emitted through the projection optics via a prism positioned between the second light source and a lens of the projection optics;

determining a position of an object placed in front of the surface, based at least in part on the emitted light in the first direction, wherein the position of the object is determined based on detecting one or more edges of the object in at least one image captured from the front side of the surface; and

determining whether the object is touching the surface based at least in part on the emitted light in the second direction.

8. The method of claim 7, wherein the light emitted in the first direction and the light emitted in the second direction are emitted in an alternating fashion.

9. The method of claim 7, wherein the display is projected on the surface via a short-throw projector.

10. The method of claim 7, wherein determining the position of the object comprises determining one or more pixel coordinates associated with the object.

11. The method of claim 7, further comprising determining a region of interest that includes the determined position of the object, and wherein determining whether the object is touching the surface is based at least in part on the determined region of interest.

12. The method of claim 11, wherein further comprising analyzing brightness variation within the region of interest, and wherein determining whether the object is touching the surface is based at least in part on the analyzed brightness variation.

13. A projector system, comprising:

means for projecting a display on a surface from a front side of the surface;

means for emitting, via a first light source, light in a first direction towards the surface from the front side of the surface;

means for emitting, via a second light source, light in a second direction towards the surface from the front side of the surface, wherein the light emitted in the second direction is emitted through the projection optics via a prism positioned between the second light source and a lens of the projection optics;

means for determining a position of an object placed in front of the surface, based at least in part on the emitted light in the first direction, wherein the position of the object is determined based on detecting one or more edges of the object in at least one image captured from the front side of the surface; and

means for determining whether the object is touching the surface based at least in part on the emitted light in the second direction.

14. The projector system of claim 13, wherein the light emitted in the first direction and the light emitted in the second direction are emitted in an alternating fashion.

15. The projector system of claim 13, wherein the display is projected on the surface via a short-throw projector.

16. The projector system of claim 13, wherein the means for determining the position of the object comprises means for determining one or more pixel coordinates associated with the object.

17. The projector system of claim 13, further comprising means for determining a region of interest that includes the determined position of the object, and wherein determining whether the object is touching the surface is based at least in part on the determined region of interest.

18. The projector system of claim 17, further comprising means for analyzing brightness variation within the region of interest, and wherein determining whether the object is touching the surface is based at least in part on the analyzed brightness variation.

19. A non-transitory computer-readable medium containing instructions which, when executed by a processor, cause the processor to perform steps of, comprising:

projecting a display on a surface from a front side of the surface;

emitting, via a first light source, light in a first direction towards the surface from the front side of the surface;

emitting, via a second light source, light in a second direction towards the surface from the front side of the surface, wherein the light emitted in the second direction is emitted through the projection optics via a prism positioned between the second light source and a lens of the projection optics;

determining a position of an object placed in front of the surface, based at least in part on the emitted light in the first direction, wherein the position of the object is determined based on detecting one or more edges of the object in at least one image captured from the front side of the surface; and

determining whether the object is touching the surface based at least in part on the emitted light in the second direction.

20. The non-transitory computer-readable medium of claim 19, wherein the light emitted in the first direction and the light emitted in the second direction are emitted in an alternating fashion.

21. The non-transitory computer-readable medium of claim 19, wherein the display is projected on the surface via a short-throw projector.

22. The non-transitory computer-readable medium of claim 19, wherein determining the position of the object comprises determining one or more pixel coordinates associated with the object.

23. The non-transitory computer-readable medium of claim 19, wherein the instructions further cause the processor to determine a region of interest that includes the determined position of the object, and wherein determining whether the object is touching the surface is based at least in part on the determined region of interest.

24. The non-transitory computer-readable medium of claim 23, wherein the instructions further cause the processor to analyze brightness variation within the region of interest, and wherein determining whether the object is touching the surface is based at least in part on the analyzed brightness variation.