Device using a camera and light polarization for the remote displacement of a cursor on a display

• US 7,542,072 B2
• Filed: 07/26/2005
• Issued: 06/02/2009
• Est. Priority Date: 07/28/2004
• Status: Expired due to Fees

• Current Assignee: University of Maryland
• Original Assignee: University of Maryland
• Inventors: DeMenthon, Daniel
• Primary Examiner(s): Ye; Lin
• Assistant Examiner(s): CAMARGO, MARLY S.B.
• Application Number:
  US11/189,377
• Publication Number:
  US 20060023111A1
• Time in Patent Office: 1,407 Days
• Field of Search: 348/360, 348/148, 348/36, 348/143, 348/272, 348/207.99, 348/211.99, 345/156, 345/157, 345/158, 345/166, 345/160, 345/163, 345/690, 345/694, 345/696, 345/605, 345/589, 345/596, 345/597, 382/189, 382/190
• US Class Current: 348/207.99
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Abstract

A system and method is provided for imaging a screen of a display device, including filtering light emitted or reflected from the screen and a background region in proximity to the screen using first and second polarizing angles; sensing the filtered light; and generating at least one image set corresponding to the sensed filtered light. Each image set includes at least one image having a first plurality of pixels corresponding to the sensed filtered light using the first polarizing angle and a second plurality of pixels corresponding to the sensed filtered light using the second polarizing angle.

Claims

1. A system for imaging a device having a screen which emits a polarized light, the system comprising:

a photo sensor array for imaging the polarized light emitting screen including sensing incident polarized light emitted from the screen and incident light reflected from a background region near the screen and generating an image set corresponding to the sensing;

at least one polarizing filter including at least first and second polarizing regions for filtering light incident on the photo sensor array, wherein the first and second polarizing regions have first and second polarizing angles, respectively, wherein the image set includes at least one image having a first plurality of pixels corresponding to the sensing of incident light filtered by the first polarizing region and a second plurality of pixels corresponding to the sensing of incident light filtered by the second polarizing region; and

at least one processor for executing a series of programmable instructions which provide for determining a location of the imaged screen in the imaged set by comparing the first and second plurality of pixels.

2. The system according to claim 1, wherein determining the location comprises:

computing intensity differentials between first and second pixels of respective pixel pairs of the image set, wherein for each pixel pair the first pixel is included in the first plurality of pixels and the second pixel is included in the second plurality of pixels;

comparing the intensity differentials of the respective pixel pairs of the image set; and

associating pixel pairs having a relative high intensity differential with at least one imaged feature of the imaged screen for determining the location of the imaged feature in the image set.

3. The system according to claim 2, wherein the polarized light emitted by the screen is strongly polarized in a single direction relative to incidental light reflected from the background region.

4. The system according to claim 2, wherein the at least one imaged feature includes at least a portion of at least one edge of the imaged screen.

5. The system according to claim 2, wherein the series of programmable instructions further provides for determining a region of the processed image set that corresponds to the imaged screen in accordance with the determined location of the at least one imaged feature of the imaged screen.

6. The system according to claim 5, wherein the series of programmable instructions further provides for:

recognizing characters in the determined region of the processed image set that corresponds to the imaged screen; and

generating corresponding digital representations of the recognized characters.

7. The system according to claim 2, wherein the series of programmable instructions further provides for tracking movement of the at least one imaged feature in the image set;

and

further comprising a communication device for providing tracking information indicative of the tracked movement to a processing device associated with the display device for controlling the position of a position indicator provided on a graphical user interface (GUI) displayed on the screen of the display device in accordance with the tracked movement of the at least one detested imaged feature.

8. The system according to claim 1, wherein:

the image set includes first and second consecutively acquired images, wherein the first acquired image includes the first plurality of pixels and the second acquired image includes the second plurality of pixels; and

the at least one filter includes a first filter including the first polarizing region and a second filter including the second polarizing region.

9. The system according to claim 1, wherein:

the image set includes a single image; and

the at least one filter includes a single filter, with the first and second polarizing regions arranged in a pattern on the single filter.

10. The system according to claim 2, wherein:

the at least one filter includes a single filter, with the at least first and second polarizing regions arranged in a pattern on the single filter, the pattern including a plurality of boundaries between the first and second polarizing regions;

the image set includes a single image; and

intensity differentials are computed for pixel pairs in which the first and second pixels of the respective pixel pairs are proximate to one another, lie on opposite sides of a theoretical boundary in the image which corresponds to a boundary of the plurality of boundaries included in the pattern, and lie near the theoretical boundary.

11. The system according to claim 10, wherein the proximate first and second pixels are nearly adjacent to one another, and intensity differentials are computed only for the pixel pairs in which the first and second pixels of the respective pairs are nearly adjacent to one another and lie on opposite sides of the theoretical boundary.

12. The system according to claim 1, wherein the difference between the first and second polarizing angles is 90 degrees.

13. The system according to claim 10, wherein the single filter further comprises third and fourth polarizing regions having third and fourth polarization angles, wherein the pattern on the single filter includes a plurality of boundaries between the third and fourth polarizing regions.

14. The system according to claim 13, wherein the difference between the third and fourth polarizing angles is 90 degrees.

15. The system according to claim 13, wherein the difference between the first and third polarizing angles is 45 degrees.

16. The system according to claim 10, wherein the pattern includes a plurality of sectors, with respective adjacent sectors of the plurality of sectors corresponding to different regions of the at least first and second polarizing regions.

17. The system according to claim 16, wherein the plurality of sectors are arranged concentrically.

18. The system according to claim 10, wherein the pattern includes a checkerboard arrangement of a plurality of shapes, with respective adjacent shapes corresponding to different regions of the at least first and second polarizing regions.

19. A method for imaging a device having a screen which emits a polarized light, the method comprising the steps of:

filtering light emitted from the screen and incident light reflected from a background region in proximity to the screen using first and second polarizing angles;

sensing the filtered light;

generating an image set corresponding to the sensed filtered light, the image set including a first plurality of pixels corresponding to the sensed filtered light using the first polarizing angle and a second plurality of pixels corresponding to the sensed filtered light using the second polarizing angle; and

determining a location of the imaged screen in the at least one image set by comparing the first and second plurality of pixels.

20. The method according to claim 19, wherein the determining comprises:

computing intensity differentials between first and second pixels of respective pixel pairs of the image set, wherein for each pair of pixels the first pixel is included in the first plurality of pixels and the second pixel is included in the second plurality of pixels;

comparing the intensity differentials of the respective pixel pairs of the image set; and

associating pixel pairs having a relative high intensity differential with at least one imaged feature of the imaged screen for determining the location of the imaged feature in the image set.

21. The method according to claim 20, wherein the polarized light emitted by the screen is strongly polarized in a single direction relative to incidental light reflected from the background region.

22. The method according to claim 20, wherein the at least one imaged feature includes at least a portion of at least one edge of the imaged screen.

23. The method according to claim 20, further comprising the step of determining a region of the processed image set that corresponds to the imaged screen in accordance with the determined location of the at least one imaged feature of the imaged screen.

24. The method according to claim 23, further comprising the steps of:

recognizing characters in the determined region of the image set that corresponds to the imaged screen; and

generating corresponding digital representations of the recognized characters.

25. The method according to claim 21, further comprising the steps of:

tracking movement of the at least one imaged feature in the image set; and

transmitting tracking information indicative of the tracked movement to a processing device associated with the display device for controlling the position of a position indicator provided on a graphical user interface (GUI) displayed on the screen of the display device in accordance with the tracked movement of the at least one imaged feature.

26. A system for remotely controlling a cursor displayed on a graphical user interface (GUI) displayed on a screen of a display device, wherein the screen emits a polarized light, said system comprising:

a polarizing filter including at least first and second polarizing regions arranged in an arrangement for filtering light, wherein the first and second polarizing regions have first and second polarizing angles, respectively;

a photo sensor array for imaging the polarized light emitting screen including sensing incident light emitted from the screen and incident light reflected from a background region in proximity to the screen, said incident light being filtered by the first and second polarizing regions, said photo sensor array generating at least one image corresponding to the sensed filtered incident light;

at least one processor for executing a series of programmable instructions which provide for;

comparing intensities between first and second pixels of respective pixel pairs of an image of the at least one image, wherein the first pixel corresponds to incident light filtered by the first polarizing region and the second pixel corresponds to incident light filtered by the second polarizing region;

determining a location of at least one imaged feature of the imaged screen in the image based on the intensity comparisons of the respective pixel pairs; and

tracking movement of the location of the at least one imaged feature in at least two images of the at least one image; and

a communication device for transmitting tracking information indicative of the tracked movement to a processing device associated with the display device for controlling the position of a position indicator provided on a GUI displayed on the screen of the display device in accordance with the tracked movement of the location of the at least one imaged feature.

27. The system according to claim 26, wherein the comparing step comprises computing a difference between the intensities of the first and second pixels.

28. The system according to claim 27, wherein the determining step comprises:

determining intensity differentials of the computed intensity differentials that satisfy a condition; and

detecting a pattern formed by at least a portion of the pixel pairs that correspond to the determined intensity differentials, wherein the pattern corresponds to the at least one feature of the screen in the image.

29. The system according to claim 26, wherein the arrangement includes a plurality of boundaries between the at least first and second polarizing regions, and the first and second pixels of the respective pixel pairs are proximate to one another in the image and are located on opposite sides of a theoretical boundary in the image which corresponds to a boundary of the plurality of boundaries.

30. The system according to claim 29, wherein the proximate first and second pixels of the respective pixel pairs are nearly adjacent to one another.

31. The system according to claim 26, wherein the difference between the first and second polarizing angles is 90 degrees.