Enabling physical controls on an illuminated surface

• US 10,488,948 B2
• Filed: 03/14/2013
• Issued: 11/26/2019
• Est. Priority Date: 03/14/2012
• Status: Active Grant

• Current Assignee: Texas Instruments, Inc.
• Original Assignee: Texas Instruments, Inc.
• Inventors: Sharma, Vinay, Sharp, Frank
• Primary Examiner(s): Gupta, Parul H
• Application Number:
  US13/828,526
• Publication Number:
  US 20130241822A1
• Time in Patent Office: 2,448 Days
• Field of Search: None
• CPC Class Codes: G06F 3/02 Input arrangements using manually operated switches, e.g. using keyboards or dials
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0362 with detection of 1D translations or rotations of an operating part of the device, e.g. scroll wheels, sliders, knobs, rollers or belts
• CPC Class Codes: G06F 3/039 Accessories therefor, e.g. mouse pads
• CPC Class Codes: G06F 3/0393 Accessories for touch pads or touch screens, e.g. mechanical guides added to touch screens for drawing straight lines, hard keys overlaying touch screens or touch pads
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386

Abstract

A method for enabling physical controls in a digital system is provided that includes receiving an image of an illuminated surface in the digital system, wherein the image is captured by a camera in the digital system, determining a state of a physical control mounted on the illuminated surface by analyzing the image; and outputting an indication of the state of the physical control.

Claims

1. A method comprising:

projecting, by an infrared light source, infrared light through a rear surface of an illuminated surface of a touch screen display,

receiving a reflection of the projected infrared light as an infrared image, wherein the infrared image is captured by an infrared camera;

identifying a physical control attached to the illuminated surface of the touch screen display by analyzing a reflection of a first binary pattern of the physical control in the infrared image, the physical control having a non-rotatable portion and a rotatable portion, a back surface of the non-rotatable portion of the physical control comprising the first binary pattern and a back surface of the rotatable portion of the physical control comprising a second binary pattern;

determining a rotational state of the physical control attached to the illuminated surface of the touch screen display by analyzing a reflection of the second binary pattern of the physical control in the infrared image; and

outputting an indication of the rotational state of the physical control.

2. The method of claim 1, further comprising:

outputting an identity indicator for the physical control in association with the rotational state.

3. The method of claim 2, wherein identifying the physical control comprises:

determining a set of points that lie along the non-rotatable portion of the physical control;

extracting a set of pixel values corresponding to the set of points from the infrared image; and

determining the identity indicator based on the set of pixel values.

4. The method of claim 3, wherein determining a set of points comprises:

computing the set of points from a sub-image of the infrared image of the illuminated surface captured by the infrared camera, the sub-image corresponding to a pre-determined bounding box specified for the physical control.

5. The method of claim 1, wherein determining a rotational state of a physical control comprises:

determining a set of points that lie along the rotatable portion of the physical control;

extracting a set of pixel values corresponding to the set of points from the infrared image; and

determining the rotational state based on the set of pixel values.

6. The method of claim 5, wherein determining a set of points comprises:

computing the set of points from a sub-image of the infrared image of the illuminated surface captured by the infrared camera, the sub-image corresponding to a pre-determined bounding box specified for the physical control.

7. The method of claim 5, wherein determining the rotational state comprises:

locating a subset of consecutive pixel values of like intensity distinct from other pixel values in the set of pixel values, wherein the subset corresponds to a state indicator encoded on the rotatable portion of the physical control; and

determining the rotational state from a position of the subset corresponding to the state indicator in the set of pixel values.

8. The method of claim 1, wherein the physical control is a knob, and the rotational state of the physical control is an angle of rotation of the knob.

9. A digital system, comprising:

a touch screen display having an illuminated surface;

a physical control mounted on the illuminated surface of the touch screen display, the physical control having a non-rotatable portion and a rotatable portion, a back surface of the non-rotatable portion of the physical control comprising a first binary pattern and a back surface of the rotatable portion of the physical control comprising a second binary pattern;

an infrared light source configured to project infrared light through a rear surface of the illuminated surface of the touch screen display;

an infrared camera positioned to capture a reflection of the projected infrared light as an infrared image; and

one or more processors configured to;

receive the infrared image captured by the infrared camera;

identify the physical control by analyzing a reflection of the first binary pattern of the physical control in the infrared image;

determine a rotational state of the physical control by analyzing a reflection of the second binary pattern of the physical control in the infrared image; and

output an indication of the rotational state of the physical control.

10. The digital system of claim 9, wherein the one or more processors are further configured to:

output an identity indicator for the physical control in association with the rotational state.

11. The digital system of claim 10, wherein the one or more processors are further configured to:

determine a set of points that lie along a fixed portion of the physical control;

extract a set of pixel values corresponding to the set of points from the infrared image; and

determine the identity indicator based on the set of pixel values.

12. The digital system of claim 11, wherein the one or more processors are further configured to:

compute the set of points from a sub-image of the infrared image of the illuminated surface captured by the infrared camera, the sub-image corresponding to a pre-determined bounding box specified for the physical control.

13. The digital system of claim 9, wherein the one or more processors are further configured to:

determine a set of points that lie along the rotatable part of the physical control;

extract a set of pixel values corresponding to the set of points from the infrared image; and

determine the rotational state based on the set of pixel values.

14. The digital system of claim 13, wherein the one or more processors are further configured to:

compute the set of points from a sub-image of the infrared image of the illuminated surface captured by the infrared camera, the sub-image corresponding to a pre-determined bounding box specified for the physical control.

15. The digital system of claim 13, wherein the one or more processors are further configured to:

locate a subset of consecutive pixel values of like intensity distinct from other pixel values in the set of pixel values, wherein the subset corresponds to a state indicator encoded on the rotatable part of the physical control; and

determine the rotational state from a position of the subset corresponding to the state indicator in the set of pixel values.

16. The system of claim 9, wherein the physical control is a knob, and the rotational state of the physical control is an angle of rotation of the knob.

17. A non-transitory computer readable medium storing instructions that, when executed by a processor comprised in a digital system, cause the digital system to:

project an infrared light through a rear surface of an illuminated surface of a touch screen display;

receive a reflection of the projected infrared light as an infrared image;

identify a physical control attached to the illuminated surface of the touch screen display by analyzing a reflection of a first binary pattern of the physical control in the infrared image, the physical control having a non-rotatable portion and a rotatable portion, a back surface of the non-rotatable portion of the physical control comprising the first binary pattern and a back surface of the rotatable portion of the physical control comprising a second binary pattern;

determine a rotational state of a physical control by analyzing a reflection of the second binary pattern of the physical control in the infrared image; and

output an indication of the rotational state of the physical control.

18. The computer readable medium of claim 17, wherein the instructions further cause the digital system to:

determine a set of points that lie along the rotatable portion of the physical control;

extract a set of pixel values corresponding to the set of points from the infrared image; and

determine the rotational state based on the set of pixel values.