Identifying a target touch region of a touch-sensitive surface based on an image

• US 10,324,563 B2
• Filed: 09/24/2013
• Issued: 06/18/2019
• Est. Priority Date: 09/24/2013
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Tretter, Daniel R, Kang, Jinman, Tan, Kar Han, Hong, Wei, Suggs, Bradley N, Short, David Bradley, Sievert, Otto
• Primary Examiner(s): Amadiz, Rodney
• Application Number:
  US15/024,100
• Publication Number:
  US 20160231862A1
• Time in Patent Office: 2,093 Days
• CPC Class Codes: G06F 2203/04106 Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/04186 Touch location disambiguation
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G09G 2360/144 the light being ambient light
• CPC Class Codes: G09G 3/002 to project the image of a two-dimensional display, such as an array of light emitting or modulating elements or a CRT

Abstract

Examples disclosed herein relate to identifying a target touch region of a touch-sensitive surface based on an image. Examples include a touch input detected at a location of a touch-sensitive surface, an image representing an object disposed between a camera that captures the image and the touch-sensitive surface, identifying a target touch region of a touch-sensitive surface based on an image, and rejecting the detected touch input when the location of the detected touch input is not within any of the at least one identified target touch region of the touch-sensitive surface.

Claims

1. A computing system comprising:

a camera to capture an image representing an object disposed between the camera and a touch-sensitive surface, the touch-sensitive surface to detect a touch input at a location of the touch-sensitive surface;

a processor; and

a non-transitory storage medium storing instructions executable on the processor to;

identify, based on a characteristic of the object represented in the image, a target touch region of the touch-sensitive surface, the target touch region for receiving touch input, wherein the identifying of the target touch region comprises determining that a first region of the touch-sensitive surface is within a threshold distance from a target portion of the object, determining that a second region of the touch-sensitive surface is outside the threshold distance from the target portion of the object, and including the first region in the target touch region and excluding the second region from the target touch region,

determine that the location of the detected touch input is not within the identified target touch region of the touch-sensitive surface, and

reject the detected touch input in response to the determining.

2. The computing system of claim 1, wherein the instructions are executable on the processor to:

identify a segmentation boundary at least partially representing an outer edge of the object based on the image,

wherein the identifying of the target touch region of the touch-sensitive surface is based on the characteristic of the object corresponding to the segmentation boundary.

3. The computing system of claim 2, comprising:

a plurality of cameras of different types to capture a plurality of images, each respective image of the plurality of images captured by a respective camera of the plurality of cameras and representing the object disposed between the touch-sensitive surface and the respective camera,

wherein the identifying of the segmentation boundary is based on each of the plurality of images; and

wherein the plurality of cameras includes the camera and the plurality of images includes the image.

4. The computing system of claim 3, wherein:

each of the plurality of cameras is a different one of an RGB camera, an infrared (IR) camera, a depth camera, and a gesture camera, and

the identified target touch region of the touch-sensitive surface is smaller than the full area of the touch-sensitive surface.

5. The computing system of claim 1, wherein:

the object is a hand, and

the identified target touch region includes a respective region of the touch-sensitive surface over which a fingertip of the hand is disposed and excludes another region of the surface that is outside of the threshold distance from the fingertip.

6. The computing system of claim 1, wherein the object comprises a hand or a stylus, and wherein the identifying of the target touch region comprises determining that the target touch region is within the threshold distance of an identified fingertip of the hand or an identified tip of the stylus represented in the image.

7. The computing system of claim 1, wherein the identifying of the target touch region based on the characteristic of the object represented in the image comprises identifying the target touch region based on at least one selected from among:

a position of the object, an orientation of the object, and a shape of the object.

8. The computing system of claim 1, wherein a location of the identified target touch region varies with different characteristics of the object represented in the image.

9. A non-transitory machine-readable storage medium comprising instructions executable by a processing resource of a computing system comprising a touch-sensitive surface and a camera disposed above and pointed at the touch-sensitive surface, the instructions executable to:

acquire, from the touch-sensitive surface, a location of a touch input detected by the touch-sensitive surface;

acquire, from the camera, an image representing an object disposed between the camera and the touch-sensitive surface;

identify a current touch input scenario for the computing system of a plurality of touch input scenarios for the computing system, including a hand input scenario and a stylus input scenario;

identify a target touch region of the touch-sensitive surface based on a location of a portion of the object as represented in the image, wherein the target touch region is identified differently when the hand input scenario is identified than when the stylus input scenario is identified, wherein the identifying of the target touch region comprises determining that a first region of the touch-sensitive surface is within a threshold distance from the portion of the object, determining that a second region of the touch-sensitive surface is outside the threshold distance from the portion of the object, and including the first region in the target touch region and excluding the second region from the target touch region;

determine whether the location of the touch input is within the identified target touch region of the touch-sensitive surface; and

reject the touch input in response to determining that the location of the touch input is not within the identified target touch region of the touch-sensitive surface.

10. The non-transitory machine-readable storage medium of claim 9, wherein the instructions to reject comprise instructions to:

receive touch rejection information from another component of the computing system; and

determine whether the received touch rejection information indicates that the detected touch input is to be rejected.

11. The non-transitory machine-readable storage medium of claim 10, wherein the instructions to reject comprise instructions to:

provide the detected touch input to a touch input processing function of the computing system in response to determining that the location of the touch input is within the identified touch region of the touch-sensitive surface and determining that the touch rejection information does not indicate that the detected touch input is to be rejected.

12. The non-transitory machine-readable storage medium of claim 10, wherein:

the instructions to receive the touch rejection information comprise instructions to receive the touch rejection information from an application that is in focus on a display of the computing system, the touch rejection information indicating a non-target touch region of the touch-sensitive surface, and

the instructions to reject comprise instructions to reject the touch input in response to determining that the location of the touch input is within the non-target touch region of the surface.

13. The non-transitory machine-readable storage medium of claim 9, wherein the identified current touch input scenario is the hand input scenario and the object is a hand, and the instructions to identify the target touch region comprise instructions to, in response to the identified hand input scenario:

identify a segmentation boundary at least partially representing an outer edge of the hand based on the image,

extract a skeletal model for the hand based on characteristics of the object corresponding to the segmentation boundary,

identify respective locations of fingertips of the hand relative to the touch-sensitive surface, based on the skeletal model, and

for each respective fingerprint location of the identified locations of the fingertips, identify a region of the touch-sensitive surface that is within the threshold distance of the respective fingertip location as a target touch region.

14. The non-transitory machine-readable storage medium of claim 9, wherein the identified current touch input scenario is the stylus input scenario, the object comprises a stylus, and the instructions to identify the target touch region comprise instructions to, in response to identified the stylus input scenario:

identify, based on characteristics of the object represented in the image, a location of a tip of the stylus relative to the touch-sensitive surface, and

identify, as the target touch region, a region of the touch-sensitive surface that is within the threshold distance of the identified location of the tip.

15. The non-transitory machine-readable storage medium of claim 9, wherein the identified current touch input scenario is an object placement scenario, the object is disposed on the touch-sensitive surface, and the instructions to identify the target touch region comprise instructions to, in response to the identified object placement scenario:

identify a given region of the touch-sensitive surface including surface locations on which the object is disposed, and

identify, as a non-target touch region, the given region and a further region outside of and bordering the given region.

16. The non-transitory machine-readable storage medium of claim 9, wherein a location of the identified target touch region varies with different locations of the portion of the object represented in the image.

17. A method comprising:

detecting, with a touch-sensitive surface, a touch input at a location of the touch-sensitive surface;

capturing, with a camera of a computer system disposed above and pointed at the touch-sensitive surface, an image representing an object disposed between the camera and the touch-sensitive surface;

identifying a target touch region of the touch-sensitive surface based on a characteristic of the object represented in the image, wherein the identifying of the target touch region comprises determining that a first region of the touch-sensitive surface is within a threshold distance from a target portion of the object, determining that a second region of the touch-sensitive surface is outside the threshold distance from the target portion of the object, and including the first region in the target touch region and excluding the second region from the target region;

comparing the location of the touch input to the identified target touch region of the touch-sensitive surface; and

in response to the comparing indicating that the location of the touch input is not within the identified target touch region, rejecting the detected touch input.

18. The method of claim 17, further comprising:

detecting a trigger to initiate a touch rejection process, wherein the capturing is performed in response to the detecting of the trigger; and

in response to the comparing indicating that the location of the touch input is within the identified target touch region, providing the detected touch input to a touch input processing function of the computer system.

19. The method of claim 17, wherein:

the object is a hand, and

the first region is within the threshold distance of a fingertip of the hand.

20. The method of claim 17, wherein the identifying of the target touch region based on the characteristic of the object represented in the image comprises identifying the target touch region based on at least one selected from among:

a position of the object, an orientation of the object, and a shape of the object.

21. The method of claim 17, wherein a location of the identified target touch region varies with different characteristics of the object represented in the image.