Touch discrimination

• US 8,432,366 B2
• Filed: 03/03/2009
• Issued: 04/30/2013
• Est. Priority Date: 03/03/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Hodges, Stephen E., Benko, Hrvoje, Sands, Ian M., Butler, David Alexander, Izadi, Shahram, Kunz, William Ben, Hinckley, Kenneth P.
• Primary Examiner(s): Haley, Joseph
• Assistant Examiner(s): LEE, NICHOLAS J
• Application Number:
  US12/396,701
• Publication Number:
  US 20100225595A1
• Time in Patent Office: 1,519 Days
• Field of Search: 345172-179
• US Class Current: 345/173
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/04186 Touch location disambiguation
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386

Abstract

The claimed subject matter provides a system and/or a method that facilitates distinguishing input among one or more users in a surface computing environment. A variety of information can be obtained and analyzed to infer an association between a particular input and a particular user. Touch point information can be acquired from a surface wherein the touch point information relates to a touch point. In addition, one or more environmental sensors can monitor the surface computing environment and provide environmental information. The touch point information and the environmental information can be analyzed to determine direction of inputs, location of users, and movement of users and so on. Individual analysis results can be correlated and/or aggregated to generate a inference of association between a touch point and user.

Claims

1. A system comprising:

one or more processors;

one or more computer-readable storage media including instructions executable by the one or more processors;

a touch input component to;

obtain touch point information from at least one surface in a surface computing environment, and

determine a vector of incidence based on the touch point information;

one or more sensors that collect information related to a surrounding environment of the at least one surface that is used by a plurality of users;

a touch discrimination component to;

infer an association between a user of the plurality of users and a touch point based on at least one of the touch point information, accelerometer information obtained from a user device associated with the user, or environmental information, and

distinguish input received from the user relative to other users of the plurality of users;

a biometric sensor to identify a fingerprint based on the touch point information, to determine whether the fingerprint is associated with the user, and to associate the fingerprint with a new user when the fingerprint is unassociated with the user;

an overhead sensor analysis component to use overhead optical sensors to

acquire overhead environmental information, and

ascertain movements of at least one user of the plurality of users based on the overhead environmental information;

a facial analysis component to use one or more optical imaging sensors to determine a line of sight of at least one user of the plurality of users; and

a threshold component to apply one or more thresholds to an image when the image is included in the touch point information;

a fusion component to apply, based on a context of the surface, weightings to the touch point information and to the environmental information.

2. The system of claim 1, further comprising a touch point analysis component that evaluates the touch point information to determine a direction from which the touch point originated.

3. The system of claim 2, the threshold component operable to generate a binary image by applying the one or more thresholds to the image.

4. The system of claim 3, further comprising a connection component that utilized connected components analysis techniques to associate portions of the binary image.

5. The system of claim 1, further comprising a vector analysis component that evaluates the touch point information to ascertain a vector of incidence associated with the touch point.

6. The system of claim 1, in which the touch point information includes at least one of a location of the touch point, a size of the touch point, a shape of the touch point, or an incoming force of the touch point.

7. The system of claim 1, further comprising an environment analysis component that evaluates the environmental information to determine a location and a movement associated with at least one user of the plurality of users.

8. The system of claim 7, further comprising a lateral sensor analysis component that evaluates side-ways environment information acquired by side-ways facing optical sensors, the lateral sensor analysis component determines a location of at least one user relative to the at least one surface.

9. The system of claim 7, in which the environment analysis component uses computer vision techniques on the overhead environmental information acquired by the overhead optical sensors to ascertain the movements.

10. The system of claim 7, in which the biometric analysis component evaluates biometric data associated with the touch point to facilitate distinguishing touch input among the plurality of users.

11. The system of claim 1, further comprising a context determination component that ascertains a set of context parameters that specify the context of the surface.

12. The system of claim 11, in which the set of context parameters includes at least one of a size of the surface, a shape of the surface, a number of users of the surface, or a number of applications receiving input from the surface.

13. The system of claim 11, further comprising a heuristic component that applies weightings to the touch point information and the environmental information in accordance with the set of context parameters.

14. The system of claim 11, further comprising an aggregation component that generates a composite inference of association from individual evaluations on the touch point information and the environmental information.

15. The system of claim 1, in which the touch discrimination component analyzes the accelerometer information obtained from a user device associated with the user to infer the association between the user and the touch point.

19. The system of claim 1, wherein the vector of incidence of the touch point is determined based on at least one of the touch point on the at least one surface, an image of objects and shadows immediately above the at least one surface, or a shape of the touch point.

20. The system of claim 1, further comprising at least one range finder device to:

detect when one or more users of the plurality of users approaches the at least one surface; and

determine a location of the one or more users relative to the at least one surface.

16. A method comprising:

obtaining touch point information associated with a touch point generated by a user interacting with a surface of a surface-based computing environment;

determining a vector of incidence of a touch point based on at least one of the touch point, an image of objects and shadows immediately above the surface, or a shape of the touch point;

collecting environmental information acquired by one or more environmental sensors that monitor the surface;

analyzing the touch point information and the environmental information;

inferring an association between the touch point and the user based upon analyzing one or more of the touch point information, accelerometer information obtained from a user device associated with the user, or the environmental information;

distinguishing input received from the user relative to other users of the surface-based computing environment;

identifying a fingerprint based on the touch point information;

determining whether the fingerprint is associated with the user;

associating the fingerprint with a new user when the fingerprint is unassociated with the user to enable distinguishing the touch point from other touch points generated by the other users interacting with the surface;

acquiring overhead environmental information using overhead optical sensors;

ascertaining movements of the user based on the overhead environmental information;

determining a line of sight of the user using the optical imaging sensors;

applying one or more thresholds to an image when the image is included in the touch point information; and

applying, based on a context of the surface, weightings to the touch point information and to the environmental information.

17. The method of claim 16, in which analyzing the touch point and the environmental information comprises applying weightings to the information in accordance with a context of the surface.

18. One or more computer readable media not consisting of signals, the one or more computer readable media including instructions executable by one or more processors to perform acts comprising:

obtaining touch point information associated with a touch point generated by a user interacting with a surface of a surface-based computing environment;

determining a vector of incidence of a touch point based on at least one of the touch point, an image of objects and shadows immediately above the surface, or a shape of the touch point;

collecting environmental information acquired by one or more environmental sensors that monitor the surface;

analyzing the touch point information and the environmental information;

inferring an association between the touch point and the user based upon analyzing one or more of the touch point information, accelerometer information obtained from a user device associated with the user, or the environmental information;

distinguishing input received from the user relative to other users of the surface-based computing environment;

identifying a fingerprint based on the touch point information;

determining whether the fingerprint is associated with the user;

associating the fingerprint with a new user when the fingerprint is unassociated with the user to enable distinguishing the touch point from other touch points generated by the other users interacting with the surface;

acquiring overhead environmental information using overhead optical sensors;

ascertaining movements of the user based on the overhead environmental information;

determining a line of sight of the user using the optical imaging sensors;

applying one or more thresholds to an image when the image is included in the touch point information; and

applying, based on a context of the surface, weightings to the touch point information and to the environmental information.