SYSTEM AND METHOD FOR PREDICTING A TOUCH POSITION OF A POINTER ON A TOUCH-ENABLED UNIT OR DETERMINING A POINTING DIRECTION IN 3D SPACE

• US 20190004667A1
• Filed: 06/08/2018
• Published: 01/03/2019
• Est. Priority Date: 07/03/2017
• Status: Active Grant

• Current Assignee: Aptiv Technologies AG (Aptiv PLC)
• Original Assignee: Delphi Technologies LLC (BorgWarner Incorporated)
• Inventors: Barth, Alexander
• Granted Patent: US 10,620,752 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 2203/04108 Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0418 for error correction or compensation, e.g. based on parallax, calibration or alignment
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

A system for predicting a touch position of a pointer on a touch-enabled unit includes a 3D imaging unit, a processing unit coupled to the 3D imaging unit, and a touch-enabled unit coupled to the processing unit. The 3D imaging unit is configured to monitor an interaction zone in front of the touch-enabled unit. The processing unit includes a prediction module that is configured to predict where a pointer that approaches the touch-enabled unit and is monitored by the 3D imaging unit will touch the touch-enabled unit, and a calibration module that is configured to generate at least one calibration parameter by comparing the predicted touch position to the actual touch position of the pointer detected by the touch-enabled unit and to transfer the at least one calibration parameter to the prediction module in order to calibrate the prediction module.

Claims

1. A system (10) for predicting a touch position of a pointer (21) on a touch-enabled unit (13), said system comprising:

a 3D imaging unit (11);

a processing unit (12) coupled to the 3D imaging unit (11);

a touch-enabled unit (13) coupled to the processing unit (12), wherein the 3D imaging unit (11) is configured to monitor an interaction zone in front of the touch-enabled unit (13), and the processing unit (12) comprises a prediction module (14) that is configured to predict where a pointer (21) that approaches the touch-enabled unit (13) and is monitored by the 3D imaging unit (11) will touch the touch-enabled unit (11); and

a calibration module (15) that is configured to generate at least one calibration parameter (31) by comparing the predicted touch position (28) to the actual touch position (29) of the pointer (21) detected by the touch-enabled unit (13) and to transfer the at least one calibration parameter (31) to the prediction module (14) to calibrate the prediction module (14).

2. The system (10) as claimed in claim 1, wherein the prediction module (14) is configured to provide coordinates of the predicted touch position (28) in a coordinate system attached to the 3D imaging unit (11) and the touch-enabled unit (13) is configured to provide coordinates of the actual touch position (29) in a coordinate system attached to the touch-enabled unit (13), and wherein the at least one calibration parameter (31) comprises translating and/or rotating and/or scaling parameters for at least one of the coordinate systems attached to the 3D imaging unit (11) and attached to the touch-enabled unit (13) such that a deviation between the coordinates of the predicted touch position (28) in the coordinate system attached to the 3D imaging unit (11) and the coordinates of the actual touch position (29) in the coordinate system attached to the touch-enabled unit (13) is minimized.

3. The system (10) as claimed in claim 1, wherein a 3D image of a scene captured by the 3D image unit (11) comprises a 2D intensity image of the scene and a depth map of the scene, and wherein the at least one calibration parameter (31) comprises an offset for the values of the depth map.

4. The system (10) as claimed in one of the preceding claims, wherein the calibration module (15) is configured to use a plurality of pairs for the generation of the at least one calibration parameter (31), wherein each of the pairs contains a predicted touch position (28) and a corresponding actual touch position (29).

5. The system (10) as claimed in claim 4, wherein the calibration module (15) is configured to use only those pairs of the plurality of pairs for the generation of the at least one calibration parameter (31) that contain a predicted touch position (28) that was predicted when the distance of the pointer (21) from the touch-enabled unit (13) was in a predetermined distance range, and/or

wherein the calibration module (15) is configured to use only those pairs of the plurality of pairs for the generation of the at least one calibration parameter (31) that contain a predicted touch position (28) that was predicted during a predetermined time interval before the pointer (21) touched the touch-enabled unit (13).

6. The system (10) as claimed in claim 4, wherein the calibration module (15) is configured to weight each pair with a respective uncertainty factor before using the pairs for the generation of the at least one calibration parameter (31), wherein the uncertainty factor depends on the distance of the pointer (21) from the touch-enabled unit (13) when the prediction module (14) predicted the touch position of the respective pair.

7. The system (10) as claimed in claim 1, wherein the calibration module (15) is configured to request a user to touch at least one predetermined position on the touch-enabled unit (13) with the pointer (21), wherein the prediction module (14) is configured to predict where the pointer (21) that approaches the touch-enabled unit (13) to touch the at least one predetermined position will touch the touch-enabled unit (13), and wherein the calibration module (15) is configured to generate the at least one calibration parameter (31) by comparing the predicted touch position (28) to the at least one predetermined position.

8. The system (10) as claimed in claim 1, said system further comprising a user identification module (32) that is configured to identify a user using the system (10), wherein the calibration module (15) is configured to generate the at least one calibration parameter (31) for a respective user and to store the at least one calibration parameter (31) in a memory, wherein the at least one calibration parameter is transferred to the prediction module (14) from the memory if the user identification module (32) identifies the respective user using the system (10).

9. A method for predicting a touch position of a pointer (21) on a touch-enabled unit (13), said method comprising:

monitoring an interaction zone in front of the touch-enabled unit (13) by using a 3D imaging unit (11);

predicting by using a prediction method where a pointer (21) that approaches the touch-enabled unit (13) and is monitored by the 3D imaging unit (11) will touch the touch-enabled unit (13); and

generating at least one calibration parameter (31) by comparing the predicted touch position (28) to the actual touch position (29) of the pointer (21) detected by the touch-enabled unit (13), and using the at least one calibration parameter (31) to calibrate the prediction method.

10. A system (50) for determining a pointing direction of a pointer (21) in 3D space, said system comprising:

a 3D imaging unit (11), and

a processing unit (51) coupled to the 3D imaging unit (11), wherein the processing unit (51) comprises a determination module (52) that is configured to determine a pointing direction (55) of a pointer (21) of a user that is monitored by the 3D imaging unit (11) when the user has the intention to point to an object (54), and a calibration module (53) that is configured to generate at least one calibration parameter (56) by comparing the determined pointing direction (55) to the position of the object (54) and to transfer the at least one calibration parameter (56) to the determination module (52) in order to calibrate the determination module (52).

11. The system (50) as claimed in claim 10, further comprising an input unit (57) for inputting a command by the user, wherein the calibration module (53) is configured to assume that the user has the intention to point to the object (54) if the command input by the user in the input unit (57) is related to the object (54).

12. The system (50) as claimed in claim 10, wherein the calibration module (53) is configured to request the user to point with the pointer (21) to a predetermined object (54), wherein the determination module (52) is configured to determine the pointing direction (55) of the pointer (21) when pointing to the predetermined object (54), and wherein the calibration module (53) is configured to generate the at least one calibration parameter (56) by comparing the determined pointing direction (55) to the position of the predetermined object (54).

13. The system (50) as claimed in claim 10, wherein the determination module (52) is configured to determine the pointing direction (55) of the pointer (21) pointing to the same object (54) for a plurality of times and to determine a peak in the distribution of the determined pointing directions (55), and the calibration module (53) is configured to generate the at least one calibration parameter (56) by comparing the pointing direction (55) corresponding to the peak in the distribution of the determined pointing directions (55) to the position of the object (54).

14. The system (50) as claimed in claim 10, wherein the 3D imaging unit (11) is configured to capture consecutive images of a scene, wherein at least a first one of the images comprises a 2D intensity image and a depth map, and at least a second one of the images comprise only a 2D intensity image, but not a depth map, wherein the scene comprises a display showing an element and the determination module (52) is configured to determine the coordinates of the element in a coordinate system attached to the 3D imaging unit (11) and the calibration module (53) is configured to generate at least one further calibration parameter for calibrating the coordinate system attached to the 3D imaging unit (11) by comparing the coordinates of the element in the coordinate system attached to the 3D imaging unit (11) to predetermined coordinates of the element in a fixed coordinate system.

15. A method for determining a pointing direction of a pointer (21) in 3D space, said method comprising:

monitoring a pointer (21) of a user by a 3D imaging unit (11) when the user has the intention to point to an object (54);

determining a pointing direction (55) of the pointer (21) by using a determination method; and

generating at least one calibration parameter (56) by comparing the determined pointing direction (55) to the position of the object (54) and using the at least one calibration parameter (56) to calibrate the determination method.