ELECTRONIC SYSTEM WITH GESTURE CALIBRATION MECHANISM AND METHOD OF OPERATION THEREOF

• US 20160188109A1
• Filed: 12/30/2014
• Published: 06/30/2016
• Est. Priority Date: 12/30/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Wang, Rongrong, Treskunov, Anton
• Granted Patent: US 10,452,195 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0418 for error correction or compensation, e.g. based on parallax, calibration or alignment

Abstract

An electronic system includes a display interface configured to display a calibration point; a sensor, coupled to the display interface, and configured to capture a first gesture directed at the calibration point from a first position; capture a second gesture directed at the calibration point from a second position, wherein the second position is different from the first position; and a control unit, coupled to the display interface, and configured to display a position indicator on the display interface based on a directive gesture captured by the sensor, the first gesture, and the second gesture.

Claims

1. An electronic system comprising:

a display interface configured to display a calibration point;

a sensor, coupled to the display interface, and configured to;

capture a first gesture directed at the calibration point from a first position;

capture a second gesture directed at the calibration point from a second position, wherein the second position is different from the first position; and

a control unit, coupled to the display interface, and configured to;

display a position indicator on the display interface based on a directive gesture captured by the sensor, the first gesture, and the second gesture.

2. The system as claimed in claim 1 wherein the control unit is further configured to generate a transformation matrix based on the calibration point in a display coordinate system and an intersection in a sensor coordinate system.

3. The system as claimed in claim 1 wherein the control unit is further configured to:

generate a first calibration vector for representing the first gesture directed at the calibration point from the first position;

generate a second calibration vector for representing the second gesture directed at the calibration point from the second position; and

determine an intersection based on the first calibration vector and the second calibration vector.

4. The system as claimed in claim 1 wherein the control unit is further configured to:

determine a first appendage position of the directive gesture in a sensor coordinate system;

determine a second appendage position of the directive gesture in the sensor coordinate system;

calculate a transformed first point in a display coordinate system by applying a transformation matrix to the first appendage position;

calculate a transformed second point in the display coordinate system by applying the transformation matrix to the second appendage position;

generate a ray casting vector based on the transformed first point and the transformed second point; and

generate the position indicator based on an intersection of the ray casting vector with a display plane in the display coordinate system.

5. The system as claimed in claim 1 wherein the control unit is further configured to:

determine a reference point of the directive gesture in a sensor coordinate system;

calculate a transformed referencing point of the directive gesture in a display coordinate system by applying a transformation matrix to the reference point; and

generate an orthographic projection of the directive gesture on the display interface based on the transformed referencing point in the display coordinate system.

6. The system as claimed in claim 1 wherein the directive gesture is a typing gesture for generating the position indicator over a keyboard key.

7. A method of operation of an electronic system comprising:

displaying, with a display interface, a calibration point;

capturing, with a sensor coupled to the display interface, a first gesture directed at the calibration point from a first position;

capturing a second gesture directed at the calibration point from a second position, wherein the second position is different from the first position; and

displaying, with a control unit coupled to the display interface, a position indicator on the display interface based on a directive gesture captured by the sensor, the first gesture, and the second gesture.

8. The method as claimed in claim 7 further comprising generating a transformation matrix based on the calibration point in a display coordinate system and an intersection in a sensor coordinate system.

9. The method as claimed in claim 7 further comprising:

generating a first calibration vector for representing the first gesture directed at the calibration point from the first position;

generating a second calibration vector for representing the second gesture directed at the calibration point from the second position; and

determining an intersection based on the first calibration vector and the second calibration vector.

10. The method as claimed in claim 7 further comprising:

determining a first appendage position of the directive gesture in a sensor coordinate system;

determining a second appendage position of the directive gesture in the sensor coordinate system;

calculating a transformed first point in a display coordinate system by applying a transformation matrix to the first appendage position;

calculating a transformed second point in the display coordinate system by applying the transformation matrix to the second appendage position;

generating a ray casting vector based on the transformed first point and the transformed second point; and

generating the position indicator based on an intersection of the ray casting vector with a display plane in the display coordinate system.

11. The method as claimed in claim 7 further comprising:

determining a reference point of the directive gesture in a sensor coordinate system;

calculating a transformed referencing point of the directive gesture in a display coordinate system by applying a transformation matrix to the reference point; and

generating an orthographic projection of the directive gesture on the display interface based on the transformed referencing point in the display coordinate system.

12. The method as claimed in claim 7 wherein the directive gesture is a typing gesture for generating the position indicator over a keyboard key.

13. A non-transitory computer readable medium including instructions for execution, comprising:

displaying a calibration point;

capturing a first gesture directed at the calibration point from a first position;

capturing a second gesture directed at the calibration point from a second position, wherein the second position is different from the first position; and

displaying a position indicator on a display interface based on a directive gesture captured by a sensor, the first gesture, and the second gesture.

14. The non-transitory computer readable medium as claimed in claim 13 further comprising generating a transformation matrix based on the calibration point in a display coordinate system and an intersection in a sensor coordinate system.

15. The non-transitory computer readable medium as claimed in claim 13 further comprising:

generating a first calibration vector for representing the first gesture directed at the calibration point from the first position;

generating a second calibration vector for representing the second gesture directed at the calibration point from the second position; and

determining an intersection based on the first calibration vector and the second calibration vector.

16. The non-transitory computer readable medium as claimed in claim 13 further comprising:

determining a first appendage position of the directive gesture in a sensor coordinate system;

determining a second appendage position of the directive gesture in the sensor coordinate system;

calculating a transformed first point in a display coordinate system by applying a transformation matrix to the first appendage position;

calculating a transformed second point in the display coordinate system by applying the transformation matrix to the second appendage position;

generating a ray casting vector based on the transformed first point and the transformed second point; and

generating the position indicator based on an intersection of the ray casting vector with a display plane in the display coordinate system.

17. The non-transitory computer readable medium as claimed in claim 13 further comprising:

determining a reference point of the directive gesture in a sensor coordinate system;

calculating a transformed referencing point of the directive gesture in a display coordinate system by applying a transformation matrix to the reference point; and

generating an orthographic projection of the directive gesture on the display interface based on the transformed referencing point in the display coordinate system.

18. The non-transitory computer readable medium as claimed in claim 13 wherein the directive gesture is a typing gesture for generating the position indicator over a keyboard key.

19. An electronic system comprising:

a sensor configured to capture a contact point between a calibration gesture and a display interface; and

a control unit, coupled to the sensor, and configured to display a position indicator on the display interface based on a directive gesture captured by the sensor and the contact point.

20. The system as claimed in claim 19 wherein the control unit is further configured to generate a transformation matrix based on a calibration point in a display coordinate system and the contact point in a sensor coordinate system.

21. The system as claimed in claim 19 wherein the display interface is further configured to display a calibration point for guiding the calibration gesture prior to the sensor capturing the contact point.

22. The system as claimed in claim 19 wherein the control unit is further configured to:

determine a first appendage position of the directive gesture in a sensor coordinate system;

determine a second appendage position of the directive gesture in the sensor coordinate system;

calculate a transformed first point in a display coordinate system by applying a transformation matrix to the first appendage position;

calculate a transformed second point in the display coordinate system by applying the transformation matrix to the second appendage position;

generate a ray casting vector based on the transformed first point and the transformed second point; and

generate the position indicator based on an intersection of the ray casting vector with a display plane in the display coordinate system.

23. The system as claimed in claim 19 wherein the control unit is further configured to:

determine a reference point of the directive gesture in a sensor coordinate system;

calculate a transformed referencing point of the directive gesture in a display coordinate system by applying a transformation matrix to the reference point; and

generate an orthographic projection of the directive gesture on the display interface at the transformed referencing point in the display coordinate system.

24. The system as claimed in claim 19 wherein the directive gesture is a typing gesture for generating the position indicator over a keyboard key.

25. A method of operation of an electronic system comprising:

capturing, with a sensor, a contact point between a calibration gesture and a display interface; and

displaying, with a control unit coupled to the sensor, a position indicator on the display interface based on a directive gesture captured by the sensor and the contact point.

26. The method as claimed in claim 25 further comprising generating a transformation matrix based on a calibration point in a display coordinate system and the contact point in a sensor coordinate system.

27. The method as claimed in claim 25 further comprising displaying a calibration point for guiding the calibration gesture prior to the sensor capturing the contact point.

28. The method as claimed in claim 25 further comprising:

determining a first appendage position of the directive gesture in a sensor coordinate system;

determining a second appendage position of the directive gesture in the sensor coordinate system;

calculating a transformed first point in a display coordinate system by applying a transformation matrix to the first appendage position;

calculating a transformed second point in the display coordinate system by applying the transformation matrix to the second appendage position;

generating a ray casting vector based on the transformed first point and the transformed second point; and

generating the position indicator based on an intersection of the ray casting vector with a display plane in the display coordinate system.

29. The method as claimed in claim 25 further comprising:

determining a reference point of the directive gesture in a sensor coordinate system;

calculating a transformed referencing point of the directive gesture in a display coordinate system by applying a transformation matrix to the reference point; and

generating an orthographic projection of the directive gesture on the display interface at the transformed referencing point in the display coordinate system.

30. The method as claimed in claim 25 wherein the directive gesture is a typing gesture for generating the position indicator over a keyboard key.

31. A non-transitory computer readable medium including instructions for execution, comprising:

capturing a contact point between a calibration gesture and a display interface; and

displaying a position indicator on a display interface based on a directive gesture captured by a sensor and the contact point.

32. The non-transitory computer readable medium as claimed in claim 31 further comprising generating a transformation matrix based on the calibration point in a display coordinate system and an intersection in a sensor coordinate system.

33. The non-transitory computer readable medium as claimed in claim 31 further comprising:

generating a first calibration vector for representing the first gesture directed at the calibration point from the first position;

generating a second calibration vector for representing the second gesture directed at the calibration point from the second position; and

determining an intersection based on the first calibration vector and the second calibration vector.

34. The non-transitory computer readable medium as claimed in claim 31 further comprising:

determining a first appendage position of the directive gesture in a sensor coordinate system;

determining a second appendage position of the directive gesture in the sensor coordinate system;

calculating a transformed first point in a display coordinate system by applying a transformation matrix to the first appendage position;

calculating a transformed second point in the display coordinate system by applying the transformation matrix to the second appendage position;

generating a ray casting vector based on the transformed first point and the transformed second point; and

generating the position indicator based on an intersection of the ray casting vector with a display plane in the display coordinate system.

35. The non-transitory computer readable medium as claimed in claim 31 further comprising:

determining a reference point of the directive gesture in a sensor coordinate system;

calculating a transformed referencing point of the directive gesture in a display coordinate system by applying a transformation matrix to the reference point; and

generating an orthographic projection of the directive gesture on the display interface based on the transformed referencing point in the display coordinate system.

36. The non-transitory computer readable medium as claimed in claim 31 wherein the directive gesture is a typing gesture for generating the position indicator over a keyboard key.