Electronic system with gesture calibration mechanism and method of operation thereof

• US 10,452,195 B2
• Filed: 12/30/2014
• Issued: 10/22/2019
• Est. Priority Date: 12/30/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Wang, Rongrong, Treskunov, Anton
• Primary Examiner(s): Blaufeld, Justin R.
• Assistant Examiner(s): Chung, Mong-Shune
• Application Number:
  US14/586,142
• Publication Number:
  US 20160188109A1
• Time in Patent Office: 1,757 Days
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0418 for error correction or compensation, e.g. based on parallax, calibration or alignment

Abstract

An electronic system includes a display interface configured to display a calibration point; a sensor, coupled to the display interface, and configured to capture a first gesture directed at the calibration point from a first position; capture a second gesture directed at the calibration point from a second position, wherein the second position is different from the first position; and a control unit, coupled to the display interface, and configured to display a position indicator on the display interface based on a directive gesture captured by the sensor, the first gesture, and the second gesture.

Claims

1. An electronic device comprising:

a curved display interface divided into multiple display regions grouped by curvature and configured to display a calibration point in each of the display regions to direct one or more calibration gestures to each of the display regions to tailor a calibration to different areas of the curved display interface;

a sensor, coupled to the curved display interface, and configured to;

capture a first gesture directed at the calibration point from a first position, wherein the first gesture is for calibrating the electronic device, and

capture a second gesture directed at the calibration point from a second position, wherein the second position is different from the first position, and wherein the second gesture is for calibrating the electronic device; and

a control unit, coupled to the curved display interface, and configured to;

generate a first calibration vector for representing the first gesture directed at the calibration point from the first position,

generate a second calibration vector for representing the second gesture directed at the calibration point from the second position,

determine an intersection based on the first calibration vector and the second calibration vector by determining a midpoint of a perpendicular intersecting segment connecting the first calibration vector and the second calibration vector,

display a position indicator on the curved display interface based on a directive gesture captured by the sensor, wherein the position indicator is based on the first gesture and the second gesture, and wherein the directive gesture is for manipulating a graphic or a user interface of the curved display interface, and

recalibrate the electronic device upon detecting a change in an orientation or a location of the sensor.

2. The electronic device as claimed in claim 1, wherein the control unit is further configured to generate a transformation matrix based on the calibration point in a display coordinate system and an intersection in a sensor coordinate system.

3. The electronic device as claimed in claim 1, wherein the first gesture and the second gesture include a user touching the calibration point displayed on the curved display interface using an appendage of the user.

4. The electronic device as claimed in claim 1, wherein the control unit is further configured to:

determine a first appendage position of the directive gesture in a sensor coordinate system;

determine a second appendage position of the directive gesture in the sensor coordinate system;

calculate a transformed first point in a display coordinate system by applying a transformation matrix to the first appendage position;

calculate a transformed second point in the display coordinate system by applying the transformation matrix to the second appendage position;

generate a ray casting vector based on the transformed first point and the transformed second point; and

generate the position indicator based on an intersection of the ray casting vector with a display plane in the display coordinate system.

5. The electronic device as claimed in claim 1, wherein the control unit is configured to:

determine a reference point of the directive gesture in a sensor coordinate system;

calculate a transformed referencing point of the directive gesture in a display coordinate system by applying a transformation matrix to the reference point; and

generate an orthographic projection of the directive gesture on the curved display interface based on the transformed referencing point in the display coordinate system.

6. The electronic device as claimed in claim 1, wherein the directive gesture is a typing gesture for generating the position indicator over a keyboard key.

7. A method of operation of an electronic device comprising:

displaying, with a curved display interface divided into multiple display regions grouped by curvature, a calibration point in each of the display regions to direct one or more calibration gestures to each of the display regions to tailor a calibration to different areas of the curved display interface;

capturing, with a sensor coupled to the curved display interface, a first gesture directed at the calibration point from a first position, wherein the first gesture is for calibrating the electronic device;

capturing a second gesture directed at the calibration point from a second position, wherein the second position is different from the first position, and wherein the second gesture is for calibrating the electronic device;

generating a first calibration vector for representing the first gesture directed at the calibration point from the first position;

generating a second calibration vector for representing the second gesture directed at the calibration point from the second position; and

determining an intersection based on the first calibration vector and the second calibration vector by determining a midpoint of a perpendicular intersecting segment connecting the first calibration vector and the second calibration vector;

displaying, with a control unit coupled to the curved display interface, a position indicator on the curved display interface based on a directive gesture captured by the sensor, wherein the position indicator is based on the first gesture and the second gesture, and wherein the directive gesture is for manipulating a graphic or a user interface of the curved display interface; and

recalibrating the electronic device upon detecting a change in an orientation or a location of the sensor.

8. The method as claimed in claim 7, further comprising generating a transformation matrix based on the calibration point in a display coordinate system and an intersection in a sensor coordinate system.

9. The method as claimed in claim 7, further comprising tailoring the calibration by capturing a user touching the calibration point displayed on the curved display interface using an appendage of the user.

10. The method as claimed in claim 7, further comprising:

determining a first appendage position of the directive gesture in a sensor coordinate system;

determining a second appendage position of the directive gesture in the sensor coordinate system;

calculating a transformed first point in a display coordinate system by applying a transformation matrix to the first appendage position;

calculating a transformed second point in the display coordinate system by applying the transformation matrix to the second appendage position;

generating a ray casting vector based on the transformed first point and the transformed second point; and

generating the position indicator based on an intersection of the ray casting vector with a display plane in the display coordinate system.

11. The method as claimed in claim 7, further comprising:

determining, a reference point of the directive gesture in a sensor coordinate system;

calculating a transformed referencing point of the directive gesture in a display coordinate system by applying a transformation matrix to the reference point; and

generating an orthographic projection of the directive gesture on the curved display interface based on the transformed referencing point in the display coordinate system.

12. The method as claimed in claim 7, wherein the directive gesture is a typing gesture for generating the position indicator over a keyboard key.

13. A non-transitory computer readable medium including instructions that, when executed by at least one processor of an electronic device, are configured to cause the at least one processor to:

display, with a curved display interface divided into multiple display regions grouped by curvature, a calibration point in each of the display regions to direct one or more calibration gestures to each of the display regions to tailor a calibration to different areas of the curved display interface;

capture, with a sensor coupled to the curved display interface, a first gesture directed at the calibration point from a first position, wherein the first gesture is for calibrating the electronic device;

capture a second gesture directed at the calibration point from a second position, wherein the second position is different from the first position, and wherein the second gesture is for calibrating the electronic device;

generate a first calibration vector for representing the first gesture directed at the calibration point from the first position;

generate a second calibration vector for representing the second gesture directed at the calibration point from the second position; and

determine an intersection based on the first calibration vector and the second calibration vector by determining a midpoint of a perpendicular intersecting segment connecting the first calibration vector and the second calibration vector;

display a position indicator on the curved display interface based on a directive gesture captured by a sensor, wherein the position indicator is based on the first gesture and the second gesture, and wherein the directive gesture is for manipulating a graphic or a user interface of the curved display interface; and

recalibrate the electronic device upon detecting a change in an orientation or a location of the sensor.

14. The non-transitory computer readable medium as claimed in claim 13, the instructions further comprising generating a transformation matrix based on the calibration point in a display coordinate system and an intersection in a sensor coordinate system.

15. The non-transitory computer readable medium as claimed in claim 13, the instructions further comprising tailoring the calibration by capturing a user touching the calibration point displayed on the curved display interface using an appendage of the user.

16. The non-transitory computer readable medium as claimed in claim 13, the instructions comprising:

determining a first appendage position of the directive gesture in a sensor coordinate system;

determining a second appendage position of the directive gesture in the sensor coordinate system;

calculating a transformed first point in a display coordinate system by applying a transformation matrix to the first appendage position;

calculating a transformed second point in the display coordinate system by applying the transformation matrix to the second appendage position;

generating a ray casting vector based on the transformed first point and the transformed second point; and

generating the position indicator based on an intersection of the ray casting vector with a display plane in the display coordinate system.

17. The non-transitory computer readable medium as claimed in claim 13, the instructions further comprising:

determining a reference point of the directive gesture in a sensor coordinate system;

calculating a transformed referencing point of the directive gesture in a display coordinate system by applying a transformation matrix to the reference point; and

generating an orthographic projection of the directive gesture on the curved display interface based on the transformed referencing point in the display coordinate system.

18. The non-transitory computer readable medium as claimed in claim 13, wherein the directive gesture is a typing gesture for generating the position indicator over a keyboard key.