Method and device for gesture recognition diagnostics for device orientation

• US 8,941,587 B2
• Filed: 08/29/2011
• Issued: 01/27/2015
• Est. Priority Date: 04/13/2011
• Status: Expired due to Fees

• Current Assignee: LG Electronics, Inc. (LG Corporation)
• Original Assignee: LG Electronics, Inc. (LG Corporation)
• Inventors: Kim, Sangki, Yeo, Junhee
• Primary Examiner(s): Johnson, Gerald
• Application Number:
  US13/220,361
• Publication Number:
  US 20120262372A1
• Time in Patent Office: 1,247 Days
• Field of Search: 345/158, 455/556.1
• US Class Current: 345/158
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 2200/1614 Image rotation following screen orientation, e.g. switching from landscape to portrait mode
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G09G 2340/0442 Handling or displaying different aspect ratios, or changing the aspect ratio
• CPC Class Codes: G09G 2340/0492 Change of orientation of the displayed image, e.g. upside-down, mirrored

Abstract

Systems, circuits, and devices for recognizing gestures are discussed. A mobile device includes a housing, an orientation sensor, a camera implemented on the housing, a memory for storing a lookup table comprising multiple gestures and corresponding commands, and a controller coupled to the orientation sensor, the camera, and the memory. The controller is configured to generate trace data corresponding to a gesture captured by the camera, wherein x, y, and z coordinates of the trace data are applied according to an orientation of the housing during the gesture. The controller is also configured to determine an orientation angle of the housing detected by the orientation sensor. The controller is further configured to recognize the gesture through accessing the lookup table based on the trace data and the orientation angle of the housing.

Claims

1. An image display device, comprising:

a housing;

an orientation sensor configured to determine an orientation angle of the housing;

a display configured to display an image;

a camera configured to capture a user gesture in a field of view of the camera; and

a controller operatively coupled to the orientation sensor, the camera, and the display, the controller configured to;

generate trace data corresponding to the gesture captured by the camera in response to a receipt of the user gesture;

convert the user gesture to an execution command based on the orientation angle of the housing and the trace data; and

execute the execution command upon recognition of the user gesture as the execution command.

2. The image display device of claim 1, wherein the image display device is a mobile device.

3. The image display device of claim 2, wherein the mobile device includes a transceiver configured to communicate to another device.

4. The image display device of claim 1, wherein the trace data includes at least two of x, y, and z orthogonal coordinates relative to one of the camera and the orientation angle.

5. The image display device of claim 4, wherein the controller is configured to convert the captured user gesture into the trace data by

capturing, by the camera, a plurality of frames from an initial instance of the gesture to a final instance of the gesture;

and

determining the at least two of the x, y, and z orthogonal coordinates of the trace data from the plurality of frames.

6. The image display device of claim 4, wherein the controller is configured to recognize the user gesture based on the trace data.

7. The image display device of claim 6, further comprising:

a memory operatively coupled to the controller and configured to store gesture information and a gesture-specific execution command that corresponds to the orientation angle,

wherein the controller is configured to access the gesture information and convert the recognized user gesture to the gesture-specific execution command that corresponds to the orientation angle.

8. The image display device of claim 7, wherein the controller is configured to convert the recognized user gesture into the gesture-specific execution command by determining that a pattern of the trace data substantially matches a pattern in the memory.

9. The image display device of claim 1, wherein the orientation sensor is an accelerometer, a gyroscope, or a magnetometer.

10. The image display device of claim 1, wherein the camera is a depth-aware camera, a stereo camera, or a digital camera.

11. The image display device of claim 1, wherein the controller is configured to control the display to display the image in one of a landscape orientation and a portrait orientation when the orientation angle of the housing is less than a threshold angle and to display the image in another of the landscape orientation and the portrait orientation when the orientation angle of the housing is greater than the threshold angle.

12. The image display device of claim 1, wherein the controller is configured to obtain the orientation angle by measuring an angle between a straight line crossing the camera in a lengthwise direction of the housing and a straight line crossing the camera in a widthwise direction of the housing.

13. An image display device, comprising:

a housing;

an orientation sensor configured to determine an orientation of the housing relative to a predetermined coordinate system;

a display configured to display an image;

a camera configured to capture an object in a field of view of the camera; and

a controller coupled to the display, the orientation sensor and the camera, the controller configured to;

display a pointer on the display at a specific location relative to the displayed image, the specific location corresponding to a first orientation of the housing and a detected location of the object, and

when the orientation of the housing is changed to a second orientation, reorient the image according to the second orientation while maintaining the pointer at the specific location relative to the displayed image.

14. The image display device of claim 13,

wherein the controller is configured to identify coordinates of the pointer when the housing is in the first orientation, and

wherein the controller is configured to maintain the pointer at the specific location relative to the displayed image when the housing is in the second orientation based on the coordinates.

15. The image display device of claim 13,

wherein the controller is configured to determine a first distance between the pointer and a first part of the image and a second distance between the pointer and a second part of the image when the housing is in the first orientation, and

wherein the controller is configured to maintain the pointer at the specific location relative to the displayed image when the housing is in the second orientation based on the first and second distances.

16. The image display device of claim 13, wherein the first orientation is one of a landscape orientation and a portrait orientation, and the second orientation is another of the landscape orientation and the portrait orientation.

17. The image display device of claim 13, wherein the controller is configured to detect the change of the housing to the second orientation when tilt data generated by the orientation sensor is greater than a threshold angle.

18. An image display device, comprising:

a housing;

an orientation sensor configured to determine an orientation of the housing;

a display configured to display an image;

a camera configured to capture an object in a field of view of the camera; and

a controller coupled to the display, the orientation sensor and the camera, the controller configured to;

display a pointer on the display at a specific location, the specific location corresponding to the housing and a detected location of the object,

reorient the image according to a second orientation when the orientation of the housing is changed to the second orientation, and

display a pointer on the display at the specific location, the specific location corresponding to the housing and the detected location of the object regardless of the orientation of the housing.