Method of recognizing contactless user interface motion and system there-of

• US 9,207,779 B2
• Filed: 09/18/2013
• Issued: 12/08/2015
• Est. Priority Date: 09/18/2012
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kong, Myoung Seok, Park, Hyun Woo, Shin, Taek Kyun
• Primary Examiner(s): Sheng, Tom
• Application Number:
  US14/030,099
• Publication Number:
  US 20140078048A1
• Time in Patent Office: 811 Days
• Field of Search: 345/156, 348/E13.064, 348/51
• US Class Current: 1/1
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image

Abstract

A contactless user-interface (UI) motion recognizing device and method of controlling the same are provided. The method includes: obtaining a left-eye image and a right-eye image; determining an object position of an object in the obtained left-eye image and the obtained right-eye image; determining an object brightness of the object; determining depth information of the object using the determined object brightness; determining a three-dimensional (3D) object position of the object using the determined object position and the determined depth information; determining an object moving velocity based on the determined 3D object position and a previous 3D object position; and determining a UI pattern based on the determined 3D object position and the determined object moving velocity, and executing an operation according to the determined UI pattern.

Claims

1. A method of controlling a contactless user-interface (UI) motion recognizing device, the method comprising:

obtaining a left-eye image and a right-eye image;

determining an object position of an object in the obtained left-eye image and the obtained right-eye image;

determining an object brightness of the object;

determining depth information of the object using the determined object brightness;

determining a three-dimensional (3D) object position of the object using the determined object position and the determined depth information;

determining an object moving velocity based on the determined 3D object position and a previous 3D object position; and

determining a UI pattern based on the determined 3D object position and the determined object moving velocity, and executing an operation according to the determined UI pattern,

wherein the method further comprises calculating a disparity between the obtained left-eye image and the obtained right-eye image, and

wherein the determining the object brightness comprises determining the object brightness based on the calculated disparity.

2. The method according to claim 1, further comprising:

converting the calculated disparity to brightness information; and

creating a disparity map based on the brightness information,

wherein the determining the object brightness based on the calculated disparity comprises determining the object brightness based on the created disparity map.

3. The method according to claim 2, wherein the determining the object brightness based on the created disparity map comprises determining the object brightness corresponding to a brightest portion of the disparity map.

4. The method according to claim 2, wherein the determining the object position comprises determining the object position corresponding to a position of a brightest portion of the disparity map.

5. The method according to claim 1, wherein the determining the UI pattern comprises determining whether the determined object moving velocity is greater than a predetermined threshold, and executing the operation in response to determining that the determined object moving velocity is greater than the predetermined threshold.

6. The method according to claim 1, wherein the determining the object moving velocity comprises determining the object moving velocity according to:

OMV=(X_N+1−

X_N,Y_N+1−

Y_N,Z_N+1−

Z_N),

where OMV is the object moving velocity, X_N+1 is a first dimension coordinate of the determined 3D object position, Y_N+1 is a second dimension coordinate of the determined 3D object position, Z_N+1 is a third dimension coordinate of the determined 3D object position, X_N is a first dimension coordinate of the previous 3D object position, Y_N is a second dimension coordinate of the previous 3D object position, and Z_N is a third dimension coordinate of the previous 3D object position.

7. The method according to claim 1, further comprising:

determining whether the object is a pre-defined object,

wherein the UI pattern is determined based on the determined 3D object position, the determined object moving velocity, and whether the object is determined as the pre-defined object.

8. The method according to claim 7, wherein the determining whether the object is the pre-defined object comprises determining whether a color of the object is a pre-defined color according to color information corresponding to a pixel group of the determined object position.

9. The method according to claim 1, wherein the left-eye image and the right-eye image are obtained using a stereo camera of the contactless UI motion recognizing device.

10. The method according to claim 1, wherein the determining the object position comprises sensing an end of the object and determining an object position of the sensed end of the object.

11. The method according to claim 10, wherein a brightest portion of the obtained left-eye image and the obtained right-eye image is sensed as the end of the object.

12. The method according to claim 1, further comprising:

measuring a temperature of a surrounding; and

detecting the object based on the measured temperature and obtaining a temperature object detection signal according to the detecting,

wherein UI pattern is determined based on the determined 3D object position, the determined object moving velocity, and the obtained temperature object detection signal.

13. The method according to claim 12, further comprising:

converting the measured temperature to temperature-based depth information,

wherein the UI pattern is determined based on the determined 3D object position, the determined object moving velocity, the obtained temperature object detection signal, and the temperature-based depth information.

14. The method according to claim 1, further comprising:

sensing a sensing value of the object using a sensor;

detecting the object based on the sensed sensing value and obtaining a sensor object detection signal according to the detecting,

wherein UI pattern is determined based on the determined 3D object position, the determined object moving velocity, and the obtained sensor object detection signal, and

wherein the sensor comprises at least one of an infrared camera, a thermal sensor, a gyro sensor, an acceleration sensor, an earth magnetic field sensor, a barometer, a gesture sensor, a proximity sensor, an illuminance sensor, a Red Green Blue (RGB) sensor, a thermal/humidity sensor, and a grip sensor.

15. A method of controlling a contactless user-interface (UI) motion recognizing device, the method comprising:

obtaining a left-eye image and a right-eye image;

determining an object position of an object in the obtained left-eye image and the obtained right-eye image;

determining an object brightness of the object;

determining depth information of the object using the determined object brightness;

determining a three-dimensional (3D) object position of the object using the determined object position and the determined depth information;

determining an object moving velocity based on the determined 3D object position and a previous 3D object position; and

determining a UI pattern based on the determined 3D object position and the determined object moving velocity, and executing an operation according to the determined UI pattern,

wherein the method further comprises;

sensing a sensing value of the object using a sensor;

detecting the object based on the sensed sensing value and obtaining a sensor object detection signal according to the detecting; and

determining variations of the sensing value over time,

wherein the UI pattern is determined based on the determined 3D object position, the determined object moving velocity, the obtained sensor object detection signal, and the determined variations of the sensing value, and

wherein the sensor comprises at least one of an infrared camera, a thermal sensor, a gyro sensor, an acceleration sensor, an earth magnetic field sensor, a barometer, a gesture sensor, a proximity sensor, an illuminance sensor, a Red Green Blue (RGB) sensor, a thermal/humidity sensor, and a grip sensor.

16. A method of controlling a contactless user-interface (UI) motion recognizing device, the method comprising:

obtaining a left-eye image and a right-eye image;

determining an object position of an object in the obtained left-eye image and the obtained right-eye image;

determining an object brightness of the object;

determining depth information of the object using the determined object brightness;

determining a three-dimensional (3D) object position of the object using the determined object position and the determined depth information; and

determining a UI pattern based on the determined 3D object position and executing an operation according to the determined UI pattern,

wherein the method further comprises calculating a disparity between the obtained left-eye image and the obtained right-eye image;

wherein the determining the object brightness comprises determining the object brightness based on the calculated disparity.

17. The method according to claim 16, further comprising:

converting the calculated disparity to brightness information; and

creating a disparity map based on the brightness information,

wherein the determining the object brightness based on the calculated disparity comprises determining the object brightness based on the created disparity map.

18. The method according to claim 17, wherein the determining the object brightness based on the created disparity map comprises determining the object brightness corresponding to a brightest portion of the disparity map.

19. The method according to claim 17, wherein the determining the object position comprises determining the object position corresponding to a position of a brightest portion of the disparity map.