Apparatus and method for recognizing proximity motion using sensors

• US 10,261,612 B2
• Filed: 01/08/2014
• Issued: 04/16/2019
• Est. Priority Date: 02/22/2013
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Han, Seung Ju, Park, Joon Ah, Ryu, Hyun Surk, Park, Du Sik, Lee, Ki Myung
• Primary Examiner(s): Patel, Nitin
• Assistant Examiner(s): Bogale, Amen
• Application Number:
  US14/150,051
• Publication Number:
  US 20140240248A1
• Time in Patent Office: 1,924 Days
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 2203/04106 Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/04166 Details of scanning methods, e.g. sampling time, grouping of sub areas or time sharing with display driving Synchronisation with the driving of the display or the backlighting unit to avoid interferences generated internally G06F3/04184
• CPC Class Codes: G06F 3/044 by capacitive means

Abstract

An apparatus for recognizing a proximity motion using sensors may include a first sensor to sense a first input region, a second sensor to sense a second input region distinct from the first input region, and an information transfer controller to transfer information related to a transition, between the first sensor and the second sensor.

Claims

1. An apparatus for recognizing a proximity motion using sensors, the apparatus comprising:

sensors, operatively connected and comprising;

a first sensor configured to sense an off-surface first input region at a distance closer than a predetermined distance off of a reference surface for recognizing a proximate motion of a first input object within the off-surface first input region, and

a second sensor simultaneously operable with the first sensor and configured to sense a second input region at a distance farther than the predetermined distance from the reference surface, wherein the second sensor senses a rate of change of light of each pixel over time; and

an information transfer controller to transfer information between the first sensor and the second sensor, the information related to a transition of the first input object in response to the transition occurring between the first input region and the second input region.

2. The apparatus of claim 1, further comprising:

a proximity measurer to measure a proximity between the first input object and the reference surface; and

a sensor controller to selectively activate the first sensor and the second sensor, based on the measured proximity.

3. The apparatus of claim 2, wherein the sensor controller comprises:

a proximity comparator to compare the proximity to the predetermined distance; and

an operation mode controller to control an operation mode of the first sensor and an operation mode of the second sensor, based on the comparison,

wherein the operation mode comprises at least one of an active mode, an inactive mode, and a stand-by mode.

4. The apparatus of claim 3, wherein the operation mode controller controls the second sensor to transition from the active mode to the stand-by mode or the inactive mode and controls the first sensor to transition from the inactive or the stand-by mode to the active mode when the first input object is determined to be transitioning into the first input region from the second input region based on the measured proximity.

5. The apparatus of claim 3, wherein the operation mode controller controls the first sensor to transition from the active mode to the standby mode or the inactive mode and controls the second sensor to transition from the inactive or the standby mode to the active mode when the first input object is determined to be transitioning into the second input region from the first input region based on the measured proximity.

6. The apparatus of claim 1, wherein the information related to the transition comprises an x-coordinate and a y-coordinate of the first input object being sensed in at least one of the first input region and the second input region.

7. The apparatus of claim 1, wherein:

the first sensor senses and indicates a three-dimensional location, including sensed x, y, and z-coordinates, of the first input object in the first input region, and

the second sensor senses and indicates a two-dimensional location, including sensed x and y-coordinates, of the first input object in the second input region.

8. The apparatus of claim 1, wherein:

the first sensor comprises a touch sensor to distinguish between a touching of the reference surface by the first input object and the object being sensed within the first input region, and

the second sensor comprises a vision sensor and indicates a two-dimensional location of the first input object based on a captured depth image.

9. The apparatus of claim 1, further comprising:

a display to display a proximity motion interface; and

an interface controller to control sizing of an area of the proximity motion interface indicating the first input object based on a proximity between the first input object and the reference surface, in response to the first input object being input in the first input region.

10. The apparatus of claim 9, wherein:

the proximity motion interface comprises at least one icon, and

the interface controller controls, in response to the at least one icon being an icon indicated by the first input object, each of sizing of the icon to increase as the proximity between the first input object and the reference surface increases, and selection of the icon, such that the icon is selected in response to the first input object being in contact with the reference surface.

11. The apparatus of claim 1, further comprising:

a display to display a proximity motion interface; and

an interface controller to control the proximity motion interface to dispose at least one predetermined icon along a perimeter of a point indicated by a second input object, in response to the second input object being input in the second input region.

12. The apparatus of claim 11, further comprising:

a signal processor to extract an axis and an endpoint of the first input object based on an output signal of the first sensor, when the first input object is input in the first input region,

wherein the interface controller controls, using the axis and the endpoint of the first input object, the proximity motion interface to dispose the at least one predetermined icon along a perimeter of an area in which the display is obscured by the first input object, in response to the first input object being input in the first input region.

13. The apparatus of claim 12, wherein the interface controller controls the proximity motion interface to change sizing of the at least one predetermined icon.

14. The apparatus of claim 1, further comprising:

input sensing units to determine whether input objects, comprising the first and second input objects, are sensed, based on at least one of an output signal of the first sensor and an output signal of the second sensor; and

an input selector to select at least one of the input objects based on a predetermined mode, in response to the input sensing units determining that the input objects are sensed.

15. The apparatus of claim 14, wherein the predetermined mode comprises a mode in which the input objects are received, a mode in which an input object most proximate to the reference surface is selected, and a mode in which an input object is selected when a position indicated by the input object is most proximate to a predetermined position above the reference surface.

16. The apparatus of claim 14, wherein the interface controller controls sizing of the at least one predetermined icon based on a proximity between at least one input object and the reference surface.

17. The apparatus of claim 14, further comprising:

a display to display a proximity motion interface;

a signal processor to extract an axis and an endpoint of at least one input object based on an output signal of the first sensor, in response to the at least one input object being sensed in the first input region; and

an interface controller to control the proximity motion interface to dispose at least one predetermined icon along a perimeter of a point indicated by at least one of the input objects, in response to the at least one input object being sensed in the second input region, and to control, using the axis and the endpoint of the at least one input object, the proximity motion interface to dispose the at least one predetermined icon along a perimeter of an area in which the display is obscured by the at least one input object, in response to the at least one input object being sensed in the first input region.

18. The apparatus of claim 1, further comprising:

an input pattern recognizer to recognize an input pattern based on at least one of an output signal of the first sensor and an output signal of the second sensor; and

a function performer to perform a function corresponding to the input pattern.

19. The apparatus of claim 18, wherein:

the input pattern recognizer comprises;

a calculator to calculate at least one of a velocity and an angular velocity of the first input object, based on at least one of the output signal of the first sensor and the output signal of the second sensor, and

the function performer comprises;

a detector to detect a function corresponding to the input pattern, based on a result of the calculating.

20. The apparatus of claim 1, further comprising:

a display to provide a two-dimensional (2D) interface and a three-dimensional (3D) interface;

a calculator to calculate at least one of a position, a velocity, and an angular velocity of the first input object, based on at least one of an output signal of the first sensor and an output signal of the second sensor; and

a display controller to control an operation mode of the display based on a result of the calculating.

21. An apparatus for recognizing a proximity motion using sensors, the apparatus comprising:

sensors, operatively connected and comprising;

a first sensor configured to sense a first input region at a distance closer than a predetermined distance from a reference surface for recognizing a proximate motion, and

a second sensor simultaneously operable with the first sensor and configured to sense a second input region at a distance farther than the predetermined distance from the reference surface, wherein the second sensor senses a rate of change of light of each pixel over time;

an information transfer controller to transfer information between the first sensor and the second sensor, the information related to a transition of first input object in response to the transition occurring between the first input region and the second input region;

a display to display a proximity motion interface;

an interface controller to control the proximity motion interface for at least one predetermined icon to be disposed along a perimeter of a point indicated by a second input object, when the second input object is input in the second input region; and

a signal processor to extract an axis and an endpoint of the first input object based on an output signal of the first sensor, in response to the first input object being input in the first input region,

wherein the interface controller controls, using the axis and the endpoint of the first input object, the proximity motion interface to dispose the at least one predetermined icon along a perimeter of an area of the interface in which the display is obscured by the first input object, in response to the first input object being input in the first input region.

22. The apparatus of claim 21, wherein the interface controller controls sizing of the at least one predetermined icon based on the proximity between the first input object and the reference surface.

23. An apparatus for recognizing a proximity motion using sensors, the apparatus comprising:

sensors, operatively connected and comprising;

a first sensor configured to sense a first input region at a distance closer than a predetermined distance from a reference surface for recognizing a proximate motion;

a second sensor simultaneously operable with the first sensor and configured to sense a second input region at a distance farther than the predetermined distance from the reference surface, wherein the second sensor senses a rate of change of light of each pixel over time;

an information transfer controller to transfer information between the first sensor and the second sensor, the information related to a transition of an input object, in response to the transition occurring between the first input region and the second input region;

sensing units to determine whether input objects are sensed, based on at least one of an output signal of the first sensor and an output signal of the second sensor;

an input selector to select at least one of the input objects based on a predetermined mode, in response to the input sensing units determining that the input objects are sensed;

a display to display a proximity motion interface;

a signal processor to extract an axis and an endpoint of at least one input object based on an output signal of the first sensor, in response to the at least one input object being sensed in the first input region; and

an interface controller to control the proximity motion interface to dispose at least one predetermined icon along a perimeter of a point indicated by at least one of the input objects, in response to the at least one input object being sensed in the second input region, and to control, using the axis and the endpoint of the at least one input object, the proximity motion interface to dispose the at least one predetermined icon along a perimeter of an area in which the display is obscured by the at least one input object, in response to the at least one input object being sensed in the first input region.

24. A method of recognizing a proximity motion using each of operatively connected sensors, the method comprising:

sensing, using a first sensor, an off-surface first input region in response to a first input object being present in the first input region at a distance closer than a predetermined distance off of a reference surface for recognizing a proximity motion of the first input object off of the reference surface;

sensing, using a second sensor, a second input region the first input object is present in the second input region at a distance farther than the predetermined distance from the reference surface, and sensing a rate of change of light of each pixel over time, the second sensor being simultaneously operable with the first sensor; and

transferring, between the first sensor and the second sensor, information related to a transition of the first input object in response to the transition occurring between the first input region and the second input region.

25. The method of claim 24, further comprising:

measuring a proximity between the first input object and the reference surface; and

selectively activating the first sensor and the second sensor, based on the measured proximity.

26. The method of claim 25, wherein the selective activation includes controlling the second sensor to transition from an active mode to a standby mode or inactive mode, and controlling the first sensor to transition from the inactive or the standby mode to the active mode, in response to the first input object being determined to be transitioning into the first input region from the second input region based on the measured proximity.

27. The method of claim 25, wherein the selective activation includes controlling the first sensor to transition from an active mode to a standby mode or inactive mode, and controlling the second sensor to transition from the inactive or the standby mode to the active mode, in response to the first input object being determined to be transitioning into the second input region from the first input region based on the measured proximity.

28. The method of claim 24, wherein the information related to the transition comprises an x-coordinate and a y-coordinate of the first input object being sensed in at least one of the first input region and the second input region.

29. The method of claim 24, wherein:

the first sensor senses and indicates a three-dimensional location, including sensed x, y, and z-coordinates, of the first input object in the first input region, and

the second sensor senses and indicates a two-dimensional location, including sensed x and y-coordinates, of the first input object in the second input region.

30. The method of claim 24, wherein:

the first sensor comprises a touch sensor to distinguish between a touching of the reference surface by the first input object and sensing within the first input region, and

the second sensor comprises a vision sensor, and indicates a two-dimensional location of the first input object based on a captured depth image.

31. The method of claim 24, further comprising:

controlling a proximity motion interface to change sizing of an area indicated by the first input object based on a proximity between the first input object and the reference surface, in response to the first input object being input in the first input region; and

displaying the proximity motion interface.

32. The method of claim 31, wherein:

the proximity motion interface comprises at least one icon, and

the controlling comprises;

controlling, in response to the at least one icon corresponding to an icon indicated by the first input object, the proximity motion interface to increase a size of the icon as the proximity between the first input object and the reference surface increases; and

controlling, in response to the at least one icon corresponding to the icon indicated by the first input object, the proximity motion interface to select the icon, in response to the first input object being in contact with the reference surface.

33. The method of claim 24, further comprising:

controlling a proximity motion interface to dispose at least one predetermined icon along a perimeter of a point indicated by a second input object, in response to the second input object being input in the second input region; and

displaying the proximity motion interface.

34. The method of claim 33, further comprising:

extracting an axis and an endpoint of the first input object based on an output signal of the first sensor, in response to the first input object being input in the first input region,

wherein the controlling comprises controlling, using the axis and the endpoint of the first input object, the proximity motion interface to dispose the at least one predetermined icon along a perimeter of an area in which the display is obscured by the first input object, in response to the first input object being input in the first input region.

35. The method of claim 24, further comprising:

determining whether input objects, comprising the first input object and the second input object, are sensed, based on at least one of an output signal of the first sensor and an output signal of the second sensor; and

selecting at least one of the input objects based on a predetermined mode, in response to the input sensing units determining that the input objects are sensed.

36. The method of claim 35, wherein the predetermined mode comprises a mode in which the input objects are received, a mode in which an input object most proximate to the reference surface is selected, and a mode in which an input object is selected in response to a position indicated by the input object being most proximate to a predetermined position above the reference surface.

37. The method of claim 24, further comprising:

calculating at least one of a velocity and an angular velocity of the first input object, based on at least one of the output signal of the first sensor and the output signal of the second sensor; and

performing a function corresponding to the input pattern, based on the calculation.

38. The method of claim 24, further comprising:

calculating at least one of a position, a velocity, and an angular velocity of the first input object, based on at least one of an output signal of the first sensor and an output signal of the second sensor; and

switching an operation mode of a display between a two-dimensional (2D) interface and a three-dimensional (3D) interface, based on the calculation.

39. A non-transitory computer-readable medium comprising a program for instructing a computer to perform the method of claim 24.

40. A method of recognizing a proximity motion using each of operatively connected sensors, the method comprising:

sensing, using a first sensor, a first input region in response to a first input object being present in the first input region at a distance closer than a predetermined distance from a reference surface for recognizing a proximity motion;

sensing, using a second sensor, a second input region when the first input object is present in the second input region at a distance farther than the predetermined distance from the reference surface, and sensing a rate of change of light of each pixel over time, the second sensor being simultaneously operable with the first sensor;

transferring, between the first sensor and the second sensor, information related to a transition of the first input object in response to the transition occurring between the first input region and the second input region;

controlling a proximity motion interface to dispose at least one predetermined icon along a perimeter of a point indicated by a second input object, in response to the second input object being input in the second input region;

displaying the proximity motion interface; and

extracting an axis and an endpoint of the first input object based on an output signal of the first sensor, in response to the first input object being input in the first input region,

wherein the controlling comprises controlling, using the axis and the endpoint of the first input object, the proximity motion interface to dispose the at least one predetermined icon along a perimeter of an area in which the display is obscured by the first input object, in response to the first input object being input in the first input region.

41. A method of controlling a user interface device using each of operatively connected sensors, the method comprising:

controlling the user interface device to actuate a first sensor to sense for an input in a first off-surface region off of a touch input surface of the user interface device;

controlling the user interface device to actuate a second sensor to sense for an input in a second off-surface region measured from the touch input surface, and farther away from the touch input surface than the first region, and sense a rate of change of light of each pixel over time, the second sensor being simultaneously operable with the first sensor;

actuating an information transfer controller to transfer information between the first sensor and the second sensor, the information related to a transition of the first input object in response to the transition occurring between the first input region and the second input region; and

selectively controlling a user interface of the user interface device in a first manner and a second manner based on at least one of a position and a movement of the sensed input within the first region and the second region.