Three-dimensional man/machine interface

• US 10,303,266 B2
• Filed: 01/30/2012
• Issued: 05/28/2019
• Est. Priority Date: 01/31/2011
• Status: Active Grant

• Current Assignee: Quickstep Technologies, LLC (Broadcom, Inc.)
• Original Assignee: Quickstep Technologies, LLC (Broadcom, Inc.)
• Inventors: Roziere, Didier
• Primary Examiner(s): Rayan, Mihir K
• Application Number:
  US13/982,791
• Publication Number:
  US 20130307776A1
• Time in Patent Office: 2,675 Days
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06F 3/044 by capacitive means
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

A method is provided for selecting controls, which implements a control interface, a display, and at least one sensor capable of detecting at least one control object, including a step (i) of obtaining information on the distance between the control object(s) and the control interface using the sensor(s), and a step (ii) of displaying, on the display, at least one symbol representing a control or set of controls according to a display mode, wherein the method further includes a step of using the distance information to determine the display mode of the symbol(s). A device is also provided for implementing the method and to an apparatus.

Claims

1. A method for selecting commands on a command interface including a display and at least one sensor configured for detecting at least one command object, comprising:

displaying a plurality of sets of commands, wherein one or more of the sets of commands includes a plurality of hierarchical levels of commands;

obtaining vertical distance data and horizontal position data of said at least one command object in relation to the command interface using said at least one sensor; and

using said vertical distance data and said horizontal position data to browse the commands in the plurality of sets of commands, the browsing including;

selecting one set of commands from the displayed plurality of sets of commands,

in accordance with a determination that the vertical distance of said at least one command object is a first distance from said command interface while the horizontal position data of said at least one command object corresponds to a location of the selected one set of commands, displaying, at a respective location on the display, a first command in the selected one set of commands, and

in accordance with a determination that the vertical distance of said at least one command object is a second distance, different from the first distance, from said command interface while the horizontal position data of said at least one command object corresponds to the location of the selected one set of commands, displaying, at the respective location, a second command in the selected one set of commands without displaying the first command.

2. The method of claim 1, further comprising:

using said horizontal position data in order to determine a display mode of the plurality of sets of commands.

3. The method of claim 2, further comprising using at least one of the vertical distance data, and the vertical distance data and the horizontal position data, to determine the display mode.

4. The method of claim 1, further comprising at least one of:

measuring capacitive interactions between the at least one command object and the at least one sensor; and

measuring variations of light intensity due to a presence of the at least one command object.

5. The method of claim 1, further comprising selecting, while the command object hovers over the command interface, a particular command of the commands in the hierarchical levels of commands by verifying at least one selection condition based on at least one of the vertical distance data, and the vertical distance data and the horizontal position data, wherein the at least one selection condition comprises one or more of:

the distance of the at least one command object is less than a predetermined selection distance, and

the distance of the at least one command object is less than a predetermined selection distance during a predetermined minimum duration.

6. The method of claim 5, further comprising executing the particular command by performing one or more of executing a computer program, executing an application, displaying content of a file stored on computer storage media, displaying an image, playing a sound, and reading multimedia content.

7. The method of claim 5, further comprising executing a command for moving a symbol by:

displaying said symbol according to a position representing a position of a command object or a set of command objects; and

validating a movement of said symbol by verifying at least one validation condition based on at least one of the distance data, and the distance and position data.

8. The method of claim 7, the at least one validation condition comprising one or more of:

the distance of the at least one command object is less than a predetermined selection distance,

the distance of the at least one command object is less than a predetermined selection distance during a predetermined minimum duration,

the at least one command object is in contact with a surface of the command interface,

the at least one command object performs a rapid to-and-fro movement over distance, and

at least two command objects perform a divergent movement in position around a predetermined position.

9. The method of claim 1, further comprising selecting, while the command object hovers over the command interface, a particular command of the commands in the hierarchical levels of commands by verifying at least one selection condition based on at least one of the vertical distance data, and the vertical distance data and the horizontal position data, wherein the at least one selection condition comprises one or more of:

the at least one command object performs a rapid to-and-fro movement over distance, and

at least two command objects perform a convergent movement in position towards a predetermined position.

10. The method of claim 1, further comprising displaying symbols in the form of icons.

11. The method of claim 1, further comprising activating the display when the vertical distance data of the at least one command object is less than a predetermined activation distance.

12. The method of claim 1, wherein the selecting the one set of commands comprises:

in accordance with a determination that the vertical distance of said at least one command object is a third distance, different from and greater than the first distance and the second distance, from said command interface, selecting the one set of commands from the displayed plurality of sets of commands.

13. The method of claim 12, wherein the third distance is less than a first threshold distance from said command interface but greater than or equal to a second threshold distance, smaller than the first threshold distance, from said command interface;

and

wherein a vertical distance of said at least one command object less than the first threshold distance but greater than or equal to the second threshold distance selects the one set of commands.

14. The method of claim 13, wherein the first distance is less than the second threshold distance from said command interface but greater than or equal to a third threshold distance, smaller than the second threshold distance, from said command interface;

wherein a vertical distance of said at least one command object less than the second threshold distance but greater than or equal to the third threshold distance corresponds to the display of the first command;

wherein the second distance is less than the third threshold distance from said command interface but greater than or equal to a fourth threshold distance, smaller than the third threshold distance, from said command interface; and

wherein a vertical distance of said at least one command object less than the third threshold distance but greater than or equal to the fourth threshold distance corresponds to the display of the second command.

15. The method of claim 14, wherein the first threshold distance is a predetermined distance and the second, third and fourth threshold distances are determined as functions of a number of commands or hierarchical levels of commands corresponding to the selected one set of commands.

16. The method of claim 1, wherein the respective location on the display is the same as a location of the selected one set of commands, such that displaying the first command or the second command comprises superimposing the first command or the second command substantially over the selected set of commands.

17. The method of claim 1, wherein the respective location on the display is different than, but proximate to, a location of the selected set of commands.

18. The method of claim 1, wherein the browsing further includes navigating from the selected one set of commands to the first or second command based on horizontal position data.

19. The method of claim 1, further comprising selecting a particular command of the commands in the hierarchical levels of commands by verifying at least one selection condition based on the vertical distance data, wherein the at least one selection condition further comprises one or more of:

the distance of the at least one command object is less than a predetermined selection distance,

the distance of the at least one command object is less than a predetermined selection distance during a predetermined minimum duration, and

the at least one command object is in contact with the surface of the command interface.

20. A device for selecting commands, comprising:

a command interface;

a display;

at least one sensor configured for detecting at least one command object; and

a microprocessor unit capable of;

causing a displaying of a plurality of sets of commands, wherein one or more of the sets of commands includes a plurality of hierarchical levels of commands,

obtaining vertical distance data and horizontal position data of said at least one command object in relation to the command interface using said at least one sensor, and

using said vertical distance data and said horizontal position data to browse the commands in the plurality of sets of commands, the browsing including;

selecting one set of commands from the displayed plurality of sets of commands, in accordance with a determination that the vertical distance of said at least one command object is a first distance from said command interface while the horizontal position data of said at least one command object corresponds to a location of the selected one set of commands,

displaying, at a respective location on the display, a first command in the selected one set of commands, and in accordance with a determination that the vertical distance of said at least one command object is a second distance, different from the first distance, from said command interface while the horizontal position data of said at least one command object corresponds to the location of the selected one set of commands,

displaying, at the respective location, a second command in the selected one set of commands without displaying the first command.

21. The device of claim 20, the at least one sensor comprising one or more of:

capacitive sensors, and

optical sensors.

22. The device of claim 20, incorporated into one of a computer, telephone, smartphone, tablet, display screen, and terminal.

23. The device of claim 20, the microprocessor unit further capable of selecting, while the command object hovers over the command interface, a particular command of the commands in the hierarchical levels of commands by verifying at least one selection condition based on at least one of the vertical distance data, and the vertical distance data and the horizontal position data, wherein the at least one selection condition comprises one or more of:

the distance of the at least one command object is less than a predetermined selection distance, and

the distance of the at least one command object is less than a predetermined selection distance during a predetermined minimum duration.

24. The device of claim 20, the microprocessor unit further capable of selecting, while the command object hovers over the command interface, a particular command of the commands in the hierarchical levels of commands by verifying at least one selection condition based on at least one of the vertical distance data, and the vertical distance data and the horizontal position data, wherein the at least one selection condition comprises one or more of:

the at least one command object performs a rapid to-and-fro movement over distance, and

at least two command objects perform a convergent movement in position towards a predetermined position.

25. The device of claim 20, wherein the selecting the one set of commands comprises:

in accordance with a determination that the vertical distance of said at least one command object is a third distance, different from and greater than the first distance and the second distance, from said command interface, selecting the one set of commands from the displayed plurality of sets of commands.

26. The device of claim 25, wherein the third distance is less than a first threshold distance from said command interface but greater than or equal to a second threshold distance, smaller than the first threshold distance, from said command interface;

and

wherein a vertical distance of said at least one command object less than the first threshold distance but greater than or equal to the second threshold distance selects the one set of commands.

27. The device of claim 26 wherein the first distance is less than the second threshold distance from said command interface but greater than or equal to a third threshold distance, smaller than the second threshold distance, from said command interface;

wherein a vertical distance of said at least one command object less than the second threshold distance but greater than or equal to the third threshold distance corresponds to the display of the first command;

wherein the second distance is less than the third threshold distance from said command interface but greater than or equal to a fourth threshold distance, smaller than the third threshold distance, from said command interface; and

wherein a vertical distance of said at least one command object less than the third threshold distance but greater than or equal to the fourth threshold distance corresponds to the display of the second command.

28. The device of claim 27 wherein the first threshold distance is a predetermined distance and the second, third and fourth threshold distances are determined as functions of a number of commands or hierarchical levels of commands corresponding to the selected one set of commands.

29. The device of claim 20, wherein the respective location on the display is the same as a location of the selected one set of commands, such that displaying the first command or the second command comprises superimposing the first command or the second command substantially over the selected set of commands.

30. The device of claim 20, wherein the respective location on the display is different than, but proximate to, a location of the selected set of commands.

31. The device of claim 20, wherein the browsing further includes navigating from the selected one set of commands to the first or second command based on horizontal position data.

32. The device of claim 20, the microprocessor unit further capable of selecting a particular command of the commands in the hierarchical levels of commands by verifying at least one selection condition based on the vertical distance data, wherein the at least one selection condition further comprises one or more of:

the distance of the at least one command object is less than a predetermined selection distance,

the distance of the at least one command object is less than a predetermined selection distance during a predetermined minimum duration, and

the at least one command object is in contact with the surface of the command interface.

33. A non-transitory computer-readable storage medium storing instructions, which when executed by a device including a display, at least one sensor configured for detecting at least one command object and one or more processors, cause the one or more processors to perform a method comprising:

displaying a plurality of sets of commands, wherein one or more of the sets of commands includes a plurality of hierarchical levels of commands;

obtaining vertical distance data and horizontal position data of said at least one command object in relation to the command interface using said at least one sensor; and

using said vertical distance data and said horizontal position data to browse the commands in the plurality of sets of commands, the browsing including;

selecting one set of commands from the displayed plurality of sets of commands, in accordance with a determination that the vertical distance of said at least one command object is a first distance from said command interface while the horizontal position data of said at least one command object corresponds to a location of the selected one set of commands, displaying, at a respective location on the display, a first command in the selected one set of commands, and in accordance with a determination that the vertical distance of said at least one command object is a second distance, different from the first distance, from said command interface while the horizontal position data of said at least one command object corresponds to the location of the selected one set of commands, displaying, at the respective location, a second command in the selected one set of commands without displaying the first command.

34. The non-transitory computer-readable storage medium of claim 33, the method further comprising:

using said horizontal position data in order to determine a display mode of the plurality of sets of commands.

35. The non-transitory computer-readable storage medium of claim 34, the method further comprising using at least one of the vertical distance data, and the vertical distance data and the horizontal position data, to determine the display mode.

36. The non-transitory computer-readable storage medium of claim 33, the method further comprising at least one of:

measuring capacitive interactions between the at least one command object and the at least one sensor; and

measuring variations of light intensity due to a presence of the at least one command object.

37. The non-transitory computer-readable storage medium of claim 33, the method further comprising selecting, while the command object hovers over the command interface, a particular command of the commands in the hierarchical levels of commands by verifying at least one selection condition based on at least one of the vertical distance data, and the vertical distance data and the horizontal position data, wherein the at least one selection condition comprises one or more of:

the distance of the at least one command object is less than a predetermined selection distance, and

the distance of the at least one command object is less than a predetermined selection distance during a predetermined minimum duration.

38. The non-transitory computer-readable storage medium of claim 37, the method further comprising executing the particular command by performing one or more of executing a computer program, executing an application, displaying content of a file stored on computer storage media, displaying an image, playing a sound, and reading multimedia content.

39. The non-transitory computer-readable storage medium of claim 37, the method further comprising executing a command for moving a symbol by:

displaying said symbol according to a position representing a position of a command object or a set of command objects; and

validating a movement of said symbol by verifying at least one validation condition based on at least one of the distance data, and the distance and position data.

40. The non-transitory computer-readable storage medium of claim 39, the at least one validation condition comprising one or more of:

the distance of the at least one command object is less than a predetermined selection distance,

the distance of the at least one command object is less than a predetermined selection distance during a predetermined minimum duration,

the at least one command object is in contact with a surface of the command interface,

the at least one command object performs a rapid to-and-fro movement over distance, and

at least two command objects perform a divergent movement in position around a predetermined position.

41. The non-transitory computer-readable storage medium of claim 33, the method further comprising selecting, while the command object hovers over the command interface, a particular command of the commands in the hierarchical levels of commands by verifying at least one selection condition based on at least one of the vertical distance data, and the vertical distance data and the horizontal position data, wherein the at least one selection condition comprises one or more of:

the at least one command object performs a rapid to-and-fro movement over distance, and

at least two command objects perform a convergent movement in position towards a predetermined position.

42. The non-transitory computer-readable storage medium of claim 33, the method further comprising displaying symbols in the form of icons.

43. The non-transitory computer-readable storage medium of claim 33, the method further comprising activating the display when the vertical distance data of the at least one command object is less than a predetermined activation distance.

44. The non-transitory computer-readable storage medium of claim 33, wherein the selecting the one set of commands comprises:

in accordance with a determination that the vertical distance of said at least one command object is a third distance, different from and greater than the first distance and the second distance, from said command interface, selecting the one set of commands from the displayed plurality of sets of commands.

45. The non-transitory computer-readable storage medium of claim 44, wherein the third distance is less than a first threshold distance from said command interface but greater than or equal to a second threshold distance, smaller than the first threshold distance, from said command interface;

and wherein a vertical distance of said at least one command object less than the first threshold distance but greater than or equal to the second threshold distance selects the one set of commands.

46. The non-transitory computer-readable storage medium of claim 45, wherein the first distance is less than the second threshold distance from said command interface but greater than or equal to a third threshold distance, smaller than the second threshold distance, from said command interface;

wherein a vertical distance of said at least one command object less than the second threshold distance but greater than or equal to the third threshold distance corresponds to the display of the first command;

wherein the second distance is less than the third threshold distance from said command interface but greater than or equal to a fourth threshold distance, smaller than the third threshold distance, from said command interface; and

wherein a vertical distance of said at least one command object less than the third threshold distance but greater than or equal to the fourth threshold distance corresponds to the display of the second command.

47. The non-transitory computer-readable storage medium of claim 46, wherein the first threshold distance is a predetermined distance and the second, third and fourth threshold distances are determined as functions of a number of commands or hierarchical levels of commands corresponding to the selected one set of commands.

48. The non-transitory computer-readable storage medium of claim 33, wherein the respective location on the display is the same as a location of the selected one set of commands, such that displaying the first command or the second command comprises superimposing the first command or the second command substantially over the selected set of commands.

49. The non-transitory computer-readable storage medium of claim 33, wherein the respective location on the display is different than, but proximate to, a location of the selected set of commands.

50. Non-transitory computer-readable storage medium of claim 33, wherein the browsing further includes navigating from the selected one set of commands to the first or second command based on horizontal position data.

51. The non-transitory computer-readable storage medium of claim 33, the method further comprising selecting a particular command of the commands in the hierarchical levels of commands by verifying at least one selection condition based on the vertical distance data, wherein the at least one selection condition further comprises one or more of:

the distance of the at least one command object is less than a predetermined selection distance,

the distance of the at least one command object is less than a predetermined selection distance during a predetermined minimum duration, and

the at least one command object is in contact with the surface of the command interface.