Touch and non touch based interaction of a user with a device

• US 9,448,714 B2
• Filed: 09/27/2012
• Issued: 09/20/2016
• Est. Priority Date: 09/27/2011
• Status: Active Grant

• Current Assignee: Elo Touch Solutions, Inc.
• Original Assignee: Elo Touch Solutions, Inc.
• Inventors: Duheille, Remi
• Primary Examiner(s): LEE, NICHOLAS J
• Application Number:
  US13/808,054
• Publication Number:
  US 20140055386A1
• Time in Patent Office: 1,454 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1684 Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

The invention relates to method for providing an input to a device characterized in that the input is based on a combination of at least one touch based interaction and at least one non-touch based interaction with the device, wherein the one type of input influences the action instructed to the device in response to the other type of input. The invention also relates to a corresponding device. To further improve the interaction between a user and a device, feedback providing means can be provided.

Claims

1. A method for interacting with a device to provide an input to the device characterized in that the interaction is based on a combination of at least one touch based interaction and at least one non-touch based interaction with the device, wherein the one type of interaction adapts the input attributed to the interaction with the device in response to the other type of interaction, comprising:

sensing a touch based interaction on the device;

sensing a non-touch based interaction;

in response to sensing the non-touch based interaction, adapting the input attributed to the non-touch based interaction; and

selecting an object and providing an instruction to the device in relation to the selected object by;

selecting the object using the touch based interaction;

sensing a gesture based on the non-touch based interaction;

classifying the gesture and depending on the classification, inputting the instruction to the device, wherein the instruction scales with at least one parameter independent of the gesture including at least one of a dimension of the device and a dimension of the object presented on the display; and

providing the instruction in relation to the selected object based on the non-touch based interaction.

2. The method according to claim 1, wherein at least one realization characteristic of the gesture influences the input attributed to the interaction with the device, the at least one realization characteristic selected from a group consisting of:

a trajectory of the gesture;

a speed of the gesture; and

a duration in time of the gesture.

3. The method according to claim 1, wherein the touch based interaction and the non-touch based interaction occur one after the other or alternately.

4. The method according to claim 1, wherein the touch based interaction and the non-touch based interaction overlap at least partially in time.

5. The method according to claim 1, wherein the touch based interaction occurs on at least one of a frontside, a backside or a side wall of the device.

6. The method according to claim 1, wherein the interaction comprises:

a registration phase for initiating the input to the device,

a continuation phase for providing the input to the device, and

a termination phase for confirming the input to the device.

7. The method according to claim 6, wherein in the termination phase the touch based interaction is used to confirm the input provided in the continuation phase via the non-touch based interaction.

8. The method according to claim 1, further comprising inputting the instruction to move the object on a display of the device by:

sensing a pinch or squeeze or rotation type gesture provided by the touch based interaction on the device at a first position;

sensing a translating gesture based on the non-touch based interaction in which the device is untouched;

sensing a release gesture by either a second touch based or a second non-touch based gesture at or above the device at a second position; and

moving the object on the display from the first position to the second position based on the pinch or squeeze or rotation gesture, the translating gesture, and the release gesture.

9. The method according to claim 1, further comprising:

sensing a second touch based interaction;

in response to the second touch based interaction, setting a parameter;

sensing at the same time a second non-touch based gesture; and

in response to the second non-touch based gesture and the parameter, instructing a corresponding action to the device.

16. The method according to claim 1, further comprising:

in response to sensing the touch based interaction, determining an axis of rotation for an object presented on a display of the device based on a location of the touch based interaction on the device; and

in response to sensing the non-touch based interaction, rotating the object presented on the display along the axis of rotation based on the non-touch based interaction.

18. The method according to claim 1, wherein the non-touch based interaction includes a second non-touch based interaction that is based on an environmental parameter including one or more of:

luminosity, an electric field, a magnetic field, electromagnetic radiation, pressure, temperature, humidity, vibrations, and geographical coordinates.

10. A computer program product, comprising one or more non-transitory computer readable media having computer-executable instructions that when executed by a processor cause the processor to perform a method comprising:

sensing a touch based interaction on a device;

sensing a non-touch based interaction;

in response to sensing the non-touch based interaction, adapting the input attributed to the non-touch based interaction; and

selecting an object and providing an instruction to the device in relation to the selected object by;

selecting the object using the touch based interaction;

sensing a gesture based on the non-touch based interaction;

classifying the gesture and depending on the classification, inputting the instruction to the device, wherein the instruction scales with at least one parameter independent of the gesture including at least one of a dimension of the device and a dimension of the object presented on the display; and

providing the instruction in relation to the selected object based on the non-touch based interaction.

11. A device, comprising:

a touch based user interaction component configured to sense touch based interactions;

a non-touch based user interaction component configured to sense non-touch based interactions within a predetermined volume around the device;

a feedback providing component configured to output a first signal including one or more of a visual and acoustic output when an object enters into the predetermined volume from outside of the predetermined volume; and

processing circuitry configured to;

determine a touch based interaction based on the touch based user interaction component sensing the touch based interaction on the device;

determine a non-touch based interaction based on the non-touch based user interaction component sensing the non-touch based interaction;

in response to sensing the non-touch based interaction, adapt the input attributed to the non-touch based interaction; and

select an object and providing an instruction to the device in relation to the selected object by;

selecting the object using the touch based interaction; and

providing the instruction in relation to the selected object based on the non-touch based interaction.

12. The device according to claim 11, wherein the feedback providing component is configured to output a second signal including one or more of a second visual, haptic, and acoustic output that is different from the first signal when the object is at least one of inside the predetermined volume and touching the device.

13. The device according to claim 12, wherein:

the feedback providing component is configured to output the second signal as long as at least one of;

the touch based interaction is sensed by the touch based user interaction component; and

the non-touch based interaction is sensed by the non-touch based user interaction component; and

the feedback providing component is configured to output a third signal including one or more of a third visual, acoustic, and haptic output when one or more of;

the touch based interaction is terminated;

the non-touch based interaction is terminated;

the device is switched off; and

the device is in an idle state.

14. The device according to claim 11, further comprising a display on its front side and wherein the feedback providing component is configured to provide a visual indication on the display when an object is detected in the predetermined volume on backside of the device.

17. The device according to claim 11, wherein the processing circuitry is further configured to:

in response to sensing the touch based interaction, determine an axis of rotation for an object presented on a display of the device based on a location of the touch based interaction on the device; and

in response to sensing the non-touch based interaction, rotate the object presented on the display along the axis of rotation based on the non-touch based interaction.

19. The device according to claim 11, wherein the non-touch based interaction includes a second non-touch based interaction that is based on an environmental parameter including one or more of:

luminosity, an electric field, a magnetic field, electromagnetic radiation, pressure, temperature, humidity, vibrations, and geographical coordinates.

15. A device, comprising:

at least one touch based user interaction component configured to sense touch based interactions;

at least one non-touch based user interaction component configured to sense non-touch based interactions; and

processing circuitry configured to;

determine a touch based interaction based on the at least one touch based user interaction component sensing the touch based interaction on the device;

determine a non-touch based interaction based on the at least one non-touch based user interaction component sensing the non-touch based interaction;

determine a pinch or squeeze or rotation type gesture provided by a second touch based interaction on the device at a first position;

determine a translating gesture based on a second non-touch based interaction in which the device is untouched;

determine a release gesture by either a third touch based or a third non-touch based gesture at or above the device at a second position; and

move the object in the display from the first position to the second position based on the pinch or squeeze or rotation gesture, the translating gesture, and the release gesture.