USER INTERFACE ELEMENTS AUGMENTED WITH FORCE DETECTION

• US 20120293551A1
• Filed: 03/09/2012
• Published: 11/22/2012
• Est. Priority Date: 05/19/2011
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Momeyer, Brian, Beckwith, Kevin M.
• Granted Patent: US 8,952,987 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

A computing device includes a touch screen display with at least one force sensor, each of which provides a signal in response to contact with the touch screen display. Using force signals from the at least one force sensor that result from contact with the touch screen, the operation of the computing device may be controlled, e.g. to select one of a plurality of overlaying interface elements, to prevent the unintended activation of suspect commands that require secondary confirmation, and to mimic the force requirements of real-world objects in augmented reality applications.

Claims

1. A method comprising:

displaying a plurality of selectable objects that are at least partially overlapping on a touch screen display operatively coupled to a computing device;

determining a force level of a contact with the touch screen display using at least one force sensor coupled to the touch screen display;

using the force level of the contact to select an object from the plurality of selectable objects; and

controlling the computing device based on the object selected.

2. The method of claim 1, further comprising detecting a location of the contact on the touch screen display and using the location and the force level to select the object.

3. The method of claim 1, wherein the plurality of selectable objects comprises at least one semi-transparent object.

4. The method of claim 3, wherein the at least one semi-transparent object is displayed as opaque when the at least one semi-transparent object is the object selected.

5. The method of claim 1, wherein controlling the computing device comprises causing the touch screen display to indicate the object that is selected.

6. The method of claim 1, wherein controlling the computing device comprises causing haptic feedback to indicate the object is selected.

7. The method of claim 1, further comprising determining a change in the force level of the contact and selecting a different object based on the change.

8. The method of claim 1, further comprising detecting a termination of the contact and using the termination of the contact to accept the object selected.

9. An apparatus comprising:

a housing;

a touch screen display comprising a display and at least one force sensor coupled between the housing and the display; and

a processor connected to receive data from the at least one force sensor, wherein the processor is configured to cause the touch screen display to display a plurality of selectable objects that are at least partially overlapping, determine a force level of a contact with the touch screen display based on the data from the at least one force sensor, use the force level of the contact to select an object from the plurality of selectable objects, and to initiate a command associated with the object selected.

10. The apparatus of claim 9, wherein the touch screen display further comprises a touch sensor coupled to the display, the processor coupled to receive data from the touch sensor, wherein the processor is further configured to detect a location of the contact on the touch screen display based on the data from the touch sensor and to use the location and the force level to select the object.

11. The apparatus of claim 9, wherein the plurality of selectable objects comprises at least one semi-transparent object.

12. The apparatus of claim 11, wherein the processor is configured to cause the at least one semi-transparent object to be displayed as opaque when the at least one semi-transparent object is the object selected.

13. The apparatus of claim 9, wherein the processor is further configured to cause the touch screen display to indicate the object that is selected.

14. The apparatus of claim 9, wherein the processor is further configured to cause haptic feedback to indicate the object is selected.

15. The apparatus of claim 9, wherein the processor is further configured to determine a change in the force level of the contact based on the data from the at least one force sensor and to select a different object based on the change.

16. The apparatus of claim 9, wherein the processor is further configured to detect a termination of the contact and to use the termination of the contact to accept the object selected.

17. An apparatus comprising:

means for displaying a plurality of selectable objects that are at least partially overlapping on a touch screen display operatively coupled to a computing device;

means for determining a force level of a contact with the touch screen display using at least one force sensor coupled to the touch screen display;

means for using the force level of the contact to select an object from the plurality of selectable objects; and

means for controlling the computing device based on the object selected.

18. The apparatus of claim 17, further comprising means for detecting a location of the contact on the touch screen display and using the location and the force level to select the object.

19. A non-transitory computer-readable medium including program code stored thereon, comprising:

program code to display a plurality of selectable objects that are at least partially overlapping on a touch screen display operatively coupled to a computing device;

program code to determine a force level of a contact with the touch screen display using at least one force sensor coupled to the touch screen display;

program code to use the force level of the contact to select an object from the plurality of selectable objects; and

program code to control the computing device based on the object selected.

20. The non-transitory computer-readable medium of claim 19, further comprising program code to detect a location of the contact on the touch screen display and using the location and the force level to select the object.

21. A method comprising:

detecting a contact with a selectable object on a touch screen display, the selectable object is associated with a command to control a computing device;

determining that the command to control the computing device requires secondary confirmation;

determining a force level of the contact with the touch screen display using at least one force sensor coupled to the touch screen display; and

initiating the command in response to the contact with the selectable object and the force level.

22. The method of claim 21, further comprising comparing the force level to a threshold, wherein the command is initiated when the force level is greater than the threshold.

23. The method of claim 21, further comprising comparing the force level to a threshold, wherein a secondary dialog box is displayed when the force level is less than the threshold.

24. The method of claim 21, wherein the command is determined to require secondary confirmation when performing the command may cause unusual, non-default, irreversible or potentially system dangerous actions.

25. The method of claim 21, wherein the command is at least one of a non-default option, command to manipulate system files or directories, open content in non-default application, and sending email with spelling errors, grammatical errors, offensive content, and multiple recipients.

26. An apparatus comprising:

a housing;

a touch screen display comprising a display, a touch sensor coupled to the display, and at least one force sensor coupled between the housing and the display; and

a processor connected to receive data from the touch sensor and the at least one force sensor, wherein the processor is configured to detect a contact with a selectable object on the touch screen display based on the data from the touch sensor, the selectable object is associated with a command, determine that the command requires secondary confirmation, determine a force level of the contact with the touch screen display based on data from the at least one force sensor, and initiate the command in response to the contact with the selectable object and the force level.

27. The apparatus of claim 26, wherein the processor is further configured to compare the force level to a threshold, wherein the command is initiated when the force level is greater than the threshold.

28. The apparatus of claim 26, wherein the processor is further configured to compare the force level to a threshold and to cause the display to display a secondary dialog box when the force level is less than the threshold.

29. The apparatus of claim 26, wherein the command is determined to require secondary confirmation when performing the command may cause unusual, irreversible, non-default, or potentially system dangerous actions.

30. The apparatus of claim 26, wherein the command is at least one of a non-default option, command to manipulate system files or directories, open content in non-default application, and sending email with spelling errors, grammatical errors, offensive content, and multiple recipients.

31. An apparatus comprising:

means for detecting a contact with a selectable object on a touch screen display, the selectable object is associated with a command to control a computing device;

means for determining that the command to control the computing device requires secondary confirmation;

means for determining a force level of the contact with the touch screen display using at least one force sensor coupled to the touch screen display; and

means for initiating the command in response to the contact with the selectable object and the force level.

32. The apparatus of claim 31, further comprising means for comparing the force level to a threshold, wherein the command is initiated when the force level is greater than the threshold.

33. A non-transitory computer-readable medium including program code stored thereon, comprising:

program code to detect a contact with a selectable object on a touch screen display, the selectable object is associated with a command to control a computing device;

program code to determine that the command to control the computing device requires secondary confirmation;

program code to determine a force level of the contact with the touch screen display using at least one force sensor coupled to the touch screen display; and

program code to initiate the command in response to the contact with the selectable object and the force level.

34. The non-transitory computer-readable medium of claim 33, further comprising program code to compare the force level to a threshold, wherein the command is initiated when the force level is greater than the threshold.

35. A method comprising:

displaying an augmented reality representation of a real-world device on a touch screen display operatively coupled to a computing device, wherein the augmented reality representation comprises a selectable object that is associated with a real-world selectable object on the real-world device, the real-world selectable object having a force requirement to select;

detecting a contact with the selectable object on the touch screen display;

determining a force level of the contact with the touch screen display using at least one force sensor coupled to the touch screen display;

comparing the force level to a force threshold that is associated with the force requirement; and

controlling the computing device in response to the contact when the force level is greater than the force threshold.

36. The method of claim 35, wherein the augmented reality representation comprises an image of the real-world device.

37. The method of claim 35, wherein controlling the computing device comprises manipulating the augmented reality representation of the real-world device.

38. The method of claim 35, wherein controlling the computing device comprises causing the computing device to control the real-world device.

39. The method of claim 35, wherein the force threshold is equal to the force requirement.

40. The method of claim 35, wherein the real-world device comprises a second real-world selectable object with a second force requirement, and wherein the force threshold is based on a ratio of the force requirement and the second force requirement.

41. An apparatus comprising:

a housing;

a camera to capture images of a real-world device with a real-world selectable object that has a force requirement to select;

a touch screen display comprising a display, a touch sensor coupled to the display, and at least one force sensor coupled between the housing and the display; and

a processor connected to receive the images from the camera and data from the touch sensor and the at least one force sensor, wherein the processor is configured to cause the display to display an augmented reality representation of the real-world device on the display, wherein the augmented reality representation comprises a selectable object that is associated with a real-world selectable object, detect a contact with the selectable object on the touch screen display based on the data from the touch sensor, determine a force level of the contact with the touch screen display based on data from the at least one force sensor, compare the force level to a force threshold that is associated with the force requirement, and to initiate a command associated with the selectable object in response to the contact when the force level is greater than the force threshold.

42. The apparatus of claim 41, wherein the augmented reality representation comprises an image of the real-world device.

43. The apparatus of claim 41, wherein the command associated with the selectable object causes the processor to manipulate the augmented reality representation of the real-world device.

44. The apparatus of claim 41, further comprising a wireless transmitter capable of wirelessly communicating with the real-world device, wherein the command associated with the selectable object causes the processor to control the real-world device with the wireless transmitter.

45. The apparatus of claim 41, wherein the force threshold is equal to the force requirement.

46. The apparatus of claim 41, wherein the real-world device comprises a second real-world selectable object with a second force requirement, and wherein the force threshold is based on a ratio of the force requirement and the second force requirement.

47. An apparatus comprising:

means for displaying an augmented reality representation of a real-world device on a touch screen display operatively coupled to a computing device, wherein the augmented reality representation comprises a selectable object that is associated with a real-world selectable object on the real-world device, the real-world selectable object having a force requirement to select;

means for detecting a contact with the selectable object on the touch screen display;

means for determining a force level of the contact with the touch screen display using at least one force sensor coupled to the touch screen display;

means for comparing the force level to a force threshold that is associated with the force requirement; and

means for controlling the computing device in response to the contact when the force level is greater than the force threshold.

48. A non-transitory computer-readable medium including program code stored thereon, comprising:

program code to display an augmented reality representation of a real-world device on a touch screen display operatively coupled to a computing device, wherein the augmented reality representation comprises a selectable object that is associated with a real-world selectable object on the real-world device, the real-world selectable object having a force requirement to select;

program code to detect a contact with the selectable object on the touch screen display;

program code to determine a force level of the contact with the touch screen display using at least one force sensor coupled to the touch screen display;

program code to compare the force level to a force threshold that is associated with the force requirement; and

program code to control the computing device in response to the contact when the force level is greater than the force threshold.