Providing virtual buttons in a handheld device

• US 10,719,232 B2
• Filed: 06/08/2016
• Issued: 07/21/2020
• Est. Priority Date: 06/08/2016
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Tse, Justin
• Primary Examiner(s): Wills-Burns, Chineyere D
• Application Number:
  US15/177,151
• Publication Number:
  US 20170357440A1
• Time in Patent Office: 1,504 Days
• Field of Search: 345173
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0414 using force sensing means to determine a position
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06V 40/1306 non-optical, e.g. ultrasonic or capacitive sensing

Abstract

Embodiments of apparatus, computer program product, and method for providing virtual buttons are disclosed. In one embodiment, a method of providing virtual buttons in a device includes detecting a grip around a perimeter of the device, where the perimeter of the device includes one or more ultrasonic sensors; determining one or more locations of one or more fingers of a user using data collected by the one or more ultrasonic sensors; and providing one or more virtual buttons around the perimeter of the device based on the one or more locations of the one or more fingers of the user.

Claims

1. A method of providing virtual buttons in a device, comprising:

detecting a grip around a perimeter of the device, wherein the perimeter of the device includes one or more ultrasonic sensors;

determining one or more locations of one or more fingers, respectively, of a user using data collected by the one or more ultrasonic sensors;

providing one or more virtual buttons on a display screen of the device;

tracking a sliding movement of the one or more fingers along the perimeter of the device using the one or more ultrasonic sensors; and

repositioning the one or more virtual buttons by rotating or flipping the one or more virtual buttons dynamically in response to the sliding movement of the one or more fingers.

2. The method of claim 1, wherein determining the one or more locations of the one or more fingers of the user comprises:

determining a thumb location of the user;

determining an index finger location of the user;

determining a long finger location of the user;

determining a ring finger location of the user;

determining a little finger location of the user;

or some combination thereof.

3. The method of claim 1, wherein providing the one or more virtual buttons comprises:

positioning the one or more virtual buttons based upon the one or more locations of the one or more fingers of the user.

4. The method of claim 3, wherein providing the one or more virtual buttons further comprises:

detecting changes in orientation of the device and changes in the locations of the one or more fingers; and

repositioning the one or more virtual buttons dynamically with respect to the changes in orientation of the device and changes in the locations of the one or more fingers.

5. The method of claim 3, wherein providing the one or more virtual buttons further comprises:

providing a set of predicted virtual buttons based on a history of user executions stored by the device.

6. The method of claim 3, further comprising:

indicating positions of the one or more virtual buttons to the user through corresponding one or more haptic feedbacks; and

acknowledging selections of the one or more virtual buttons to the user through the corresponding one or more haptic feedbacks.

7. The method of claim 1, further comprising:

detecting a pressure applied to the one or more virtual buttons by the user; and

executing a command corresponding to the pressure applied to the one or more virtual buttons.

8. The method of claim 7, wherein the command corresponding to the pressure applied to the one or more virtual buttons by the user is dependent upon a determination of which finger is applying the pressure.

9. The method of claim 1, further comprising:

detecting the device being held by a second user, in an unlocked state of the device;

determining whether the second user is a second enrolled user using fingerprint images collected by the one or more ultrasonic sensors; and

providing a second set of virtual buttons corresponding to settings of the second user in response to a determination of the second user being the second enrolled user.

10. The method of claim 1, wherein detecting the grip around the perimeter of the device comprises:

determining whether the user is an enrolled user using fingerprint images collected by the one or more ultrasonic sensors; and

unlocking the device in response to a determination of the user being the enrolled user.

11. The method of claim 10, further comprising:

continuously monitoring subsequent operations of the device are being performed by the enrolled user; and

terminating an application program, or hold the application program in an unresponsive state in response to a determination that the subsequent operations of the device are not being performed by the enrolled user.

12. The method of claim 1, wherein tracking the sliding movement of the one or more fingers comprises tracking one or more fingers sliding linearly or rotationally along the perimeter of the device.

13. A device configured to provide virtual buttons, comprising:

one or more ultrasonic sensors configured to detect a grip around a perimeter of the device; and

one or more processors configured to;

determine one or more locations of one or more fingers, respectively, of a user using data collected by the one or more ultrasonic sensors;

provide one or more virtual buttons on a display screen of the device;

track a sliding movement of the one or more fingers along the perimeter of the device;

using the one or more ultrasonic sensors; and

reposition the one or more virtual buttons by rotating or flipping the one or more virtual buttons dynamically in response to the sliding movement of the one or more fingers.

14. The device of claim 13, wherein the one or more processors are further configured to:

determine a thumb location of the user;

determine an index finger location of the user;

determine a long finger location of the user;

determine a ring finger location of the user;

determine a little finger location of the user;

or some combination thereof.

15. The device of claim 13, wherein the one or more processors are further configured to:

position the one or more virtual buttons based upon the one or more locations of the one or more fingers of the user.

16. The device of claim 15, wherein the one or more processors are further configured to:

detect changes in orientation of the device and changes in the locations of the one or more fingers; and

reposition the one or more virtual buttons dynamically with respect to the changes in orientation of the device and changes in the locations of the one or more fingers.

17. The device of claim 15, wherein the one or more processors are further configured to:

provide a set of predicted virtual buttons based on a history of user executions stored by the device.

18. The device of claim 15, wherein the one or more processors are further configured to:

indicate positions of the one or more virtual buttons to the user through corresponding one or more haptic feedbacks; and

acknowledge selections of the one or more virtual buttons to the user through the corresponding one or more haptic feedbacks.

19. The device of claim 18, wherein the command corresponding to the pressure applied to the one or more virtual buttons by the user is dependent upon a determination of which finger is applying the pressure.

20. The device of claim 13, wherein the one or more processors are further configured to:

detect a pressure applied to the one or more virtual buttons by the user; and

execute a command corresponding to the pressure applied to the one or more virtual buttons.

21. The device of claim 13, wherein the one or more processors are further configured to:

detect the device being held by a second user, in an unlocked state of the device;

determine whether the second user is a second enrolled user using fingerprint images collected by the one or more ultrasonic sensors; and

provide a second set of virtual buttons corresponding to settings of the second user in response to a determination of the second user being the second enrolled user.

22. The device of claim 13, wherein the one or more processors are further configured to:

determine whether the user is an enrolled user using fingerprint images collected by the one or more ultrasonic sensors; and

unlock the device in response to a determination of the user being the enrolled user.

23. The device of claim 22, the one or more processors are further configured to:

continuously monitor subsequent operations of the device are being performed by the enrolled user; and

terminate an application program, or hold the application program in an unresponsive state in response to a determination that the subsequent operations of the device are not being performed by the enrolled user.

24. The method of claim 13, wherein the one or more processors are configured to track the sliding movement of the one or more fingers by being configured to track one or more fingers sliding linearly or rotationally along the perimeter of the device.

25. A device for providing virtual buttons, comprising:

means for detecting a grip around a perimeter of the device, wherein the perimeter of the device includes one or more ultrasonic sensors;

means for determining one or more locations of one or more fingers of a user, respectively, using data collected by the one or more ultrasonic sensors;

means for providing one or more virtual buttons on a display screen of the device;

means for tracking a sliding movement of the one or more fingers along the perimeter of the device using the one or more ultrasonic sensors; and

means for repositioning the one or more virtual buttons by rotating or flipping the one or more virtual buttons dynamically in response to the sliding movement of the one or more fingers.

26. The device of claim 25, wherein the means for providing the one or more virtual buttons comprises:

means for positioning the one or more virtual buttons based upon the one or more locations of the one or more fingers of the user.

27. A non-transitory computer-readable storage medium including instructions stored thereon that, when executed, cause a device to provide virtual buttons, comprising instruction configured to:

detect a grip around a perimeter of the device, wherein the perimeter of the device includes one or more ultrasonic sensors;

determine one or more locations of one or more fingers of a user using data collected by the one or more ultrasonic sensors;

provide one or more virtual buttons on a display screen of the device;

track a sliding movement of the one or more fingers along the perimeter of the device using the one or more ultrasonic sensors; and

reposition the one or more virtual buttons by rotating or flipping the one or more virtual buttons dynamically in response to the sliding movement of the one or more fingers.

28. The non-transitory computer-readable storage medium of claim 27, wherein the instruction is further configured to:

position the one or more virtual buttons based upon the one or more locations of the one or more fingers of the user.

29. The non-transitory computer-readable storage medium of claim 28, wherein the instruction is further configured to:

track movements of the one or more fingers along the perimeter of the device; and

reposition the one or more virtual buttons dynamically in response to the movements of the one or more fingers.