Interacting with an electronic device through physical movement

• US 10,558,278 B2
• Filed: 03/27/2018
• Issued: 02/11/2020
• Est. Priority Date: 07/11/2017
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Bernstein, Jeffrey Traer, Girling, Lukas Robert Tom, Missig, Julian N., Abdollahian, Golnaz
• Primary Examiner(s): Castiaux, Brent D
• Application Number:
  US15/937,716
• Publication Number:
  US 20190018506A1
• Time in Patent Office: 686 Days
• CPC Class Codes: A61B 5/02416 using photoplethysmograph signals, e.g. generated by infrared radiation A61B5/14552 takes precedence
• CPC Class Codes: A61B 5/14552 Details of sensors specially adapted therefor
• CPC Class Codes: G04C 3/002 Position, e.g. inclination dependent switches
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/0487 using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions
• CPC Class Codes: H04M 1/72469 for operating the device by selecting functions from two or more displayed items, e.g. menus or icons
• CPC Class Codes: H04M 2250/12 including a sensor for measuring a physical value, e.g. temperature or motion telephones including GPS signal receivers H04M2250/10

Abstract

The present disclosure generally relates to interacting with an electronic device without touching a display screen or other physical input mechanisms. In some examples, the electronic device performs an operation in response to a positioning of a user'"'"'s hand and/or an orientation of the electronic device.

Claims

1. An electronic device, comprising:

a display screen;

a tilt sensor;

one or more processors; and

memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for;

displaying a first item at a first position on the display screen and a second item at a second position on the display screen, the first position and second position corresponding to positions along a line substantially perpendicular to an axis of rotation of the electronic device;

receiving a tilt sensor input associated with movement of the electronic device;

in accordance with a determination that the tilt sensor input corresponds to a rotation of the electronic device in a first direction around the axis of rotation from a neutral position to a first position;

moving the first item from the first position on the display screen to a third position along the line substantially perpendicular to the axis of rotation; and

in accordance with a determination that the tilt sensor input corresponds to a rotation of the electronic device in a second direction opposite the first direction around the axis of rotation from the neutral position to a second position;

moving the second item from the second position on the display screen to a fourth position along the line substantially perpendicular to the axis of rotation.

2. The electronic device of claim 1, wherein the axis of rotation of the electronic device corresponds to an axis of rotation of a user'"'"'s wrist.

3. The electronic device of claim 1, wherein the one or more programs further include instructions for:

in accordance with a determination that the tilt sensor input corresponds to a rotation of the electronic device in the first direction around the axis of rotation from the neutral position to the first position and from the first position back toward the neutral position within a first predetermined time;

moving the first item from the first position on the display screen to the third position along the line substantially perpendicular to the axis of rotation; and

in accordance with a determination that the tilt sensor input corresponds to a rotation of the electronic device in the second direction opposite the first direction around the axis of rotation from the neutral position to the second position and from the second position back toward the neutral position within a second predetermined time;

moving the second item from the second position on the display screen o the fourth position along the line substantially perpendicular to the axis of rotation.

4. The electronic device of claim 1, wherein the one or more programs further include instructions for:

after moving the first or second item, and in accordance with a determination that no additional tilt sensor input above a threshold value is received within a predetermined time, selecting the moved item.

5. The electronic device of claim 4, wherein the one or more programs further include instructions for:

displaying a countdown indicator corresponding to the predetermined time.

6. The electronic device of claim 1, wherein the electronic device further comprises a haptic feedback mechanism, and wherein the one or more programs further include instructions for:

further in accordance with the determination that no additional tilt sensor input above the threshold value is received within the predetermined time, providing a haptic feedback, via the haptic feedback mechanism.

7. The electronic device of claim 1, wherein the first position is in an upper half of the display screen and the second position is in a lower half of the display screen.

8. The electronic device of claim 1, wherein the third and fourth positions are in a center region of the display screen.

9. The electronic device of claim 1, wherein the one or more programs further include instructions for:

displaying an indicator that indicates the direction of rotation of the electronic device.

10. The electronic device of claim 1, wherein the electronic device further comprises a haptic feedback mechanism, and wherein the one or more programs further include instructions for:

further in accordance with the determination that the tilt sensor input corresponds to rotation of the electronic device in the first or second direction, providing a haptic feedback, via the haptic feedback mechanism.

11. The electronic device of claim 1, wherein the tilt sensor input corresponds to rotation of the electronic device in the first or second direction at a velocity greater than a predetermined minimum velocity.

12. The electronic device of claim 1, wherein the one or more programs further include instructions for:

in accordance with a determination that the tilt sensor input is not greater an a predetermined minimum velocity, forgoing moving the first or second item.

13. The electronic device of claim 1, wherein the tilt sensor input corresponds to rotation of the electronic device in the first or second direction with an acceleration greater than a predetermined minimum acceleration.

14. The electronic device of claim 1, wherein the one or more programs further include instructions for:

in accordance with a determination that the tilt sensor input is not greater than a predetermined minimum acceleration, forgoing moving the first or second item.

15. The electronic device of claim 1, wherein the tilt sensor includes an accelerometer, directional sensor, gyroscope, motion sensor, or a combination thereof.

16. The electronic device of claim 1, wherein the first or second item comprises a reply command for a received text message, a dismiss command for a received text message, a predefined response to a received text message, an answer command for a telephone call, a decline command for a telephone call, or a combination thereof.

17. A method, comprising:

at an electronic device with a display screen and a tilt sensor;

displaying a first item at a first position on the display screen and a second item at a second position on the display screen, the first position and second position corresponding to positions along a line substantially perpendicular to an axis of rotation of the electronic device;

receiving a tilt sensor input associated with movement of the electronic device;

in accordance with a determination that the tilt sensor input corresponds to a rotation of the electronic device in a first direction around the axis of rotation from a neutral position to a first position;

moving the first item from the first position on the display screen to a third position along the line substantially perpendicular to the axis of rotation; and

in accordance with a determination that the tilt sensor input corresponds to a rotation of the electronic device in a second direction opposite the first direction around the axis of rotation from the neutral position to a second position;

moving the second item from the second position on the display screen to a fourth position along the line substantially perpendicular to the axis of rotation.

18. The method of claim 17, wherein the axis of rotation of the electronic device corresponds to an axis of rotation of a user'"'"'s wrist.

19. The method of claim 17, further comprising:

in accordance with a determination that the tilt sensor input corresponds to a rotation of the electronic device in the first direction around the axis of rotation from the neutral position to the first position and from the first position back toward the neutral position within a first predetermined time;

moving the first item from the first position on the display screen to the third position along the line substantially perpendicular to the axis of rotation; and

in accordance with a determination that the tilt sensor input corresponds to a rotation of the electronic device in the second direction opposite the first direction around the axis of rotation from the neutral position to the second position and from the second position back toward the neutral position within a second predetermined time;

moving the second item from the second position on the display screen to the fourth position along the line substantially perpendicular to the axis of rotation.

20. The method of claim 17, further comprising:

after moving the first or second item, and in accordance with a determination that no additional tilt sensor input above a threshold value is received within a predetermined time, selecting the moved item.

21. The method of claim 20, further comprising:

displaying a countdown indicator corresponding to the predetermined time.

22. The method of claim 17, wherein the electronic device further comprises a haptic feedback mechanism, the method further comprising:

further in accordance with the determination that no additional tilt sensor input above the threshold value is received within the predetermined time, providing a haptic feedback, via the haptic feedback mechanism.

23. The method of claim 17, wherein the first position is in an upper half of the display screen and the second position is in a lower half of the display screen.

24. The method of claim 17, wherein the third and fourth positions are in a center region of the display screen.

25. The method of claim 17, further comprising:

displaying an indicator that indicates the direction of rotation of the electronic device.

26. The method of claim 17, wherein the electronic device further comprises a haptic feedback mechanism, the method further comprising:

further in accordance with the determination that the tilt sensor input corresponds to rotation of the electronic device in the first or second direction, providing a haptic feedback, via the haptic feedback mechanism.

27. The method of claim 17, wherein the tilt sensor input corresponds to rotation of the electronic device in the first or second direction at a velocity greater than a predetermined minimum velocity.

28. The method of claim 17, further comprising:

in accordance with a determination that the tilt sensor input is not greater than a predetermined minimum velocity, forgoing moving the first or second item.

29. The method of claim 17, wherein the tilt sensor input corresponds to rotation of the electronic device in the first or second direction with an acceleration greater than a predetermined minimum acceleration.

30. The method of claim 17, further comprising:

in accordance with a determination that the tilt sensor input is not greater than a predetermined minimum acceleration, forgoing moving the first or second item.

31. The method of claim 17, wherein the tilt sensor includes an accelerometer, directional sensor, gyroscope, motion sensor, or a combination thereof.

32. The method of claim 17, wherein the first or second item comprises a reply command for a received text message, a dismiss command for a received text message, a predefined response to a received text message, an answer command for a telephone call, a decline command for a telephone call, or a combination thereof.

33. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device with a display screen and a tilt sensor, the one or more programs including instructions for:

displaying a first item at a first position on the display screen and a second item at a second position on the display screen, the first position and second position corresponding to positions along a line substantially perpendicular to an axis of rotation of the electronic device;

receiving a tilt sensor input associated with movement of the electronic device;

in accordance with a determination that the tilt sensor input corresponds to a rotation of the electronic device in a first direction around the axis of rotation from a neutral position to a first position;

moving the first item from the first position on the display screen to a third position along the line substantially perpendicular to the axis of rotation; and

in accordance with a determination that the tilt sensor input corresponds to a rotation of the electronic device in a second direction opposite the first direction around the axis of rotation from the neutral position to a second position;

moving the second item from the second position on the display screen to a fourth position along the line substantially perpendicular to the axis of rotation.

34. The non-transitory computer-readable storage medium of claim 33, wherein the axis of rotation of the electronic device corresponds to an axis of rotation of a user'"'"'s wrist.

35. The non-transitory computer-readable storage medium of claim 33, wherein the one or more programs further include instructions for:

in accordance with a determination that the tilt sensor input corresponds to a rotation of the electronic device in the first direction around the axis of rotation from the neutral position to the first position and from the first position back toward the neutral position within a first predetermined time;

moving the first item from the first position on the display screen to the third position along the line substantially perpendicular to the axis of rotation; and

in accordance with a determination that the tilt sensor input corresponds to a rotation of the electronic device in the second direction opposite the first direction around the axis of rotation from the neutral position to the second position and from the second position back toward the neutral position within a second predetermined time;

moving the second item from the second position on the display screen to the fourth position along the line substantially perpendicular to the axis of rotation.

36. The non-transitory computer-readable storage medium of claim 33, wherein the one or more programs further include instructions for:

after moving the first or second item, and in accordance with a determination that no additional tilt sensor input above a threshold value is received within a predetermined time, selecting the moved item.

37. The non-transitory computer-readable storage medium of claim 36, wherein the one or more programs further include instructions for:

displaying a countdown indicator corresponding to the predetermined time.

38. The non-transitory computer-readable storage medium of claim 33, wherein the electronic device further comprises a haptic feedback mechanism, and wherein the one or more programs further include instructions for:

further in accordance with the determination that no additional tilt sensor input above the threshold value is received within the predetermined time, providing a haptic feedback, via the haptic feedback mechanism.

39. The non-transitory computer-readable storage medium of claim 33, wherein the first position is in an upper half of the display screen and the second position is in a lower half of the display screen.

40. The non-transitory computer-readable storage medium of claim 33, wherein the third and fourth positions are in a center region of the display screen.

41. The non-transitory computer-readable storage medium of claim 33, wherein the one or more programs further include instructions for:

displaying an indicator that indicates the direction of rotation of the electronic device.

42. The non-transitory computer-readable storage medium of claim 33, wherein the electronic device further comprises a haptic feedback mechanism, and wherein the one or more programs further include instructions for:

further in accordance with the determination that the tilt sensor input corresponds to rotation of the electronic device in the first or second direction, providing a haptic feedback, via the haptic feedback mechanism.

43. The non-transitory computer-readable storage medium of claim 33, wherein the tilt sensor input corresponds to rotation of the electronic device in the first or second direction at a velocity greater than a predetermined minimum velocity.

44. The non-transitory computer-readable storage medium of claim 33, wherein the one or more programs further include instructions for:

in accordance with a determination that the tilt sensor input is not greater than a predetermined minimum velocity, forgoing moving the first or second item.

45. The non-transitory computer-readable storage medium of claim 33, wherein the tilt sensor input corresponds to rotation of the electronic device in the first or second direction with an acceleration greater than a predetermined minimum acceleration.

46. The non-transitory computer-readable storage medium of claim 33, wherein the one or more programs further include instructions for:

in accordance with a determination that the tilt sensor input is not greater than a predetermined minimum acceleration, forgoing moving the first or second item.

47. The non-transitory computer-readable storage medium of claim 33, wherein the tilt sensor includes an accelerometer, directional sensor, gyroscope, motion sensor, or a combination thereof.

48. The non-transitory computer-readable storage medium of claim 33, wherein the first or second item comprises a reply command for a received text message, a dismiss command for a received text message, a predefined response to a received text message, an answer command for a telephone call, a decline command for a telephone call, or a combination thereof.