USER INTERFACE FOR MOBILE DEVICES

• US 20100174421A1
• Filed: 03/11/2009
• Published: 07/08/2010
• Est. Priority Date: 01/06/2009
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: TSAI, Ming-Chang, Chang, Chienchung
• Granted Patent: US 8,441,441 B2
• Time in Patent Office: Days
• US Class Current: 700/302
• CPC Class Codes: G06F 1/1616 with folding flat displays, e.g. laptop computers or notebooks having a clamshell configuration, with body parts pivoting to an open position around an axis parallel to the plane they define in closed position
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1684 Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus

Abstract

A mobile user interface suitable for mobile computing devices uses device position/orientation in real space to select a portion of content that is displayed. Content (e.g., documents, files or a desktop) is presumed fixed in virtual space with the mobile user interface displaying a portion of the content as if viewed through a camera or magnifying glass. Data from motion, distance or position sensors are used to determine the relative position/orientation of the device with respect to the content to select the portion for display. Content elements can be selected by centering the display on the desired portion, obviating the need for cursors and pointing devices (e.g., mouse or touchscreen). Magnification can be manipulated by moving the device away from or towards the user. 3-D content viewing may be enabled by sensing the device orientation and displaying content that is above or below the display in 3-D virtual space.

Claims

1. A method for providing a mobile user interface on a mobile computing device, comprising:

sensing a movement in real space of the mobile computing;

determining a change in position in real space of the mobile computing device based upon the sensed movement;

determining a viewing perspective of a content treated as fixed in virtual space based upon the determined change in position of the mobile computing device; and

generating a display of at least a portion of the content based upon the determined viewing perspective.

2. The method of claim 1, further comprising:

receiving a signal to select a selection of the content;

determining a part of the content located near a center of the display; and

selecting the part of the content located near a center of the display.

3. The method of claim 2, further comprising including a selection aid within the generated display, wherein the selection aid indicates one of the center of the display and the part of the content within the display that will be selected.

4. The method of claim 3, further comprising:

selecting a form of the selection aid depending upon the content located near the center of the display,

wherein the form of the selection aid is selected from the group consisting of crosshairs, a circle, an oval, a box, a region of enhanced brightness, a region of enhanced definition, and a region of shading.

5. The method of claim 2, wherein the signal to select a selection of the content is received as a result of a button press.

6. The method of claim 1, further comprising:

including a selection aid within the generated display, wherein the selection aid indicates a part of the content within the display that can be selected;

receiving a signal to select a selection of the content;

determining the part of the content located within the selection aid; and

performing an action on the part of the content located within the selection aid.

7. The method of claim 1, wherein sensing a movement in real space comprises receiving a signal from a sensor selected from the group consisting of an accelerometer, a distance measuring sensor, a tilt sensor, a camera, and a compass.

8. The method of claim 1, further comprising:

determining a distance between the mobile computing device and a user; and

applying a magnification to the displayed portion of the content based upon the determined distance.

9. The method of claim 1, further comprising:

sensing tilt of the mobile computing device; and

repositioning the content in virtual space based upon the sensed tilt.

10. The method of claim 1, wherein sensing a movement in real space comprises:

receiving a digital image from a camera;

comparing at least a portion of the received digital image to at least a portion of a previous digital image to identify a change in sequential images; and

determining a movement based upon the identified change in the sequential images.

11. The method of claim 1, wherein determining the viewing perspective of the content treated as fixed in virtual space is further based upon a virtual movement scaling factor applied to the determined change in position of the mobile computing device.

12. The method of claim 1, further comprising:

receiving a user interface mode selection input while in a graphical user interface mode; and

fixing the content in virtual space based upon a display of the content in the graphical user interface mode.

13. The method of claim 11, further comprising:

receiving a user interface mode selection input while in the mobile user interface mode;

fixing the content in the a mobile computing device display; and

implementing the graphical user interface mode.

14. The method of claim 12, wherein the user interface mode selection input is received as a result of a button press.

15. A mobile computing device, comprising:

a processor;

a display coupled to the processor; and

a sensor coupled to the processor, the sensor selected from the group consisting of an accelerometer, a distance measuring sensor, a tilt sensor, a camera, and a compass,

wherein the processor is configured with software executable instructions to perform operations comprising;

receiving a signal from the sensor;

determining a change in position in real space of the mobile computing device based upon the receive sensor signal;

determining a viewing perspective of a content treated as fixed in virtual space based upon the determined change in position of the mobile computing device; and

generating an image on the display of at least a portion of the content based upon the determined viewing perspective.

16. The mobile computing device of claim 15, wherein the processor is configured with software executable instructions to perform operations further comprising:

receiving a signal to select a selection of the content;

determining a part of the content located near a center of the display; and

selecting the part of the content located near a center of the display.

17. The mobile computing device of claim 16, wherein the processor is configured with software executable instructions to perform operations further comprising including a selection aid within the generated display, wherein the selection aid indicates one of the center of the display and the part of the content within the display that will be selected.

18. The mobile computing device of claim 17, wherein the processor is configured with software executable instructions to perform operations further comprising:

selecting a form of the selection aid depending upon the content located near the center of the display,

wherein the form of the selection aid is selected from the group consisting of crosshairs, a circle, an oval, a box, a region of enhanced brightness, a region of enhanced definition, and a region of shading.

19. The mobile computing device of claim 16, further comprising a button coupled to the processor wherein the signal to select a selection of the content is received as a result of a press of the button.

20. The mobile computing device of claim 15, wherein the processor is configured with software executable instructions to perform operations further comprising:

including a selection aid within the generated display, wherein the selection aid indicates a part of the content within the display that can be selected;

receiving a signal to select a selection of the content;

determining the part of the content located within the selection aid; and

performing an action on the part of the content located within the selection aid.

21. The mobile computing device of claim 15, wherein the processor is configured with software executable instructions to perform operations sensing movement based upon the signal received from the sensor.

22. The mobile computing device of claim 15, wherein the processor is configured with software executable instructions to perform operations further comprising:

determining a distance between the mobile computing device and a user; and

applying a magnification to the displayed portion of the content based upon the determined distance.

23. The mobile computing device of claim 15, further comprising a tilt sensor coupled to the processor, the tilt sensor configured to sense tilt of the mobile computing device,

wherein the processor is configured with software executable instructions to perform operations further comprising repositioning the content in virtual space based upon the sensed tilt.

24. The mobile computing device of claim 15, wherein:

the sensor comprises a camera; and

wherein the processor is configured with software executable instructions to perform operations such that sensing a movement in real space comprises;

receiving a digital image from the camera;

comparing at least a portion of the received digital image to at least a portion of a previous digital image to identify a change in sequential images; and

determining a movement based upon the identified change in the sequential images.

25. The mobile computing device of claim 15, wherein the processor is configured with software executable instructions to perform operation such that determining the viewing perspective of the content treated as fixed in virtual space is further based upon a virtual movement scaling factor applied to the determined change in position of the mobile computing device.

26. The mobile computing device of claim 15, wherein the processor is configured with software executable instructions to perform operations further comprising:

receiving a user interface mode selection input while in a graphical user interface mode;

fixing the content in virtual space based upon a display of the content in the graphical user interface mode; and

implementing a mobile user interface mode.

27. The mobile computing device of claim 26, wherein the processor is configured with software executable instructions to perform operations further comprising:

receiving a user interface mode selection input while in the mobile user interface mode;

fixing the content in the a mobile computing device display; and

implementing the graphical user interface mode.

28. The mobile computing device of claim 27, further comprising a button, wherein the user interface mode selection input is received as a result of a button press.

29. A mobile computing device, comprising:

means for sensing a movement in real space of the mobile computing;

means for determining a change in position in real space of the mobile computing device based upon the sensed movement;

means for determining a viewing perspective of a content treated as fixed in virtual space based upon the determined change in position of the mobile computing device; and

means for generating a display of at least a portion of the content based upon the determined viewing perspective.

30. The mobile computing device of claim 29, further comprising:

means for receiving a signal to select a selection of the content;

means for determining a part of the content located near a center of the display; and

means for selecting the part of the content located near a center of the display.

31. The mobile computing device of claim 30, further comprising means for including a selection aid within the generated display, wherein the selection aid indicates one of the center of the display and the part of the content within the display that will be selected.

32. The mobile computing device of claim 31, further comprising:

means for selecting a form of the selection aid depending upon the content located near the center of the display,

wherein the form of the selection aid is selected from the group consisting of crosshairs, a circle, an oval, a box, a region of enhanced brightness, a region of enhanced definition, and a region of shading.

33. The mobile computing device of claim 30, wherein means for receiving a signal to select a selection of the content comprises means for an indication of a button press.

34. The mobile computing device of claim 29, further comprising:

means for including a selection aid within the generated display, wherein the selection aid indicates a part of the content within the display that can be selected;

means for receiving a signal to select a selection of the content;

means for determining the part of the content located within the selection aid; and

means for performing an action on the part of the content located within the selection aid.

35. The mobile computing device of claim 29, wherein means for sensing a movement in real space comprises means for receiving a signal from a sensor selected from the group consisting of:

means for sensing accelerometer;

means for measuring a distance

means for measuring a tilt;

means for obtaining a digital image; and

means for measuring orientation with respect to the Earth'"'"'s magnetic field.

36. The mobile computing device of claim 29, further comprising:

means for determining a distance between the mobile computing device and a user; and

means for applying a magnification to the displayed portion of the content based upon the determined distance.

37. The mobile computing device of claim 29, further comprising:

means for sensing tilt of the mobile computing device; and

means for repositioning the content in virtual space based upon the sensed tilt.

38. The mobile computing device of claim 29, wherein sensing a movement in real space comprises:

means for obtaining a digital image;

means for comparing at least a portion of a first digital image to at least a portion of a second digital image to identify a change in sequential images; and

means for determining a movement based upon the identified change in the sequential images.

39. The mobile computing device of claim 29, wherein means for determining the viewing perspective of the content treated as fixed in virtual space includes means for determining the viewing perspective of the content based upon a virtual movement scaling factor applied to the determined change in position of the mobile computing device.

40. The mobile computing device of claim 29, further comprising:

means for receiving a user interface mode selection input while in a graphical user interface mode;

means for fixing the content in virtual space based upon a display of the content in the graphical user interface mode; and

means for implementing a mobile user interface mode.

41. The mobile computing device of claim 40, further comprising:

means for receiving a user interface mode selection input while in the mobile user interface mode;

means for fixing the content in the a mobile computing device display; and

means for implementing the graphical user interface mode.

42. The mobile computing device of claim 41, wherein means for receiving a user interface mode selection input comprising means for receiving an indication of a button press.

43. A computer program product, comprising:

a computer-readable medium, comprising;

at least one instruction for receiving a signal from a sensor selected from the group consisting of an accelerometer, a distance measuring sensor, a tilt sensor, a camera, and a compass;

at least one instruction for determining a change in position in real space of a mobile computing device based upon the receive sensor signal;

at least one instruction for determining a viewing perspective of a content treated as fixed in virtual space based upon the determined change in position of the mobile computing device; and

at least one instruction for generating an image on a display of at least a portion of the content based upon the determined viewing perspective.

44. The computer program product of claim 43, wherein the computer-readable medium further comprises:

at least one instruction for receiving a signal to select a selection of the content;

at least one instruction for determining a part of the content located near a center of the display; and

at least one instruction for selecting the part of the content located near a center of the display.

45. The computer program product of claim 44, the computer-readable medium further comprises at least one instruction for including a selection aid within the generated display, wherein the selection aid indicates one of the center of the display and the part of the content within the display that will be selected.

46. The computer program product of claim 45, the computer-readable medium further comprises:

at least one instruction for selecting a form of the selection aid depending upon the content located near the center of the display,

wherein the form of the selection aid is selected from the group consisting of crosshairs, a circle, an oval, a box, a region of enhanced brightness, a region of enhanced definition, and a region of shading.

47. The computer program product of claim 44, the computer-readable medium further comprises at least one instruction for receiving the signal to select a selection of the content as a result of a button press.

48. The computer program product of claim 43, the computer-readable medium further comprises:

at least one instruction for including a selection aid within the generated display, wherein the selection aid indicates a part of the content within the display that can be selected;

at least one instruction for receiving a signal to select a selection of the content;

at least one instruction for determining the part of the content located within the selection aid; and

at least one instruction for performing an action on the part of the content located within the selection aid.

49. The computer program product of claim 43, wherein the computer-readable medium further comprises at least one instruction for sensing movement based upon the signal received from the sensor.

50. The computer program product of claim 43, wherein the computer-readable medium further comprises:

at least one instruction for determining a distance between the mobile computing device and a user; and

at least one instruction for applying a magnification to the displayed portion of the content based upon the determined distance.

51. The computer program product of claim 43, wherein the computer-readable medium further comprises at least one instruction for repositioning the content in virtual space based upon the sensed tilt.

52. The computer program product of claim 43, wherein the computer-readable medium such that sensing a movement in real space comprises:

at least one instruction for receiving a digital image from the camera;

at least one instruction for comparing at least a portion of the received digital image to at least a portion of a previous digital image to identify a change in sequential images; and

at least one instruction for determining a movement based upon the identified change in the sequential images.

53. The computer program product of claim 43, wherein the at least one instruction for determining the viewing perspective of the content treated as fixed in virtual space includes at least one instruction for determining the viewing perspective further based upon a virtual movement scaling factor applied to the determined change in position of the mobile computing device.

54. The computer program product of claim 43, wherein the computer-readable medium further comprises:

at least one instruction for receiving a user interface mode selection input while in a graphical user interface mode;

at least one instruction for fixing the content in virtual space based upon a display of the content in the graphical user interface mode; and

at least one instruction for implementing a mobile user interface mode.

55. The computer program product of claim 54, wherein the computer-readable medium further comprises:

at least one instruction for receiving a user interface mode selection input while in the mobile user interface mode;

at least one instruction for fixing the content in the a mobile computing device display; and

at least one instruction for implementing the graphical user interface mode.

56. The computer program product of claim 55, the computer-readable medium further comprises at least one instruction for receiving the user interface mode selection input as a result of a button press.