CLICK-THROUGH CONTROLLER FOR MOBILE INTERACTION

• US 20100275122A1
• Filed: 04/27/2009
• Published: 10/28/2010
• Est. Priority Date: 04/27/2009
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Buxton, William A. S., SanGiovanni, John
• Application Number:
  US12/430,878
• Publication Number:
  US 20100275122A1
• Time in Patent Office: Days
• US Class Current: 715/728
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures

Abstract

A “Click-Through Controller” uses various mobile electronic devices (e.g., cell phones, media players, digital cameras, etc.) to provide real-time interaction with content (e.g., maps, places, images, documents, etc.) displayed on the device'"'"'s screen via selection of one or more “overlay menu items” displayed on top of that content. Navigation through displayed contents is provided by recognizing 2D and/or 3D device motions and rotations. This allows users to navigate through the displayed contents by simply moving the mobile device. Overlay menu items activate predefined or user-defined functions to interact with the content that is directly below the selected overlay menu item on the display. In various embodiments, there is a spatial correspondence between the overlay menu items and buttons or keys of the mobile device (e.g., a cell phone dial pad or the like) such that overlay menu items are directly activated by selection of one or more corresponding buttons.

Claims

1. A method for user interaction with electronic documents, comprising steps for:

displaying a view onto a region of an electronic document on a display screen of a portable electronic device, said electronic document being rendered relative to a (position in a virtual space;

displaying an overlay menu in a fixed position on the display screen on top of the electronic document, such that the electronic document is visible under the overlay menu, said overlay menu comprising a plurality of user selectable menu items;

using one or more spatial sensors within the portable electronic device to sense spatial changes of the portable electronic device;

modifying the view of the displayed electronic document relative to the sensed spatial changes of the electronic device, such that spatial changes of the portable electronic device results in shifting the view of the electronic document relative to the position in the virtual space, while maintaining the overlay menu in the fixed position on the display screen; and

wherein at least one of the menu items initiates a predetermined function relative to a particular portion of the electronic document beneath a selected menu item following user interaction with a corresponding one of the menu items.

2. The method of claim 1 further comprising steps for mapping one or more of the menu items to a corresponding button on the portable electronic device, and wherein user interaction with mapped menu items is accomplished by pressing the corresponding button.

3. The method of claim 1 wherein the display screen is a touch screen, and wherein user interaction with menu items is initiated by touching an area of the display screen which includes the selected menu item.

4. The method of claim 1 wherein the user interaction with menu items is initiated by recognizing user speech to activate the selected menu item.

5. The method of claim 1 wherein the display screen has stylus sensing capabilities, and wherein the user interaction with menu items is initiated by using the stylus to interact with an area of the display screen which includes the selected menu item.

6. The method of claim 1 wherein the overlay menu is selected from a set of overlay menus as a function of the type of electronic document being displayed on the display screen.

7. The method of claim 1 wherein the specific menu items comprising the overlay menu are included in the overlay menu as a function of content currently displayed on the display screen.

8. The method of claim 1 wherein one or more of the menu items represents a user definable function.

9. The method of claim 1 wherein each menu item is placed into a separate cell of a grid on the display screen.

10. The method of claim 9 wherein visibility of the grid is user selectable such that the user can turn the visibility of the grid on or off.

11. A system for interacting with digital content, comprising:

a portable electronic device having a display screen and spatial sensor capabilities for detecting spatial changes of the portable electronic device;

a device for rendering digital content to the display screen relative to a fixed position in a virtual space;

a device for rendering a plurality of user selectable menu items in fixed positions on the display screen on top of the digital content, such that the digital content is visible below the user selectable menu items;

a device for changing a view of the digital content on the display screen as a function of sensed spatial changes of the portable electronic device relative to the fixed position in the virtual space; and

a device for executing a function associated with any user selected menu item, and wherein at least one of the menu items initiates a function which interacts with a region of the digital content below that menu item.

12. The system of claim 11 further comprising a device for allowing the user to adjust the fixed position in virtual space.

13. The system of claim 11 further comprising a device for mapping one or more of the menu items to a corresponding key on the portable electronic device such that there is a spatial correspondence between the menu items and the keys, and wherein user selection of mapped menu items is accomplished by pressing the corresponding key.

14. The system of claim 11 wherein the overlay menu is selected from a set of overlay menus as a function of the type of digital content being rendered on the display screen.

15. The system of claim 11 wherein the digital content represents a scene from a digital camera coupled to the portable electronic device, and wherein user selection of one of the menu item allows the user to interact with corresponding objects in the scene being rendered on the display device.

16. The system of claim 11 wherein the digital content represents streaming video media.

17. A user interface implemented within a computing device, comprising:

a display screen;

means for sensing spatial changes of the computing device;

means for allowing the user to select specific digital content;

means for placing the selected digital content into a fixed, user adjustable, virtual position in a virtual space;

means for rendering a view on the display device of an initial region of the selected digital content relative to the fixed virtual position, said initial region corresponding to an initial real position of the computing device;

means for rendering a plurality of user selectable menu items in fixed positions on the display screen on top of the view of the initial region of the selected digital content, such that the region of digital content is visible below the user selectable menu items;

means for changing the view of the region of the digital content in direct correspondence to sensed spatial changes of the computing device relative to the initial position of the computing device and relative to the fixed virtual position of the digital content;

means for providing user selection of any of the menu items; and

means for executing a function associated with any user selected menu item, and wherein at least one of the menu items initiates a function which interacts with an area of the digital content below that menu item.

18. The user interface of claim 17 further comprising means for mapping one or more of the menu items to a corresponding key on the computing device such that there is a spatial correspondence between the menu items and the keys, and wherein user selection of mapped menu items is accomplished by pressing the corresponding key.

19. The user interface of claim 17 wherein the display screen is a touch screen, and further comprising means for allowing user selection of the menu items by touching an area of the display screen which includes the selected menu item.

20. The user interface of claim 17 wherein the computing device further comprises a digital camera, wherein the digital content represents a live view of a scene captured by the digital camera, and wherein user selection of one of the menu item allows the user to interact with corresponding objects in the scene.