Method, apparatus and system for interacting with content on web browsers

• US 9,575,561 B2
• Filed: 12/23/2010
• Issued: 02/21/2017
• Est. Priority Date: 12/23/2010
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Elford, Christopher L., Tsoi, Howard P.
• Primary Examiner(s): Bashore, William
• Assistant Examiner(s): Parker, Jeanette J
• Application Number:
  US12/978,082
• Publication Number:
  US 20120166974A1
• Time in Patent Office: 2,252 Days
• Field of Search: 715/760, 715/815, 715/206, 715/249
• US Class Current: 1/1
• CPC Class Codes: G06F 16/95 Retrieval from the web
• CPC Class Codes: G06F 16/957 Browsing optimisation, e.g. caching or content distillation
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence

Abstract

A method, apparatus and system enable indirect remote interaction with a web browser. In one embodiment, remote user gestures may be captured and processed to determine an action to be taken by the web browser.

Claims

1. A method comprising:

analyzing code for a web page to be displayed by a web browser application on a first device to identify an actionable area on the web page, the code having additional information, corresponding to the actionable area, such that if the web page is displayed with a second device, the second device different from the first device and having a positioning cursor, the additional information is displayed in response to a floatover by the positioning cursor of the second device, wherein the first device does not include a positioning cursor;

replacing or augmenting the actionable area in the display of the web page by the web browser application on the first device with a visual cue, prior to user interaction with the displayed web page, to suggest a user gesture to effectuate selection of the actionable area;

identifying a user motion directed at the first device;

identifying the user motion as an input gesture;

processing the input gesture to determine a selection of the actionable area; and

providing the selection to the web browser application to enable the associated additional information of the selected actionable area to be displayed on the first device.

2. The method according to claim 1 wherein identifying the user motions as the input gestures comprises determining a movement associated with the input gestures.

3. The method according to claim 2 wherein processing the input gestures to determine the web browser action comprises processing the movement associated with the input gestures to determine a corresponding web browser action.

4. The method according to claim 3 wherein processing the movement associated with the user gestures comprises:

accessing a gesture library communicatively coupled to the web browser application;

identifying the movement within the gesture library; and

selecting the web browser action corresponding to the identified movement within the gesture library.

5. The method of claim 1, wherein providing the selection to the web browser application to enable the associated additional information of the selected actionable area to be displayed on the first device comprises providing the selection to the web browser application to enable the associated additional information of the selected actionable area to be displayed as an overlay on the web page.

6. At least one non-transitory machine-accessible medium having stored thereon instructions that, when executed by a first machine, cause the first machine to:

analyze code for a web page to be displayed by a web browser application operating on the first machine to identify an actionable area on the web page, the code having additional information, corresponding to the actionable area, such that if the web page is displayed with a second machine, the second machine different from the first machine and having a positioning cursor, the additional information is displayed in response to a floatover by the positioning cursor of the second machine, wherein the first machine does not include a positioning cursor;

replace or augment the actionable area in the display of the web page on the first machine with a visual cue, prior to user interaction with the displayed web page, to suggest a user gesture to effectuate selection of the actionable area;

identify a user motion directed at the first machine;

identify the user motion as an input gesture;

process the input gesture to determine a selection of the actionable area; and

provide the selection to the web browser application to enable the additional information associated with the selected actionable area to be displayed on the first machine.

7. The at least one non-transitory machine-accessible medium of claim 6 further having stored thereon instructions that, when executed by the first machine, cause the first machine to identify actions corresponding to the user motions by determining a movement associated with the input gestures.

8. The at least one non-transitory machine-accessible medium of claim 7 further having stored thereon instructions that, when executed by the first machine, cause the first machine to process the input gestures to determine the web browser action by processing the movement associated with the input gestures to determine a corresponding web browser action.

9. The at least one non-transitory machine-accessible medium of claim 8 further having stored thereon instructions that, when executed by the first machine, cause the first machine to:

access a gesture library communicatively coupled to the web browser application;

identify the movement within the gesture library; and

select the web browser action corresponding to the identified movement within the gesture library.

10. The at least one non-transitory machine-accessible medium of claim 6, wherein provide the selection to the web browser application to enable the associated additional information of the selected actionable area to be displayed on the first device comprises provide the selection to the web browser application to enable the associated additional information of the selected actionable area to be displayed as an overlay on the web page.

11. A method comprising:

analyzing code for a web page to be displayed on a first computing platform to determine an actionable area on the web page, the actionable area providing additional information to a user in response to a user selection of the actionable area, wherein the code is configured such that if the web page is displayed with a second computing platform, the second computing platform different from the first computing platform and having a positioning cursor, the additional information is displayed in response to a floatover by the positioning cursor of the second computing platform, wherein the first device does not include a positioning cursor; and

replacing or augmenting the actionable area in a display of the web page on the first computing platform with a visual cue, prior to user interaction with the displayed web page, to suggest a user gesture to effectuate selection of the actionable area.

12. The method according to claim 11 wherein the visual cue comprises an icon, text or a shrunken portion of the web page.

13. The method of claim 11, wherein the first computing platform comprises a mobile device having motion sensor for identifying a user gesture to effectuate selection of the actionable area, the motion sensor comprising an accelerometer or camera.

14. At least one non-transitory machine-accessible medium having stored thereon instructions that, when executed by a machine, cause the machine to:

analyze code for a web page to be displayed on a first computing platform to determine an actionable area on the web page, the actionable area providing additional information in response to a user selection of the actionable area, wherein the code is configured such that if the web page is displayed with a second computing platform, the second computing platform different from the first computing platform and having a positioning cursor, the additional information is displayed in response to a floatover by the positioning cursor of the second computing platform, wherein the first machine does not include a positioning cursor; and

replacing or augmenting the actionable area in a display of the web page on the first computing platform with a visual cue, prior to user interaction with the displayed web page, to suggest a user gesture to effectuate selection of the actionable area.

15. The at least one non-transitory machine-accessible medium of claim 14 wherein the visual cue comprises an icon, text or a shrunken portion of the web page.

16. The at least one non-transitory machine-accessible medium of claim 14, wherein the first computing platform comprises a mobile device having a motion sensor for identifying a user gesture to effectuate selection of the actionable area, the motion sensor comprising an accelerometer or camera.

17. An apparatus, comprising:

one or more processing devices; and

at least one non-transitory machine-accessible medium having stored thereon instructions that, when executed by the one or more processing devices, cause the apparatus to;

analyze code for a web page to be displayed on the apparatus to determine an actionable area on the web page, the actionable area providing additional information to a user in response to a user selection of the actionable area, wherein the code is configured such that if the web page is displayed with a second apparatus, the second apparatus different from the apparatus and having a positioning cursor, the additional information is displayed in response to a floatover by the positioning cursor of the second apparatus, wherein the apparatus does not include a positioning cursor; and

replace or augmenting the actionable area in a display of the web page by the apparatus with a visual cue, prior to user interaction with the displayed web page, to suggest a user gesture to effectuate selection of the actionable area.

18. The apparatus of claim 17, the at least one non-transitory machine-accessible medium further having stored thereon instructions that, when executed by the one or more processing devices, cause the apparatus to:

identify a user motion directed at the apparatus;

identify the user motion as an input gesture;

process the input gesture to determine a selection of the actionable area; and

in response to determining a selection of the actionable area, enable the associated additional information of the selected actionable area to be displayed as an overlay on the web page.

19. The apparatus of claim 17, wherein the additional information includes a selectable link.