Camera guided web browsing based on passive object detection

• US 9,258,462 B2
• Filed: 04/18/2012
• Issued: 02/09/2016
• Est. Priority Date: 04/18/2012
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Sweet, Charles Wheeler III, Xue, Qi
• Primary Examiner(s): Patel, Jay
• Assistant Examiner(s): SUH, JOSEPH JINWOO
• Application Number:
  US13/450,312
• Publication Number:
  US 20130278777A1
• Time in Patent Office: 1,392 Days
• Field of Search: 348/169
• US Class Current: 1/1
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/9577 Optimising the visualization of content, e.g. distillation of HTML documents
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof

Abstract

Systems and methods for performing camera-guided browsing, such as web browsing, are described herein. A method for operating a camera-guided web browser as provided herein includes displaying a web page on a display associated with a portable device; passively detecting a first object within a field of view of a camera associated with the portable device; and altering at least part of the web page with first content associated with the first object in response to passively detecting the first object within the field of view of the camera.

Claims

1. A method for operating a camera-guided web browser, the method comprising:

displaying a web page on a display associated with a portable device, wherein the web page comprises content accessed over a network from a remote server and is defined by a markup language;

receiving, from the web page, an indication that the web page supports camera-guided browsing;

activating a camera associated with the portable device in response to receiving the indication;

passively using an object recognition engine to detect a plurality of objects within a field of view of the camera associated with the portable device;

weighting each of the plurality of objects based on the distance of each of the plurality of the objects from the camera to identify a first object, wherein weighting each of the plurality of objects comprises determining the relative size of each of the plurality of objects and identifying a largest object as the first object; and

altering at least part of the web page with a first content associated with the first object.

2. The method of claim 1 wherein the first content comprises content distinct from imagery captured by the camera.

3. The method of claim 1 wherein the displaying comprises displaying the web page using a web browser application.

4. The method of claim 3 wherein the object recognition engine intermittently samples the field of view.

5. The method of claim 1 wherein the object recognition engine is implemented as a computer vision plug-in of the web browser application.

6. The method of claim 1 wherein the object recognition engine is implemented as an embedded component of the web browser application.

7. The method of claim 1 wherein the altering comprises altering one or more frames associated with the web page using the first content.

8. The method of claim 1 wherein:

the displaying comprises displaying at least one static content section and at least one dynamic content section of the web page;

weighting each of the plurality of objects comprises determining the relative distance of each of the plurality of objects, wherein an object closest to the camera is identified as the first object; and

the altering comprises obtaining the first content, the first content comprising dynamic web page content associated with the first object, and displaying the first content within the at least one dynamic content section of the web page.

9. The method of claim 1 wherein receiving the indication that the web page supports camera-guided browsing includes providing an alert.

10. The method of claim 1 further comprising:

weighting each of the plurality of object to identify the first object and a second object; and

altering at least part of the web page with second content associated with the second object.

11. The method of claim 10 wherein the altering comprises replacing the first content with the second content.

12. The method of claim 10 wherein the altering comprises displaying the second content in addition to the first content.

13. The method of claim 1 wherein weighting each of the plurality of objects comprises determining a center object, wherein the center object is closest to a center of the field of view of the camera associated with the portable device, and the center objects is identified as the first object.

14. A system for operating a web browser, the system comprising:

a portable communications device having disposed therein a display screen and a camera, wherein the web browser is associated with the portable communications device and configured to display a web page on the display screen of the portable communications device and to receive an indication from the web page that the web page supports camera-guided browsing, wherein the web page comprises content accessed over a network from a remote server and is defined by a markup language;

an object recognition engine associated with the web browser, communicatively coupled to the camera, and configured to;

intermittently sample images within a field of view of the camera based on receiving an indication that the web page supports camera-guided browsing;

passively detect a plurality of objects within the images; and

weight each of the plurality of objects based on the distance of each of the plurality of the objects from the camera to identify a first object, wherein the object recognition engine is configured to determine the relative size of each of the plurality of objects and to identify a largest object as the first object;

wherein the web browser is further configured to alter at least part of the web page as displayed on the display screen with first content associated with the first object.

15. The system of claim 14 wherein the first content comprises content distinct from imagery captured by the camera.

16. The system of claim 14 wherein the object recognition engine comprises a plug-in of the web browser.

17. The system of claim 14 wherein the object recognition engine is an embedded component of the web browser.

18. The system of claim 14 wherein the web browser is configured to alter one or more frames associated with the web page with the first content.

19. The system of claim 14 wherein:

the web browser is configured to display at least one static content section and at least one dynamic content section of the web page on the display screen;

the object recognition engine is configured to instruct the camera to intermittently sample images within the field of view of the camera to passively and automatically capture at least one image and to detect the plurality of objects within the field of view and identify the first object within the at least one image; and

the web browser is further configured, in response to the object recognition engine, to obtain the first content, the first content comprising dynamic web page content associated with the first object, and to display the first content on the display screen within the at least one dynamic content section of the web page.

20. The system of claim 14 wherein:

the object recognition engine is further configured to weight each of the plurality of objects to identify the first object and a second object; and

the web browser is further configured to alter at least part of the web page with second content associated with the second object.

21. The system of claim 20 wherein the web browser is further configured to replace the first content with the second content.

22. The system of claim 20 wherein the web browser is further configured to display the second content in addition to the first content.

23. The system of claim 14 wherein the web browser is further configured to obtain an indication that the web page supports camera-guided browsing and to activate the camera in response to the indication.

24. The system of claim 14 wherein the portable communications device comprises a smartphone.

25. A system for operating a camera-guided web browser, the system comprising:

a portable device comprising a camera and a display screen;

static display means, communicatively coupled to the portable device, for displaying a web page at the display screen of the portable device, wherein the web page comprises content accessed over a network from a remote server and is defined by a markup language;

control means, communicatively coupled to the camera, for obtaining an indication, from the web page, that the web page supports camera-guided browsing and activating the camera in response to the indication;

computer vision means, communicatively coupled to the camera, configured to intermittently sample images within a field of view of the camera for passively detecting a plurality of objects within the images, and to weight each of the plurality of objects based on the distance of each of the plurality of the objects from the camera to identify a first object, wherein the computer vision means comprises means for determining the relative size of each of the plurality of objects and means for identifying a largest object as the first object; and

dynamic display means, communicatively coupled to the static display means and the computer vision means, for altering at least part of the web page with first content associated with the first object.

26. The system of claim 25 wherein the first content comprises content distinct from imagery captured by the camera.

27. The system of claim 25 wherein the dynamic display means comprises means for altering one or more frames associated with the web page using the first content.

28. The system of claim 25 wherein:

the static display means comprises means for partitioning the web page into one static content section and at least one dynamic content section;

the computer vision means comprises means for weighting each of the plurality based on a relative distance of each of the plurality of objects, wherein the closest object is identified as the first object; and

the dynamic display means comprises means for obtaining the first content, the first content comprising dynamic web page content associated with the first object, and means for displaying the first content within the at least one dynamic content section of the web page.

29. The system of claim 25 wherein:

the computer vision means comprises means for weighting each of the plurality of objects to identify the first object and a second object; and

the dynamic display means comprises means for altering at least part of the web page with second content associated with the second object.

30. A computer program product residing on a processor-executable non-transitory computer-readable storage medium, the computer program product comprising processor-executable instructions configured to cause a processor to:

display a web page on a display screen associated with a portable device, wherein the web page comprises content accessed over a network from a remote server and is defined by a markup language;

receive, from the web page, an indication that the web page supports camera-guided browsing;

activate a camera associated with the portable device in response to receiving the indication;

passively execute an object recognition engine to detect a plurality of objects within a field of view of the camera associated with the portable device;

weight each of the plurality of objects based on the distance of each of the plurality of the objects from the camera to identify a first object, wherein the instructions configured to cause the processor to weight each of the plurality of objects comprise instructions configured to cause the processor to determine the relative size of each of the plurality of objects and to identify a largest object as the first object; and

alter at least part of the web page with first content associated with the first object.

31. The computer program product of claim 30 wherein the first content comprises content distinct from imagery captured by the camera.

32. The computer program product of claim 30 wherein the instructions configured to cause the processor to alter at least part of the web page comprise instructions configured to cause the processor to alter content within one or more frames associated with the web page.

33. The computer program product of claim 30 wherein:

the computer program product further comprises instructions configured to cause the processor to display at least one static content section and at least one dynamic content section of the web page;

the instructions configured to cause the processor to passively execute the object recognition engine to detect the plurality of objects comprise instructions configured to cause the processor to;

passively and automatically capture at least one image using the camera, and

identify the plurality of objects within the at least one image; and

weight each of the plurality of objects to identify the first object; and

the instructions configured to cause the processor to alter at least part of the web page comprise instructions configured to cause the processor to;

obtain the first content, the first content comprising dynamic web page content associated with the first object, and

display the first content within the at least one dynamic content section of the web page.

34. The computer program product of claim 30 further comprising processor-executable instructions configured to cause the processor to display an alert in response to receiving the indication that the web page supports camera-guided browsing.

35. The computer program product of claim 30 further comprising processor-executable instructions configured to cause the processor to:

weight each of the plurality of objects to identify the first object and a second object; and

alter at least part of the web page with second content associated with the second object.