Image-based application launcher

• US 9,924,102 B2
• Filed: 03/14/2013
• Issued: 03/20/2018
• Est. Priority Date: 03/14/2013
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Gervautz, Michael, Wright, Jr., Gerald V., Lawrence Ashok Inigo, Roy
• Primary Examiner(s): Zhou, Zhihan
• Application Number:
  US13/804,643
• Publication Number:
  US 20140267770A1
• Time in Patent Office: 1,832 Days
• Field of Search: 348169, 709219
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 8/61 Installation
• CPC Class Codes: H04L 67/60 Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources admission control or resource allocation H04L47/70
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming

Abstract

Techniques for managing applications associated with a mobile device are provided. The techniques disclosed herein include techniques for obtaining an image of an object in the view of a camera associated with a mobile device, identifying the object in the image based on attributes of the object extracted from the image, and determining whether one or more applications are associated with the object. If there are one or more applications associated with the real-world object, an application associated with the object can be automatically launched on the mobile device. The association between a real-world object and an application may be identified by a visual indicator, such as an icon, symbol, or other markings on the object that indicates that the object is associated with one or more applications.

Claims

1. A method for managing applications associated with a mobile device, the method comprising:

obtaining an image of an object in view of a camera associated with the mobile device;

identifying a marking displayed on the object in the obtained image, wherein the marking on the object indicates that the object is associated with one or more applications and triggers an object identification procedure to identify the object in the image, followed by an application identification procedure to identify one or more applications associated with the object, wherein the object identification procedure and the application identification procedure are not performed for one or more objects before one or more markings are successfully identified on the one or more objects, and wherein the marking does not identify the object and does not identify the one or more applications;

performing the object identification procedure to identify the object in the image based on attributes of the object extracted from the image, wherein the object identification procedure is performed in response to successfully identifying the marking displayed on the object, and wherein the object identification procedure comprises comparing the attributes of the object extracted from the image with a plurality of attributes associated with a plurality of known objects and identifying the object as one of the plurality of known objects responsive to the attributes of the object matching more than a threshold number of attributes associated with the one known object;

conditional upon the object identification procedure successfully identifying the object as the one known object, performing the application identification procedure to identify an application based on a stored mapping between the one known object and the application; and

automatically launching the application on the mobile device.

2. The method of claim 1 wherein the plurality of attributes associated with the plurality of known objects are stored in a local database on the mobile device.

3. The method of claim 2 wherein the attributes associated with the plurality of known objects stored in the local database are organized into a search tree, and wherein the object identification procedure uses the attributes of the object extracted from the image and the search tree to identify the object.

4. The method of claim 2, further comprising:

sending the image of the object or the attributes extracted from the image to a visual search engine located on a remote server in response to the object not being successfully identified by comparing the attributes extracted from the image to the plurality of attributes associated with the plurality of objects stored in the local database; and

receiving an identification of the object from the visual search engine located on the remote server.

5. The method of claim 1 wherein the object identification procedure comprises:

sending the image of the object or the attributes extracted from the image to a visual search engine located on a remote server.

6. The method of claim 5 wherein the application identification procedure comprises receiving information identifying one or more applications associated with the object from the visual search engine.

7. The method of claim 1 wherein automatically launching the application comprises determining whether the application is installed on the mobile device.

8. The method of claim 7 wherein automatically launching the application on the mobile device comprises:

launching the application if the application is installed on the mobile device; and

displaying an interface for acquiring the application if the application is not installed on the mobile device, and launching the application based on input to the interface if the application is acquired.

9. The method of claim 1, further comprising:

identifying a plurality of candidate applications that are mapped to the one known object; and

selecting the application from the plurality of candidate applications.

10. The method of claim 9 wherein selecting the application from the plurality of candidate applications comprises selecting the application based on a user selection.

11. The method of claim 10, further comprising:

displaying a listing of the plurality of candidate applications; and

prompting a user to provide the user selection from the displayed listing.

12. The method of claim 11 wherein displaying the listing of the plurality of candidate applications comprises ordering the plurality of candidate applications on the listing according to estimated relevance.

13. The method of claim 9 wherein selecting the application from the plurality of candidate applications comprises selecting the application based on a user profile.

14. The method of claim 13, further comprising:

constructing the user profile based on at least one of user input or observed usage patterns of the mobile device.

15. The method of claim 1 wherein the application is an augmented reality application.

16. The method of claim 1, wherein the stored mapping is stored in a local database on the mobile device.

17. The method of claim 1, further comprising:

sending a request to a remote server for updates to content of a local database;

receiving updates to the content of the local database from the remote server; and

updating the local database with the updates.

18. The method of claim 1, wherein the application is also identified based on a usage profile associated with a user of the mobile device, the usage profile being indicative of types of applications preferred by the user of the mobile device.

19. The method of claim 1, wherein the application is also identified by comparing the object with multiple hierarchical levels of information.

20. An apparatus for managing applications associated with a mobile device, the apparatus comprising:

means for obtaining an image of an object in view of a camera associated with the mobile device;

means for identifying a marking displayed on the object in the obtained image, wherein the marking on the object indicates that the object is associated with one or more applications and triggers an object identification procedure to identify the object in the image, followed by an application identification procedure to identify one or more applications associated with the object, wherein the object identification procedure and the application identification procedure are not performed for one or more objects before one or more markings are successfully identified on the one or more objects, and wherein the marking does not identify the object and does not identify the one or more applications;

means for performing the object identification procedure to identify the object in the image based on attributes of the object extracted from the image, wherein the object identification procedure is performed in response to successfully identifying the marking displayed on the object, and wherein the means for performing the object identification procedure comprises means for comparing the attributes of the object extracted from the image with a plurality of attributes associated with a plurality of known objects and means for identifying the object as one of the plurality of known objects responsive to the attributes of the object matching more than a threshold number of attributes associated with the one known object;

means for, conditional upon the object identification procedure successfully identifying the object as the one known object, performing the application identification procedure to identify an application based on a stored mapping between the one known object and the application; and

means for automatically launching the application on the mobile device.

21. The apparatus of claim 20 wherein the plurality of attributes associated with the plurality of known objects are stored in a local database on the mobile device.

22. The apparatus of claim 21, further comprising:

means for sending the image of the object or the attributes extracted from the image to a visual search engine located on a remote server in response to the object not being successfully identified by comparing the attributes extracted from the image to the plurality of attributes associated with the plurality of objects stored in the local database; and

means for receiving an identification of the object from the visual search engine located on the remote server.

23. The apparatus of claim 20 wherein the means for performing the object identification procedure comprises:

means for sending the image of the object or the attributes extracted from the image to a visual search engine located on a remote server.

24. The apparatus of claim 23 wherein the means for performing the application identification procedure comprises means for receiving information identifying one or more applications associated with the object from the visual search engine.

25. The apparatus of claim 20 wherein the means for automatically launching the application comprises means for determining whether the application is installed on the mobile device.

26. The apparatus of claim 25 wherein the means for automatically launching the application on the mobile device comprises:

means for launching the application if the application is installed on the mobile device; and

means for displaying an interface for acquiring the application if the application is not installed on the mobile device, and launching the application based on input to the interface if the application is acquired.

27. The apparatus of claim 20, further comprising:

means for identifying a plurality of candidate applications that are mapped to the one known object; and

means for selecting the application from the plurality of candidate applications.

28. The apparatus of claim 27 wherein the means for selecting the application from the plurality of candidate applications comprises means for selecting the application based on a user selection.

29. The apparatus of claim 27 wherein the means for selecting the application from the plurality of candidate applications comprises means for selecting the application based on a user profile.

30. A non-transitory computer-readable medium, having stored thereon computer-readable instructions for managing applications associated with a mobile device, comprising instructions configured to cause a computer to:

obtain an image of an object in view of a camera associated with the mobile device;

identify a marking displayed on the object in the obtained image, wherein the marking on the object indicates that the object is associated with one or more applications and triggers an object identification procedure to identify the object in the image, followed by an application identification procedure to identify one or more applications associated with the object, wherein the object identification procedure and the application identification procedure are not performed for one or more objects before one or more markings are successfully identified on the one or more objects, and wherein the marking does not identify the object and does not identify the one or more applications;

perform the object identification procedure to identify the object in the image based on attributes of the object extracted from the image, wherein the object identification procedure is performed in response to successfully identifying the marking displayed on the object, and wherein the object identification procedure comprises comparing the attributes of the object extracted from the image with a plurality of attributes associated with a plurality of known objects and identifying the object as one of the plurality of known objects responsive to the attributes of the object matching more than a threshold number of attributes associated with the one known object;

conditional upon the object identification procedure successfully identifying the object as the one known object, performing the application identification procedure to identify an application based on a stored mapping between the one known object and the application; and

automatically launch the application on the mobile device.

31. An apparatus for managing applications associated with a mobile device, the apparatus comprising:

a transceiver configured to transmit and receive data wirelessly;

a non-transitory computer-readable memory;

a plurality of modules comprising processor executable code stored in the non-transitory computer-readable memory; and

a processor connected to the non-transitory computer-readable memory and configured to access the plurality of modules stored in the non-transitory computer-readable memory, wherein the plurality of modules include at least;

an image capture module configured to obtain an image of an object in view of a camera associated with the mobile device;

an image identification module configured to identify a marking displayed on the object in the obtained image and to perform an object identification procedure to identify the object in the image based on attributes of the object extracted from the image and to identify an application associated with the object based on the object identification procedure, wherein the marking on the object indicates that the object is associated with one or more applications and triggers the object identification procedure to identify the object in the image, followed by an application identification procedure to identify one or more applications associated with the object, wherein the object identification procedure and the application identification procedure are not performed for one or more objects before one or more markings are successfully identified on the one or more objects, wherein the marking does not identify the object and does not identify the one or more applications, wherein the object identification procedure is performed in response to successfully identifying the marking displayed on the object, wherein the object identification module is configured to compare the attributes of the object extracted from the image with a plurality of attributes associated with a plurality of known objects and to identify the object as one of the plurality of known objects responsive to the attributes of the object matching more than a threshold number of attributes associated with the one known object, and wherein the image identification module is further configured to, conditional upon the object identification procedure successfully identifying the object as the one known object, perform the application identification procedure to identify an application based on a stored mapping between the one known object and the application; and

an application launcher module to automatically launch the application on the mobile device.

32. The apparatus of claim 31 wherein the plurality of attributes associated with the plurality of known objects are stored in a local database on the mobile device.

33. The apparatus of claim 32, wherein the image identification module is further configured to:

send the image of the object or the attributes extracted from the image to a visual search engine located on a remote server in response to the object not being successfully identified by comparing the attributes extracted from the image to the plurality of attributes associated with the plurality of objects stored in the local database; and

receive an identification of the object from the visual search engine located on the remote server.

34. The apparatus of claim 31 wherein the image identification module is further configured to send the image of the object or the attributes extracted from the image to a visual search engine located on a remote server.

35. The apparatus of claim 34 wherein the application identification procedure comprises receiving information identifying one or more applications associated with the object from the visual search engine.

36. The apparatus of claim 31 wherein the application launcher module is further configured to determine whether the application is installed on the mobile device.

37. The apparatus of claim 36 wherein the application launcher module is further configured to launch the application if the application is installed on the mobile device, and wherein the apparatus further comprises an application acquisition module configured to display an interface for acquiring the application if the application is not installed on the mobile device, wherein the application launcher module is configured to launch the application based on input to the interface if the application is acquired.

38. The apparatus of claim 31 wherein the image identification module is further configured to identify a plurality of candidate applications that are mapped to the one known object and to select the application from the plurality of candidate applications.

39. The apparatus of claim 38 wherein the application launcher module is further configured to select the application based on a user selection.

40. The apparatus of claim 38 wherein the application launcher module is further configured to select the application based on a user profile.

41. The apparatus of claim 31 wherein the application is an augmented reality application.

42. A method for providing application content associated with an object or object type, the method comprising:

receiving information related to a request for application content from a mobile device or a visual search server, the information comprising an image of an object;

identifying a marking displayed on the object in the image, wherein the marking on the object indicates that the object is associated with one or more applications and triggers an object identification procedure to identify the object in the image, followed by an application identification procedure to identify one or more applications associated with the object, wherein the object identification procedure and the application identification procedure are not performed for one or more objects before one or more markings are successfully identified on the one or more objects, and wherein the marking does not identify the object and does not identify the one or more applications;

performing the object identification procedure to identify the object in the image based on the information related to the request for the application content, wherein the object identification procedure is performed in response to successfully identifying the marking displayed on the object, and wherein the object identification procedure comprises comparing attributes of the object extracted from the image with a plurality of attributes associated with a plurality of known objects or known object types and identifying the object as one of the plurality of known objects or known object types responsive to the attributes of the object matching more than a threshold number of attributes associated with the one known object or known object type;

conditional upon the object identification procedure successfully identifying the object as the one known object or known object type, performing the application identification procedure to identify an application based on a stored mapping between the one known object or known object type and the application; and

transmitting application content associated with the application to the mobile device or the visual search server.

43. The method of claim 42 wherein the application content comprises a list of applications that are mapped to the one known object or known object type.

44. The method of claim 42 wherein the application content comprises executable application content for the mobile device.

45. The method of claim 42 wherein the image is captured by the mobile device, and wherein the information comprises the attributes of the object extracted from the image.

46. The method of claim 42, further comprising:

receiving, from an application provider, an identifier of the one known object or attributes of the one known object that are extractable from one or more images of the one known object;

receiving, from the application provider, an application associated with the one known object;

storing the identifier or the attributes of the one known object in an application database; and

storing the application in the application database such that the application is mapped to the one known object and can be identified using the stored identifier or the attributes of the one known object.

47. An apparatus for providing application content associated with an object or object type, the apparatus comprising:

means for receiving information related to a request for application content from a mobile device or a visual search server, the information comprising an image of an object;

means for identifying a marking displayed on the object in the image, wherein the marking on the object indicates that the object is associated with one or more applications and triggers an object identification procedure to identify the object in the image, followed by an application identification procedure to identify one or more applications associated with the object, wherein the object identification procedure and the application identification procedure are not performed for one or more objects before one or more markings are successfully identified on the one or more objects, and wherein the marking does not identify the object and does not identify the one or more applications;

means for performing the object identification procedure to identify the object in the image based on the information related to the request for the application content, wherein the object identification procedure is performed in response to successfully identifying the marking displayed on the object, and wherein the means for performing the object identification procedure comprises means for comparing attributes of the object extracted from the image with a plurality of attributes associated with a plurality of known objects or known object types and means for identifying the object as one of the plurality of known objects or known object types responsive to the attributes of the object matching more than a threshold number of attributes associated with the one known object or known object type;

means for, conditional upon the object identification procedure successfully identifying the object as the one known object or known object type, performing the application identification procedure to identify an application based on a stored mapping between the one known object or known object type and the application; and

means for transmitting application content associated with the application to the mobile device or the visual search server.

48. The apparatus of claim 47 wherein the application content comprises a list of applications that are mapped to the one known object or known object type.

49. The apparatus of claim 47 wherein the application content comprises executable application content for the mobile device.

50. The apparatus of claim 47 wherein the image is captured by the mobile device, and wherein the information comprises the attributes of the object extracted from the image.

51. The apparatus of claim 47, further comprising:

means for receiving, from an application provider, an identifier of the one known object or attributes of the one known object that are extractable from one or more images of the one known object;

means for receiving, from the application provider, an application associated with the one known object;

means for storing the identifier or the attributes of the one known object in an application database; and

means for storing the application in the application database such that the application is mapped to the one known object and can be identified using the stored identifier or the attributes of the one known object.

52. A non-transitory computer-readable medium, having stored thereon computer-readable instructions for providing application content associated with an object or object type, comprising instructions configured to cause a computer to:

receive information related to a request for application content from a mobile device or a visual search server, the information comprising an image of an object;

identify a marking displayed on the object in the image, wherein the marking on the object indicates that the object is associated with one or more applications and triggers an object identification procedure to identify the object in the image, followed by an application identification procedure to identify one or more applications associated with the object, wherein the object identification procedure and the application identification procedure are not performed for one or more objects before one or more markings are successfully identified on the one or more objects, and wherein the marking does not identify the object and does not identify the one or more applications;

perform the object identification procedure to identify the object in the image based on the information related to the request for the application content, wherein the object identification procedure is performed in response to successfully identifying the marking displayed on the object, and wherein the object identification procedure comprises comparing attributes of the object extracted from the image with a plurality of attributes associated with a plurality of known objects or known object types and identifying the object as one of the plurality of known objects or known object types responsive to the attributes of the object matching more than a threshold number of attributes associated with the one known object or known object type;

conditional upon the object identification procedure successfully identifying the object as the one known object or known object type, perform the application identification procedure to identify an application based on a stored mapping between the one known object or known object type and the application; and

transmit application content associated with the application to the mobile device or the visual search server.

53. The non-transitory computer-readable medium of claim 52 wherein the application content comprises a list of applications that are mapped to the one known object or known object type.

54. The non-transitory computer-readable medium of claim 52 wherein the application content comprises executable application content for the mobile device.

55. The non-transitory computer-readable medium of claim 52 wherein the image is captured by the mobile device, and wherein the information comprises the attributes of the object extracted from the image.

56. The non-transitory computer-readable medium of claim 52, further comprising instructions configured to cause the computer to:

receive, from an application provider, an identifier of the one known object or attributes of the one known object that are extractable from one or more images of the one known object;

receive, from the application provider, an application associated with the one known object;

store the identifier or the attributes of the one known object in an application database, and

store the application in the application database such that the application is mapped to the one known object and can be identified using the stored identifier or the attributes of the one known object.

57. An apparatus for providing application content associated with an object or object type, the apparatus comprising:

a transceiver configured to receive information related to a request for application content from a mobile device or a visual search server, the information comprising an image of an object;

an application database comprising application content associated with a plurality of applications from a plurality of application providers; and

a processor configured to;

identify a marking displayed on the object in the image, wherein the marking on the object indicates that the object is associated with one or more applications and triggers an object identification procedure to identify the object in the image, followed by an application identification procedure to identify one or more applications associated with the object, wherein the object identification procedure and the application identification procedure are not performed for one or more objects before one or more markings are successfully identified on the one or more objects, and wherein the marking does not identify the object and does not identify the one or more applications;

perform the object identification procedure to identify the object in the image based on the information related to the request for the application content, the object identification procedure is performed in response to successfully identifying the marking displayed on the object, and wherein the object identification procedure comprises comparing attributes of the object extracted from the image with a plurality of attributes associated with a plurality of known objects or known object types and identifying the object as one of the plurality of known objects or known object types responsive to the attributes of the object matching more than a threshold number of attributes associated with the one known object or known object type;

conditional upon the object identification procedure successfully identifying the object as the one known object or known object type, perform the application identification procedure to identify an application based on a stored mapping between the one known object or known object type and the application;

wherein the transceiver is further configured to transmit application content associated with the application to the mobile device or the visual search server.

58. The apparatus of claim 57 wherein the application content comprises a list of applications that are mapped to the one known object or known object type.

59. The apparatus of claim 57 wherein the application content comprises executable application content for the mobile device.

60. The apparatus of claim 57 wherein the image is captured by the mobile device, and wherein the information comprises the attributes of the object extracted from the image.

61. The apparatus of claim 57 wherein the processor is further configured to:

receive, from an application provider, an identifier of the one known object or attributes of the one known object that are extractable from one or more images of the one known object;

receive, from the application provider, an application associated with the one known object;

store the identifier or the attributes of the one known object in an application database; and

store the application in the application database such that the application is mapped to the one known object and can be identified using the stored identifier or the attributes of the one known object.