Computerized identification of app search functionality for search engine access

• US 10,127,320 B2
• Filed: 12/29/2015
• Issued: 11/13/2018
• Est. Priority Date: 12/29/2015
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Anand, Saswat, Desineni, Kalyan, Sankaranarasimhan, Manikandan, Tsechansky, Danny
• Primary Examiner(s): Nguyen, Kim T
• Application Number:
  US14/982,192
• Publication Number:
  US 20170185677A1
• Time in Patent Office: 1,050 Days
• Field of Search: 707709
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/84 Mapping; Conversion
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 16/9554 by using bar codes

Abstract

A search system includes a device controller that provisions a device to execute a copy of a selected mobile application. A crawler extracts content and metadata from states of the selected mobile application. A search input state classifier identifies search input states within the states based on a first set of heuristics, including recognition of user-visible search indicia and recognition of metadata that correlates with search functionality. A parameter identifier identifies, for each of the search input states, necessary input parameters. A search function data store stores a record for each identified search function. Each record includes a path to reach a corresponding search input state, an indication of required input parameters, and a mapping of the input parameters to user interface widgets. A query processing system uses the stored search functions to scrape content from the selected mobile application in response to a query.

Claims

1. A search system comprising:

a device controller configured to provision a device to execute a copy of a selected mobile application;

a crawler configured to extract content and metadata from a plurality of states of the copy of the selected mobile application using the device;

a search input state classifier configured to identify search input states within the plurality of states based on a first set of heuristics, wherein the first set of heuristics includes recognition of user-visible search indicia and recognition of metadata that correlates with search functionality;

a parameter identifier configured to identify, for each of the search input states, necessary input parameters;

a search function data store configured to store a record for each identified search function in the selected mobile application, wherein each record includes;

a path to reach a search input state of the corresponding search function,

an indication of input parameters required to be supplied to the corresponding search function, and

a mapping of the input parameters to user interface widgets of the search input state; and

a query processing system configured to, in response to a query, (i) selectively perform the search function in the selected mobile application using parameters specified by the query, and (ii) scrape content from a resulting search results state.

2. The search system of claim 1 further comprising a search results state classifier configured to identify search results states within the plurality of states based on a second set of heuristics, wherein the second set of heuristics includes identification of a list of repeated widget groups.

3. The search system of claim 2 wherein the search input state classifier is configured to identify a precursor state to the identified search results states and store the precursor state as a search input state.

4. The search system of claim 2 further comprising:

an operator interface configured to allow an operator to designate a state of the selected mobile application as a search results state,

wherein the search input state classifier is configured to identify a precursor state to the operator-designated search results state and store the precursor state as a search input state.

5. The search system of claim 1 wherein:

the user-visible search indicia includes a textual hint;

the user-visible search indicia includes a search-specific keyboard;

the metadata that correlates with search functionality includes a search-specific widget type; and

the metadata that correlates with search functionality includes an accessibility tag.

6. The search system of claim 1 wherein the crawler is configured to (i) randomly select user interface elements within the selected mobile application for actuation and (ii) extract content and metadata from each state encountered as a result of the actuation of the randomly-selected user interface elements.

7. The search system of claim 1 wherein the parameter identifier is configured to gather autocomplete data from text input fields by supplying letters and numbers and scraping autocomplete suggestions.

8. The search system of claim 1 wherein the device comprises one of:

an emulator configured to execute an operating system on emulated hardware, wherein the operating system is selected based on an operating system the selected mobile application was compiled for; and

a simulator configured to simulate interaction of the operating system with the selected mobile application.

9. The search system of claim 1 further comprising a digital distribution platform interface configured to download the selected mobile application from a digital distribution platform.

10. The search system of claim 1 wherein the query processing system comprises:

a user interest finder configured to receive a query from a user device and identify a set of entity types to describe segments of the query;

a search function matcher configured to identify at least one search function from the search function data store, wherein;

each of the at least one search function has a set of required input entity types that is a subset of the identified set of entity types, and

a first search function of the at least one search function is associated with a first search input state of a first application;

a live scrape system configured to;

execute the first application within a device;

navigate to the first search input state;

supply parameters to the first search input state based on the query;

perform a search; and

scrape results from a resulting search results state; and

a results module configured to assemble results from the live scrape system for each of the at least one search function and transmit the assembled results to the user device.

11. A method of operating a search system, the method comprising:

provisioning a device to execute a copy of a selected mobile application;

extracting content and metadata from a plurality of states of the copy of the selected mobile application executing on the device;

identifying search input states within the plurality of states based on a first set of heuristics, wherein the first set of heuristics includes recognition of user-visible search indicia and recognition of metadata that correlates with search functionality;

identifying, for each of the search input states, necessary input parameters;

storing a record for each identified search function in the selected mobile application, wherein each record includes;

a path to reach a search input state of the corresponding search function,

an indication of input parameters required to be supplied to the corresponding search function, and

a mapping of the input parameters to user interface widgets of the search input state; and

in response to a query, (i) selectively performing the search function in the selected mobile application using parameters specified by the query, and (ii) scraping content from a resulting search results state.

12. The method of claim 11 further comprising identifying search results states within the plurality of states based on a second set of heuristics, wherein the second set of heuristics includes identification of a list of repeated widget groups.

13. The method of claim 12 further comprising:

identifying a precursor state to the identified search results states; and

storing the precursor state as a search input state.

14. The method of claim 12 further comprising:

allowing an operator to designate a state of the selected mobile application as a search results state;

identifying a precursor state to the operator-designated search results state; and

storing the precursor state as a search input state.

15. The method of claim 11 wherein:

the user-visible search indicia includes a textual hint;

the user-visible search indicia includes a search-specific keyboard;

the metadata that correlates with search functionality includes a search-specific widget type; and

the metadata that correlates with search functionality includes an accessibility tag.

16. The method of claim 11 further comprising:

randomly selecting user interface elements within the selected mobile application for actuation; and

extracting content and metadata from each state encountered as a result of the actuation of the randomly-selected user interface elements.

17. The method of claim 11 further comprising gathering autocomplete data from text input fields by supplying letters and numbers and scraping autocomplete suggestions.

18. The method of claim 11 wherein the device comprises one of:

an emulator configured to execute an operating system on emulated hardware, wherein the operating system is selected based on an operating system the selected mobile application was compiled for; and

a simulator configured to simulate interaction of the operating system with the selected mobile application.

19. The method of claim 11 further comprising downloading the selected mobile application from a digital distribution platform.

20. The method of claim 11 further comprising:

receiving a query from a user device;

identifying a set of entity types to describe segments of the query;

identifying at least one search function from the stored records, wherein;

each of the at least one search function has a set of required input entity types that is a subset of the identified set of entity types, and

a first search function of the at least one search function is associated with a first search input state of a first application;

executing the first application within a device;

navigating to the first search input state within the device;

supplying parameters to the first search input state based on the query;

performing a search within the device;

scraping results from a resulting search results state within the device;

assembling the scraped results for each of the at least one search function; and

transmitting the assembled results to the user device.