Computerized identification of app search functionality for search engine access

• US 9,990,428 B2
• Filed: 08/24/2016
• Issued: 06/05/2018
• Est. Priority Date: 12/29/2015
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Anand, Saswat, Desineni, Kalyan, Sankaranarasimhan, Manikandan, Tsechansky, Danny
• Primary Examiner(s): Gortayo, Dangelino N
• Application Number:
  US15/245,261
• Publication Number:
  US 20170185679A1
• Time in Patent Office: 650 Days
• Field of Search: 707706, 707733, 707734, 707726, 707728, 707731, 707752, 707755, 707766, 707912, 707632, 707709
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/84 Mapping; Conversion
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 16/9554 by using bar codes

Abstract

A search system includes a device controller that provisions a device to execute a copy of a selected mobile application. A crawler extracts content and metadata from states of the selected mobile application. A search input state classifier identifies search input states within the states based on a first set of heuristics, including recognition of user-visible search indicia and recognition of metadata that correlates with search functionality. A parameter identifier identifies, for each of the search input states, necessary input parameters. A search function data store stores a record for each identified search function. Each record includes a path to reach a corresponding search input state, an indication of required input parameters, and a mapping of the input parameters to user interface widgets. A query processing system uses the stored search functions to scrape content from the selected mobile application in response to a query.

Claims

1. A search system comprising:

a device controller configured to provision a device to execute a copy of a selected mobile application selected from among a plurality of mobile applications;

a crawler configured to extract content and metadata from a plurality of states of the copy of the selected mobile application using the device;

a search input state classifier configured to identify search input states within the plurality of states of the selected mobile application, wherein identifying search input states is performed based on a first set of heuristics, wherein the first set of heuristics includes recognition of user-visible search indicia from the extracted content and metadata of the selected mobile application and recognition of items of the extracted metadata of the selected mobile application that correlate with search functionality;

a parameter identifier configured to identify, for each of the search input states of the selected mobile application, necessary input parameters for a search function corresponding to the search input state;

a search function data store configured to store a record for each identified search function in the selected mobile application, wherein each record includes;

a navigation path within the selected mobile application to reach a search input state of the corresponding search function,

an indication of input parameters required to be supplied to the corresponding search function, and

a mapping of the input parameters to user interface widgets of the search input state; and

a query processing system configured to, in response to a query, (i) identify a first search function record from the search function data store based on the query, (ii) within a first mobile application corresponding to the first search function record, navigate to the search input state of the first search function record according to the navigation path of the first search function record;

(iii) based on parameters specified by the query, selectively perform the search function of the first search function record in the first mobile application, (iv) scrape content from a resulting search results state of the first mobile application, and (v) present the scraped content to a user.

2. The search system of claim 1 further comprising a search results state classifier configured to identify search results states within the plurality of states based on a second set of heuristics, wherein the second set of heuristics includes identification of a list of repeated widget groups.

3. The search system of claim 2 wherein the search input state classifier is configured to identify a precursor state to the identified search results states and store the precursor state as a search input state.

4. The search system of claim 2 further comprising:

an operator interface configured to allow an operator to designate a state of the selected mobile application as a search results state,

wherein the search input state classifier is configured to identify a precursor state to the operator-designated search results state and store the precursor state as a search input state.

5. The search system of claim 1 wherein:

the user-visible search indicia includes a textual hint;

the user-visible search indicia includes a search-specific keyboard;

the metadata that correlates with search functionality includes a search-specific widget type; and

the metadata that correlates with search functionality includes an accessibility tag.

6. The search system of claim 1 wherein the crawler is configured to (i) randomly select user interface elements within the selected mobile application for actuation and (ii) extract content and metadata from each state encountered as a result of the actuation of the randomly-selected user interface elements.

7. The search system of claim 1 wherein the parameter identifier is configured to gather autocomplete data from text input fields by supplying letters and numbers and scraping autocomplete suggestions.

8. The search system of claim 1 wherein the device comprises one of:

an emulator configured to execute an operating system on emulated hardware, wherein the operating system is selected based on an operating system the selected mobile application was compiled for; and

a simulator configured to simulate interaction of the operating system with the selected mobile application.

9. The search system of claim 1 further comprising a digital distribution platform interface configured to download the selected mobile application from a digital distribution platform.

10. The search system of claim 1 wherein the query processing system comprises:

a user interest finder configured to receive a query from a user device and identify a set of entity types to describe segments of the query;

a search function matcher configured to identify at least one search function record from the search function data store, wherein;

the at least one search function record includes the first search function record, and

each of the at least one search function record has a set of required input entity types that is a subset of the identified set of entity types;

a live scrape system configured to;

execute the first mobile application within a device;

within the first mobile application, navigate to the search input state of the first search function record according to the navigation path of the first search function record;

supply parameters to the search input state based on the parameters specified by the query;

perform the search function in the first mobile application; and

scrape content from a resulting search results state; and

a results module configured to assemble the content from the live scrape system for each of the at least one search function and transmit the assembled content to the user device.

11. A method of operating a search system, the method comprising:

provisioning a device to execute a copy of a selected mobile application selected from among a plurality of mobile applications;

extracting content and metadata from a plurality of states of the copy of the selected mobile application executing on the device;

identifying search input states within the plurality of states of the selected mobile application, wherein identifying the search input states is performed based on a first set of heuristics, wherein the first set of heuristics includes recognition of user-visible search indicia from the extracted content and metadata of the selected mobile application and recognition of items of the extracted metadata of the selected mobile application that correlate with search functionality;

identifying, for each of the search input states of the selected mobile application, necessary input parameters for a search function corresponding to the search input state;

storing a record for each identified search function in the selected mobile application, wherein each record includes;

a navigation path within the selected mobile application to reach a search input state of the corresponding search function,

an indication of input parameters required to be supplied to the corresponding search function, and

a mapping of the input parameters to user interface widgets of the search input state; and

in response to a query, (i) identifying a first search function record from based on the query, (ii) within a first mobile application corresponding to the first search function record, navigating to the search input state of the first search function record according to the navigation path of the first search function record, (iii) based on parameters specified by the query, selectively performing the search function of the first search function record in the first mobile application, (iv) scraping content from a resulting search results state, and (v) presenting the scraped content to a user.

12. The method of claim 11 further comprising identifying search results states within the plurality of states based on a second set of heuristics, wherein the second set of heuristics includes identification of a list of repeated widget groups.

13. The method of claim 12 further comprising:

identifying a precursor state to the identified search results states; and

storing the precursor state as a search input state.

14. The method of claim 12 further comprising:

allowing an operator to designate a state of the selected mobile application as a search results state;

identifying a precursor state to the operator-designated search results state; and

storing the precursor state as a search input state.

15. The method of claim 11 wherein:

the user-visible search indicia includes a textual hint;

the user-visible search indicia includes a search-specific keyboard;

the metadata that correlates with search functionality includes a search-specific widget type; and

the metadata that correlates with search functionality includes an accessibility tag.

16. The method of claim 11 further comprising:

randomly selecting user interface elements within the selected mobile application for actuation; and

extracting content and metadata from each state encountered as a result of the actuation of the randomly-selected user interface elements.

17. The method of claim 11 further comprising gathering autocomplete data from text input fields by supplying letters and numbers and scraping autocomplete suggestions.

18. The method of claim 11 wherein the device comprises one of:

an emulator configured to execute an operating system on emulated hardware, wherein the operating system is selected based on an operating system the selected mobile application was compiled for; and

a simulator configured to simulate interaction of the operating system with the selected mobile application.

19. The method of claim 11 further comprising downloading the selected mobile application from a digital distribution platform.

20. The method of claim 11 further comprising:

receiving a query from a user device;

identifying a set of entity types to describe segments of the query;

identifying at least one search function record from the stored records, wherein;

the at least one search function record includes the first search function record, and

each of the at least one search function record has a set of required input entity types that is a subset of the identified set of entity types;

executing the first mobile application within a device;

within the first mobile application on the device, navigating to the search input state of the first search function record according to the navigation path of the first search function record;

supplying parameters to the search input state based on the parameters specified by the query;

performing the search function within the first mobile application on the device;

scraping content from a resulting search results state within the first mobile application on the device;

assembling the scraped content for each of the at least one search function; and

transmitting the assembled content to the user device.