Operator-guided application crawling architecture

• US 10,146,785 B2
• Filed: 09/28/2015
• Issued: 12/04/2018
• Est. Priority Date: 05/13/2015
• Status: Expired due to Fees

• Current Assignee: Samsung Electronics Co. Ltd. (Samsung Group)
• Original Assignee: Samsung Electronics Co. Ltd. (Samsung Group)
• Inventors: Desineni, Kalyan, Sankaranarasimhan, Manikandan, Mohan, Sudhir
• Primary Examiner(s): Dwivedi, Mahesh H
• Application Number:
  US14/868,294
• Publication Number:
  US 20160335349A1
• Time in Patent Office: 1,163 Days
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 16/24556 Aggregation; Duplicate elimination
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: G06F 16/9024 Graphs; Linked lists G06F16/9027 takes precedence
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 16/954 Navigation, e.g. using categorised browsing
• CPC Class Codes: G06F 16/9558 Details of hyperlinks; Management of linked annotations
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]

Abstract

A system for automated acquisition of content from an application includes a guide tracker module, a link extraction controller, and a scraper. The guide tracker module monitors interaction of an operator with an executing instance of the application and records a set of guides. Each guide includes a recorded sequence of user interface interactions concluding at a respective ultimate state of the application. The link extraction controller, for each guide of the set of guides, selectively identifies additional states of the application that correspond to the respective ultimate state and adds the additional states corresponding to the respective ultimate state and the respective ultimate state to a state list. The additional states and the respective ultimate state are all directly reachable from a common penultimate state of the application. Each entry in the state list designates a state and a path of user interface interactions to arrive at the state.

Claims

1. A system for automated acquisition of content from an application, the system comprising:

a guide tracker module configured to monitor interaction of an operator with an executing instance of the application and record a set of guides, wherein each guide in the set of guides includes a recorded sequence of user interface interactions concluding at a respective ultimate state of the application;

a link extraction controller configured to, for each guide of the set of guides;

selectively identify additional states of the application that correspond to the respective ultimate state and

add the additional states corresponding to the respective ultimate state and the respective ultimate state to a state list,

wherein the additional states and the respective ultimate state are all directly reachable from a common penultimate state of the application,

wherein the common penultimate state of the application is immediately prior to the respective ultimate state in the guide, and

wherein each entry in the state list designates (i) a state and (ii) a path of user interface interactions to arrive at the state; and

a scraper module configured to, within an executing instance of the application, extract text and metadata from the states designated by each of the entries in the state list, wherein information based on the extracted text and metadata is stored in a data store.

2. The system of claim 1 wherein, for each guide in the set of guides, the sequence of user interface interactions begins at a home state of the application.

3. The system of claim 2 wherein the guide tracker module is configured to create a new guide in the set of guides in response to the operator returning to the home state of the application.

4. The system of claim 1 wherein the application is executing within an emulator.

5. The system of claim 4 wherein:

the link extraction controller is configured to execute the application within a second emulator; and

the scraper module is configured to execute the application within a third emulator.

6. The system of claim 1 wherein each path is one of (i) a sequence of user interface interactions, (ii) an application programming interface call, or (iii) an application programming interface call followed by one or more user interface interactions.

7. The system of claim 6 wherein the link extraction controller is configured to (i) identify that an application programming interface call is available to reach a first state directly, (ii) identify parameters for the application programming interface call that specify the first state, and (iii) incorporate the application programming interface call with the identified parameters into the path to the first state.

8. The system of claim 1 wherein the link extraction controller is configured to, for each guide of the set of guides, and for each intermediate state encountered along the guide:

selectively identify additional states of the application that correspond to the intermediate state and

add the additional states corresponding to the intermediate state and the intermediate state to the state list,

wherein the additional states corresponding to the intermediate state and the intermediate state are all directly reachable from a common prior state of the application, and

wherein the common prior state of the application is immediately prior to the intermediate state in the guide.

9. The system of claim 1 further comprising:

a target state tracker module configured to allow the operator to, while recording a guide of the set of guides, designate one or more states encountered along the guide as target states,

wherein the link extraction controller is configured to, for each designated target state;

selectively identify additional states of the application that correspond to the target state and

add the additional states corresponding to the target state and the target state to the state list,

wherein the additional states corresponding to the target state and the target state are all directly reachable from a common prior state of the application, and

wherein the common prior state of the application is immediately prior to the target state in the respective guide.

10. The system of claim 1 wherein the scraper module is configured to, for each entry in the state list, (i) reach the designated state by replaying the corresponding path to the executing application and (ii) extract the text and metadata from the designated state.

11. A search system comprising:

the system of claim 1;

the data store;

a set generation module configured to, in response to a query from a user device, select records from the data store to form a consideration set of records;

a set processing module configured to assign a score to each record of the consideration set of records; and

a results generation module configured to respond to the user device with a subset of the consideration set of records, wherein the subset is selected based on the assigned scores.

12. A method for automated acquisition of content from an application, the method comprising:

monitoring interaction of an operator with an executing instance of the application;

recording a set of guides according to the monitoring, wherein each guide in the set of guides includes a recorded sequence of user interface interactions concluding at a respective ultimate state of the application;

for each guide of the set of guides;

selectively identifying additional states of the application that correspond to the respective ultimate state and

adding the additional states corresponding to the respective ultimate state and the respective ultimate state to a state list,

wherein the additional states and the respective ultimate state are all directly reachable from a common penultimate state of the application,

wherein the common penultimate state of the application is immediately prior to the respective ultimate state in the guide, and

wherein each entry in the state list designates (i) a state and (ii) a path of user interface interactions to arrive at the state; and

within an executing instance of the application, extracting text and metadata from the states designated by each of the entries in the state list, wherein information based on the extracted text and metadata is stored in a data store.

13. The method of claim 12 wherein, for each guide in the set of guides, the sequence of user interface interactions begins at a home state of the application.

14. The method of claim 13 further comprising creating a new guide in the set of guides in response to the operator returning to the home state of the application.

15. The method of claim 12 further comprising executing the application within an emulator.

16. The method of claim 12 further comprising:

executing the application within a first emulator for use in monitoring the interaction of the operator;

executing the application within a second emulator for use in the identifying additional states; and

executing the application within a third emulator for use in the extracting text and metadata.

17. The method of claim 12 wherein each path is one of (i) a sequence of user interface interactions, (ii) an application programming interface call, or (iii) an application programming interface call followed by one or more user interface interactions.

18. The method of claim 17 further comprising:

identifying that an application programming interface call is available to reach a first state directly;

identifying parameters for the application programming interface call that specify the first state; and

incorporating the application programming interface call with the identified parameters into the path to the first state.

19. The method of claim 12 further comprising, for each guide of the set of guides, and for each intermediate state encountered along the guide:

selectively identifying additional states of the application that correspond to the intermediate state and

adding the additional states corresponding to the intermediate state and the intermediate state to the state list,

wherein the additional states corresponding to the intermediate state and the intermediate state are all directly reachable from a common prior state of the application, and

wherein the common prior state of the application is immediately prior to the intermediate state in the guide.

20. The method of claim 12 further comprising:

providing a user interface to the operator to allow the operator to, while recording a guide of the set of guides, designate one or more states encountered along the guide as target states; and

for each designated target state;

selectively identifying additional states of the application that correspond to the target state and

adding the additional states corresponding to the target state and the target state to the state list,

wherein the additional states corresponding to the target state and the target state are all directly reachable from a common prior state of the application, and

wherein the common prior state of the application is immediately prior to the target state in the respective guide.

21. The method of claim 12 wherein the extracting and metadata includes, for each entry in the state list, (i) reaching the designated state by replaying the corresponding path to the executing application and (ii) extracting the text and metadata from the designated state.

22. A method of operating a mobile application search system, the method comprising:

the method of claim 12;

in response to receiving a query from a user device, selecting records from the data store to form a consideration set of records;

assigning a score to each record of the consideration set of records; and

responding to the user device with a subset of the consideration set of records, wherein the subset is selected based on the assigned scores, and wherein the subset identifies application states of applications that are relevant to the query.

23. A non-transitory computer-readable medium storing processor-executable instructions configured to perform the method of claim 12.