Clustering applications using visual metadata

• US 10,242,080 B1
• Filed: 11/20/2013
• Issued: 03/26/2019
• Est. Priority Date: 11/20/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Cohen, Gabriel, Aradhye, Hrishikesh
• Primary Examiner(s): Torgrimson, Tyler J
• Application Number:
  US14/084,816
• Time in Patent Office: 1,952 Days
• Field of Search: None
• CPC Class Codes: G06F 16/26 Visual data mining; Browsing structured data
• CPC Class Codes: G06F 16/55 Clustering; Classification

Abstract

The present disclosure provides a system and method for automatic clustering and recognition of software applications using metadata. The system selects and extracts visual features from software applications which are then classified, analyzed using a cluster analysis, and then used to assign the software application to a cluster group.

Claims

1. A method, comprising:

executing, by a computing system, an application in an emulation mode;

obtaining, by the computing system and while executing the application in the emulation mode, a plurality of screen shots;

extracting, by the computing system, at least one visual feature from one or more screen shots from the plurality of screen shots using a machine learning system;

assigning, by the computing system and based on the at least one visual feature and one or more other visual features extracted from screen shots of other applications, the application to an application cluster, wherein each application included in the application cluster is more similar to each other than to applications included in other application clusters;

determining, by the computing system and based on the at least one visual feature, whether any of the plurality of screen shots includes violent content;

responsive to determining that at least one of the plurality of screen shots includes violent content, determining, by the computing system and based on respective maturity level ratings of other applications included in the application cluster, a maturity level rating for the application; and

presenting, by the computing system and for display, information about the application and information about at least one other application included in the application cluster, the information including at least the maturity level rating.

2. The method of claim 1, wherein extracting at least one visual feature from the one or more screen shots comprises applying one or more of:

image segmentation analysis, interest point-based analysis, object recognition algorithms, or histogram-based analysis to the one or more screen shots.

3. The method of claim 1, wherein assigning the application to the application cluster comprises:

assigning the application to a unique application cluster that only includes the application, wherein the unique application cluster is included in a plurality of unique application clusters; and

merging, based on one or more similarities between each of the plurality of unique application clusters, each of the plurality of unique application clusters into a hierarchical tree to form the application cluster.

4. The method of claim 1, wherein the application is included in more than one application cluster of a plurality of application clusters.

5. The method of claim 1, wherein the application is included in only one application cluster from a plurality of application clusters.

6. The method of claim 1, further comprising:

analyzing a respective set of visual features, a respective classification, and a respective cluster assignment for each of a plurality of software applications; and

storing associations between the software applications, the respective sets of visual features and the respective cluster assignments in a database.

7. A system comprising:

a microprocessor; and

a memory that stores a database, the database storing visual metadata including visual features associated with said software application,

wherein the microprocessor is configured to;

execute an application in an emulation mode;

obtain, while executing the application in the emulation mode, a plurality of screen shots;

extract at least one visual feature from one or more screen shots from the plurality of screen shots using a machine learning system;

assign, based on the at least one visual feature and one or more other visual features extracted from screen shots of other applications, the application to an application cluster, wherein each application included in the application cluster is more similar to each other than to applications included in other application clusters;

determine, based on the at least one visual features, whether any of the plurality of screen shots includes violent content;

responsive to determining that at least one of the plurality of screen shots includes violent content, determine, based on respective maturity level ratings of other applications included in the application cluster, a maturity level rating for the application; and

present, for display, information about the software application and information about the at least one other application included in the application cluster, the information including at least the maturity level rating.

8. The system of claim 7, wherein the microprocessor is configured to:

extract the at least one visual feature from the one or more screen shots by applying one or more of image segmentation analysis, interest point-based analysis, object recognition, or histogram-based analysis to the one or more screen shots.

9. The system of claim 7, wherein the microprocessor is configured to:

assign the application to a unique application cluster that only includes the application, wherein the unique application cluster is included in a plurality of unique application clusters; and

merge, based on one or more similarities between each of the plurality of unique application clusters, each of the plurality of unique application clusters into a hierarchical tree to form the application cluster.

10. The system of claim 7, wherein the application is included in more than one application cluster of a plurality of application clusters.

11. The system of claim 7, wherein the application is included in only one application cluster from a plurality of application clusters.

12. The system of claim 7, wherein the microprocessor is further configured to:

analyze a respective set of visual features, a respective classification, and a respective cluster assignment for each of a plurality of software applications; and

store associations between the software applications, the respective sets of visual features and the respective cluster assignments in the database.