Tagonomy—a system and method of semantic web tagging

• US 10,146,865 B2
• Filed: 10/03/2014
• Issued: 12/04/2018
• Est. Priority Date: 10/04/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Orions Digital Systems, Inc. (Microsoft Corporation)
• Inventors: Lahr, Nils B.
• Primary Examiner(s): RIES, LAURIE ANNE
• Application Number:
  US14/506,412
• Publication Number:
  US 20150100873A1
• Time in Patent Office: 1,523 Days
• CPC Class Codes: G06F 16/36 Creation of semantic tools, e.g. ontology or thesauri
• CPC Class Codes: G06F 40/30 Semantic analysis

Abstract

A directed graph and a semantic ontology are simultaneously employed to enable tagging of data. Tags from the directed graph contain special properties linking them to a semantic ontology such that activating a node on the graph, will provide specific actions relating to one or more ontologies. Humans or machine executable algorithms can use the directed graph as a classification system, which enables a decision making process to occur, one step at a time. Such an approach enables complex problems sets to be broken down into smaller directed graph processes. Each process can then be either automated using computer executed algorithms, manual using humans, or a combination of both. In this way an n-tiered workflow system can be developed that enables large scale asynchronous and distributed tagging.

Claims

1. A system for implementing a process to organize non-semantic visual input data so that the non-semantic visual input data is readily accessible by both a human and a computing device, by describing the non-semantic visual input data semantically, comprising:

(a) a specialized asset server computing device that accesses the non-semantic visual input data and enables semantic data to be generated and added to annotate selected elements of the non-semantic visual input data;

(b) a specialized asset viewer that presents a portion of the non-semantic visual input data to a user that is selected by the user from the non-semantic visual input data and provides a graphics user interface to enable the user to provide intelligence on a specific element selected in the portion of the non-semantic visual input data, the intelligence provided by the user being translated by the specialized asset viewer into assert data associated with the specific element, so that for a plurality of portions of the non-semantic visual input data and specific elements in the plurality of portions that are selected, corresponding assert data are created;

(c) a tagonomy editor that enables tagonomies to be created that are assigned to different parts of the process and which are employed to facilitate describing the non-semantic visual input data semantically; and

(d) a semantic engine that loads the corresponding assert data created for the specific elements selected within the plurality of portions of the non-semantic visual input data and which enables queries to be formulated and processed to retrieve desired data from the non-semantic visual input data for use in the process, until elements in the non-semantic visual input data that meet specified criteria in the queries have been identified.

2. The system of claim 1, wherein the tagonomy editor is employed to associate asserts with nodes on the tagonomy to describe the non-semantic visual input data semantically.

3. The system of claim 2, wherein the specialized asset viewer is employed to add additional asserts while elements selected from the non-semantic visual input data are being tagged, the additional asserts added by the specialized asset viewer relating to details of the elements selected from the non-semantic visual input data.

4. The system of claim 1, further comprising a computing device that is programmed with machine executable instructions to recognize an element selected in the non-semantic visual input data at one time, at other times at which the element appears in the non-semantic visual input data.

5. The system of claim 1, further comprising a computing device that is programmed with machine executable instructions to pre-identify elements in portions of the non-semantic visual input data for tagging, so that the elements that are pre-identified are reviewed for tagging with assert data using the specialized asset viewer.

6. A method for processing non-semantic visual input data to organize the non-semantic visual input data so that the non-semantic visual input data is readily accessible by both a human and a computing device, by describing the non-semantic visual input data semantically, comprising:

(a) enabling a user to select and access portions of the non-semantic visual input data that have not yet been tagged with asserts, for tagging with asserts according to defined criteria;

(b) selecting elements in the portions of the non-semantic visual input data that have been tagged with asserts, for further processing, wherein one type selected from a plurality of different types is assigned to each selected element, each type that is assigned being associated with a corresponding assert;

(c) generating additional asserts that connect the asserts associated with each type of element, to properties of said element; and

(d) for one or more types assigned to the selected elements, enabling one or more additional properties having their own tagonomy to be associated with the selected elements, wherein the asserts and properties associated with the non-semantic visual input data are in a form that enables the non-semantic visual input data to be readily accessed and queried by both a human and a computing device.

7. The method of claim 6, further comprising using a computer algorithm for detecting when and where a specific element that was selected at one point in time within the non-semantic visual input data, is also present within the non-semantic visual input data at other times.

8. The method of claim 6, further comprising the step of enabling a computing device to preselect elements in the non-semantic visual input data from among the portions of the non-semantic visual input data that have been tagged with asserts to indicate that further processing of the portions is appropriate, at least some of the elements that are preselected by the computing device being further processed to assign the type of the element.

9. The method of claim 8, wherein the computing device assigns a probable type to each element that was preselected, the elements preselected by the computing device and the probable type assigned to the elements being subject to review by a human user.

10. The method of claim 6, further comprising using a tagonomy editor to specify a tagonomy node for an element selected in the non-semantic visual input data and to associate one or more ontologies with the tagonomy node.

11. The method of claim 6, wherein once a type is assigned to an element, using a computing device to enable the user to traverse a tagonomy associated with the type assigned, to refine details of the type assigned to the element.

12. The method of claim 11, further comprising using the computing device execute an algorithm that selects an environment appropriate for the type assigned to the element, and based upon the environment that is selected, adding more details to the element based upon where the element appears at different times within the non-semantic visual input data.

13. The method of claim 6, further comprising defining how the non-semantic visual input data is processed in a workflow, so that task experts are matched with and assigned to carry out each task comprising the workflow.

14. The method of claim 13, further comprising breaking down the work process into smaller tasks, which can depend on each other, and can be split depending on a specified filter.

15. A method for tagging non-semantic visual input data to enable both computing devices and humans to readily search the non-semantic visual input data for desired content, comprising:

(a) reviewing the non-semantic visual input data to select one or more portions for tagging;

(b) tagging each portion selected with an assert that indicates whether the portion selected meets an initial criteria;

(c) for each portion of the non-semantic visual input data that has been tagged to indicate that the portion meets the initial criteria, selecting an element included in the portion and tagging the element selected with an assert to indicate a type of element that was selected;

(d) for elements that were tagged to indicate the type of element that was selected, providing additional information about the element and associating the additional information as metadata assigned to the element; and

(e) saving the asserts and additional information provided for the portions selected from the non-semantic visual input data as tagonomy data that is searchable by a computing device and by a human to find desired elements in the non-semantic visual input data.

16. The method of claim 15, further comprising using a computing device to pre-identify elements in the portions of the non-semantic visual input data that meet the initial criteria as being of a type of element.

17. The method of claim 16, further comprising enabling a human user to review and confirm the type of element pre-identified by the computing device.

18. The method of claim 15, wherein providing additional information comprises specifying details for at least some types of the elements tagged in the portions of the non-semantic visual input data meeting the initial criteria.

19. The method of claim 15, further comprising using computing device executing a computer algorithm for detecting when and where a specific element that was selected at one point in time within the non-semantic visual input data, is also present within the non-semantic visual input data at other times.

20. The method of claim 15, further comprising defining how the non-semantic visual input data is processed in a workflow, so that task experts are matched with and assigned to carry out each task comprising the workflow.