Semantic annotations for virtual objects

• US 7,836,437 B2
• Filed: 02/10/2006
• Issued: 11/16/2010
• Est. Priority Date: 02/10/2006
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Kacmarcik, Gary J.
• Primary Examiner(s): DENG, ANNA CHEN
• Application Number:
  US11/352,000
• Publication Number:
  US 20070238520A1
• Time in Patent Office: 1,740 Days
• Field of Search: 715/706, 715/757, 715/778, 715/783, 715/839, 715/764, 463/32, 707101-102, 717/139, 717/137
• US Class Current: 717/139
• CPC Class Codes: A63F 13/10 Control of the course of the game, e.g. start, progess, end
• CPC Class Codes: A63F 13/45 Controlling the progress of the video game
• CPC Class Codes: A63F 13/537 using indicators, e.g. showing the condition of a game character on screen
• CPC Class Codes: A63F 13/60 Generating or modifying game content before or while executing the game program, e.g. authoring tools specially adapted for game development or game-integrated level editor
• CPC Class Codes: A63F 2300/6009 for importing or creating game content, e.g. authoring tools during game development, adapting content to different platforms, use of a scripting language to create content
• CPC Class Codes: A63F 2300/66 for rendering three dimensional images
• CPC Class Codes: A63F 2300/807 Role playing or strategy games

Abstract

Methods are disclosed for providing virtual objects and associating semantic annotations with the virtual objects. A user may be enabled to take virtual photographs of the virtual objects, wherein the semantic annotations associated with the virtual objects visible in the virtual photograph are stored with the virtual photograph. The virtual objects may be included in a virtual world. The user may be enabled to query a computer-controlled character about a virtual object in the virtual photograph, or in a virtual world in realtime, wherein the computer-controlled character is enabled to respond to the query by combining the semantic annotation associated with the virtual object in the virtual photograph with additional semantic annotations tagged to the computer-controlled character or a context of the virtual world to dynamically provide a semantic output, in various embodiments.

Claims

1. A method, implemented at least in part by a computing device comprising one or more processors, the method comprising:

providing, using one or more of the processors, at least one virtual object;

associating, using one or more of the processors, a semantic annotation with the virtual object;

providing, using one or more of the processors, a plurality of computer-controlled characters each having a different associated semantic knowledgebase associated therewith, in which the semantic knowledgebases are independent of the virtual object and the semantic annotation associated with the virtual object;

receiving a user-input selecting the virtual object by taking a virtual photograph of the virtual object and presenting it to any given one of the computer controlled characters;

automatically generating a query to the given computer-controlled character, the query corresponding to the virtual object presented to the given computer-controlled character;

dynamically providing, using one or more of the processors, a natural language output associated with the given computer-controlled character, in which the natural language output is responsive to the query and is based on a dynamic combination of the semantic annotation associated with the virtual object and the semantic knowledgebase associated with the given computer-controlled character, such that a natural language output from another computer-controlled character, that has an associated knowledgebase different from the knowledgebase associated with the given computer-controlled character, is different from the natural language output from the given computer-controlled character;

enabling a user to take the virtual photograph of an arbitrarily selected scene within the virtual world, in which the scene comprises the virtual object; and

storing a depiction of the scene comprising the virtual object as a virtual photograph, including storing the semantic annotation associated with the virtual object in a state the semantic annotation has at the time the virtual photograph is taken, with the virtual photograph, in which the semantic annotation is attached to the image of the virtual object in the virtual photograph.

2. The method of claim 1, in which the natural language output is based on a dynamic combination with a semantic annotation that is tagged to a context of a place and time in the virtual world to dynamically create semantic output in realtime.

3. The method of claim 1, wherein the semantic annotation is created by parsing a natural language input.

4. The method of claim 1, wherein the virtual object with which the semantic annotation is associated is an invisible object.

5. The method of claim 1, further comprising altering the semantic annotations associated with the virtual object to a new state in response to an event that occurs in the virtual world, enabling the given computer-controlled character to come in contact with the virtual object in the virtual world after the event, altering the semantic knowledgebase associated with the given computer-controlled character by adding one or more of the altered semantic annotations associated with the virtual object to the semantic knowledgebase associated with the given computer-controlled character, and in which the natural language output from the given computer-controlled character is based at least in part on the altered semantic knowledgebase.

6. The method of claim 1, further comprising altering the semantic knowledgebase associated with the given computer-controlled character is based on events that the given computer-controlled character observes in the virtual world, and in which the natural language output from the given computer controlled character is based at least in part on the altered semantic knowledgebase associated with the given computer-controlled character.

7. The method of claim 1, further comprising altering the semantic annotation associated with the virtual object based on events in the virtual world that influence the virtual object, and in which the natural language output from the given computer-controlled character is based at least in part on the altered semantic annotation associated with the virtual object.

8. The method of claim 1, further comprising enabling a user to show the virtual photograph to the given computer-controlled character, and providing a natural language output associated with the given computer-controlled character in which the natural language output is based on a dynamic combination of the semantic annotation attached to the virtual object in the virtual photograph, and the semantic knowledgebase associated with the given computer-controlled character.

9. A non-transitory computer-readable storage medium comprising computer-executable instructions which, when executed by a host computer, configure the host computer to:

receive input defining an object in a virtual world;

associate a semantic annotation with the object;

provide a plurality of different computer-controlled characters each having a different semantic knowledgebase associated therewith in which the semantic knowledgebases are independent of the object in the virtual world and the semantic annotation associated with the object;

receive a user input generating a photograph of the virtual object, the photograph including the semantic annotation, and presenting the photograph to a given one of the computer-controlled characters;

in response to presenting the photograph to the given computer-controlled character, automatically generate a query to the given computer-controlled character;

dynamically provide, using one or more of the processors, a natural language output associated with the computer-controlled character, in which the natural language output is responsive to the query and is based on a combination of the semantic annotation associated with the virtual object and the semantic knowledgebase associated with the computer-controlled character; and

enable a user to take the photograph as a virtual photo of the object within the virtual world, such that the semantic annotation associated with the object is stored with the photo at the time the photo is taken, and in a state the semantic annotation is in at the time the photo is taken, and remains associated with the object in the virtual photo.

10. The medium of claim 9, wherein the executable instructions further enable a computing device to receive an input relating to the object, and provide the semantic annotation based on the input relating to the object.

11. The medium of claim 10, wherein the executable instructions further enable a computing device to receive the input relating to the object as a natural language input, and automatically create the semantic annotation by parsing the natural language input.

12. The medium of claim 9, wherein the executable instructions further enable a computing device to store the object in a semantic screenshot, in which the semantic annotation remains associated with the object, and is stored in association with the object at the time the screenshot is stored;

and to provide a set of combined semantic information based on the semantic annotation in the semantic screenshot and a set of semantic information from a separate application.

13. A method, implemented at least in part by a computing device comprising one or more data storage components, the method comprising:

storing, using one or more of the data storage components, an image of a virtual setting;

storing, using one or more of the data storage components, at least one semantic tag with the image;

storing, using one or more of the data storage components, a relation between the semantic tag and at least one portion of the image;

storing, using one or more of the data storage components, a plurality of different computer-controlled characters, each having a different semantic knowledgebase associated therewith in which the semantic knowledgebase for each computer-controlled character is independent of the image and the semantic tag stored in relation to the at least one portion of the image;

responding to a query, by a user of the virtual setting, of the at least one portion of the image, by dynamically providing a natural language output associated with a given one of the computer-controlled characters, in which the natural language output is based on a combination of the semantic tag and the semantic knowledgebase associated with the given computer-controlled character such that a response from another computer-controlled character, with a different associated semantic knowledgebase, is different from the natural language output from the given computer-controlled character;

enabling a user to take the virtual photograph of an arbitrarily selected scene within the virtual world, in which the scene comprises the virtual object; and

storing a depiction of the scene comprising the virtual object as a virtual photograph, including storing the semantic annotation associated with the virtual object in a state the semantic annotation has at the time the virtual photograph is taken, with the virtual photograph, in which the semantic annotation is attached to the image of the virtual object in the virtual photograph.

14. The method of claim 13, further comprising combining the semantic tag with additional semantic information to provide a dynamically composed output;

and providing the dynamically composed output in response to a query by a user of the virtual setting, wherein the additional semantic information is received from semantic tags associated with a context of the virtual setting when the query is received.

15. The method of claim 13, further comprising combining the semantic tag with additional semantic information to provide a dynamically composed output;

and providing the dynamically composed output in response to a query by a user of the virtual setting, wherein the additional semantic information is received from semantic tags associated with a computer-controlled character of the virtual setting when the query is received.

16. The method of claim 13, wherein the image comprises pixels that each have associated with them color component values and an object identifier, wherein the at least one portion of the image has a single object identifier associated with each of the pixels comprised in that portion of the image.

17. The method of claim 13, wherein the virtual setting is modeled as a three-dimensional virtual world, and the image is stored as a two-dimensional virtual photograph of a scene in the three-dimensional virtual world.