METHOD. APPARATUS AND COMPUTER PROGRAM PRODUCT FOR UTILIZING REAL-WORLD AFFORDANCES OF OBJECTS IN AUDIO-VISUAL MEDIA DATA TO DETERMINE INTERACTIONS WITH THE ANNOTATIONS TO THE OBJECTS

• US 20090161963A1
• Filed: 12/20/2007
• Published: 06/25/2009
• Est. Priority Date: 12/20/2007
• Status: Abandoned Application

• Current Assignee: Nokia Corporation
• Original Assignee: Nokia Corporation
• Inventors: Arrasvuori, Juha, Uusitalo, Jussi Severi
• Application Number:
  US11/961,467
• Publication Number:
  US 20090161963A1
• Time in Patent Office: Days
• US Class Current: 382/203
• CPC Class Codes: G06F 16/748 Hypervideo linking data to content, e.g. by linking an URL to a video object in the context of video distribution systems H04N21/858
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00

Abstract

An apparatus for determining interactions with annotations to objects based upon real-world affordances of the objects in audio-visual media data may include a processing element configured to receive image data describing one or more objects having real-world affordances, to identify the one or more objects having real-world affordances, and to create one or more semiotic regions by associating with the one or more objects interaction rules corresponding to the respective real-world affordances of the one or more objects.

Claims

1. A method comprising:

receiving audio-visual media data describing one or more objects having real-world affordances;

identifying the one or more objects having real-world affordances; and

in response to the one or more objects having real-world affordances, creating one or more semiotic regions by associating with the one or more objects interaction rules corresponding to the respective real-world affordances of the one or more objects.

2. The method of claim 1, wherein identifying the one or more objects having real-world affordances comprises one or more of:

analyzing image data contained within the audio-visual media data with a shape recognition algorithm for identifying the one or more objects having real-world affordances;

comparing the image data to reference image data describing predefined objects having real-world affordances to identify one or more objects having real-world affordances within the image data;

comparing audio data contained within the audio-visual media data to reference audio data corresponding to predefined sound-making objects having real-world affordances to identify one or more objects having real-world affordances within the audio data;

or

receiving an indication identifying one or more objects having real-world affordances.

3. The method of claim 1, further comprising receiving an indication of a location associated with the audio-visual media data and identifying one or more owners associated with an object appearing in the audio-visual media data based upon the indication of a location.

4. The method of claim 3, further comprising receiving indicia defining access rights to the object appearing in the audio-visual media data from the one or more identified owners associated with the object and associating those access rights with the object.

5. The method of claim 1, further comprising identifying one or more objects described by the audio-visual media data in which a third party has intellectual property rights and associating interaction rules with the one or more objects in which a third party has intellectual property rights, wherein the interaction rules prevent the attachment of tags, annotations, or other content to the associated objects without the permission of the third party.

6. The method of claim 1, further comprising identifying one or more transitory objects described by the audio-visual media data and associating interaction rules with the one or more transitory objects, wherein the interaction rules prevent the attachment of tags, annotations, or other content to the associated transitory objects.

7. The method of claim 1, wherein creating one or more semiotic regions by associating interaction rules corresponding to the respective real-world affordances of the one or more objects with the one or more objects comprises determining whether the identified object corresponds to a predefined association selected from the group comprising:

a window object wherein any associated content can be accessed, but not edited and additional annotations can not be made to the semiotic region;

a door object, wherein the door must be opened before any associated content can be accessed;

a wall object, wherein any user may access or add annotated content to the semiotic region;

a television screen object, wherein only video content may be annotated within or accessed from the semiotic region;

a bookshelf object, wherein only books or other similar written content can be annotated within or accessed from the semiotic region;

a newspaper stand object, wherein only news stories comprising one or more of links to online news stories or RSS feeds can be annotated within or accessed from the semiotic region;

a trash bin object, wherein content perceived as garbage can be annotated within or accessed from the semiotic region;

a bus object, wherein content associated with a route which the real-world bus travels may be accessed from the semiotic region;

a game object, wherein a game application may be accessed from the semiotic region; and

a tool object, wherein the tool object may be interacted with and have an effect on other objects in the audio-visual media data.

8. A computer program product comprising at least one computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions comprising:

a first executable portion for receiving audio-visual media data describing one or more objects having real-world affordances;

a second executable portion for identifying the one or more objects having real-world affordances; and

a third executable portion for creating one or more semiotic regions by associating with the one or more objects interaction rules corresponding to the respective real-world affordances of the one or more objects.

9. The computer program product of claim 8, wherein the second executable portion includes instructions for identifying one or more objects having real-world affordances by one or more of:

analyzing image data contained within the audio-visual media data with a shape recognition algorithm for identifying the one or more objects having real-world affordances;

comparing the image data to reference image data describing predefined objects having real-world affordances to identify one or more objects having real-world affordances within the image data;

comparing audio data contained within the audio-visual media data to reference audio data corresponding to predefined sound-making objects having real-world affordances to identify one or more objects having real-world affordances within the audio data;

or

receiving an indication identifying one or more objects having real-world affordances.

10. The computer program product of claim 8, further comprising a fourth executable portion for receiving an indication of a location associated with the audio-visual media data and identifying one or more owners associated with an object appearing in the audio-visual media data based upon the indication of a location.

11. The computer program product of claim 10, further comprising receiving indicia defining access rights to the object appearing in the audio-visual media data from the one or more identified owners associated with the object and associating those access rights with the object.

12. The computer program product of claim 8, further comprising a fourth executable portion for identifying one or more objects described by the audio-visual media data in which a third party has intellectual property rights and associating interaction rules with the one or more objects in which a third party has intellectual property rights, wherein the interaction rules prevent the attachment of tags, annotations, or other content to the associated objects without the permission of the third party.

13. The computer program product of claim 8, further comprising a fourth executable portion for identifying one or more transitory objects described by the audio-visual media data and associating interaction rules with the one or more transitory objects, wherein the interaction rules prevent the attachment of tags, annotations, or other content to the associated transitory objects.

14. The computer program product of claim 8, wherein the third executable portion includes instructions for creating one or more semiotic regions by associating interaction rules corresponding to the real-world affordances of the one or more objects with the one or more objects by determining whether the identified object corresponds to a predefined association selected from the group comprising:

a window object wherein any associated content can be accessed, but not edited and additional annotations can not be made to the semiotic region;

a door object, wherein the door must be opened before any associated content can be accessed;

a wall object, wherein any user may access or add annotated content to the semiotic region;

a television screen object, wherein only video content may be annotated within or accessed from the semiotic region;

a bookshelf object, wherein only books or other similar written content can be annotated within or accessed from the semiotic region;

a newspaper stand object, wherein only news stories comprising one or more of links to online news stories or RSS feeds can be annotated within or accessed from the semiotic region;

a trash bin object, wherein content perceived as garbage can be annotated within or accessed from the semiotic region;

a bus object, wherein content associated with a route which the real-world bus travels may be accessed from the semiotic region;

a game object, wherein a game application may be accessed from the semiotic region; and

a tool object, wherein the tool object may be interacted with and have an effect on other objects in the audio-visual media data.

15. An apparatus comprising a processing element configured to:

receive audio-visual media data describing one or more objects having real-world affordances;

identify the one or more objects having real-world affordances; and

create one or more semiotic regions by associating with the one or more objects interaction rules corresponding to the respective real-world affordances of the one or more objects.

16. The apparatus of claim 15, wherein the processing element is further configured to identify the one or more objects having real-world affordances based upon one or more of:

analyzing image data contained within the audio-visual media data with a shape recognition algorithm for identifying the one or more objects having real-world affordances;

comparing the image data to reference image data describing predefined objects having real-world affordances to identify one or more objects having real-world affordances within the image data;

comparing audio data contained within the audio-visual media data to reference audio data corresponding to predefined sound-making objects having real-world affordances to identify one or more objects having real-world affordances within the audio data;

or

receiving an indication identifying one or more objects having real-world affordances.

17. The apparatus of claim 15, wherein the processing element is further configured to receive an indication of a location associated with the audio-visual media data and to identify one or more owners associated with an object appearing in the image data based upon the indication of a location.

18. The apparatus of claim 16, wherein the processing element is further configured to receive indicia defining access rights to the object appearing in the audio-visual media data from the one or more identified owners associated with the object and to associate those access rights with the object.

19. The apparatus of claim 15, wherein the processing element is further configured to identify one or more objects described by the audio-visual media data in which a third party has intellectual property rights and to associate interaction rules with the one or more objects in which a third party has intellectual property rights, wherein the interaction rules prevent the attachment of tags, annotations, or other content to the associated objects without the permission of the third party.

20. The apparatus of claim 15, wherein the processing element is further configured to identify one or more transitory objects described by the audio-visual media data and to associate interaction rules with the one or more transitory objects, wherein the interaction rules prevent the attachment of tags, annotations, or other content to the associated transitory objects.

21. The apparatus of claim 15, wherein the processing element is further configured to create one or more semiotic regions by associating interaction rules corresponding to the real-world affordances of the one or more objects with the one or more objects by determining whether the identified object corresponds to a predefined association selected from the group comprising:

a window object wherein any associated content can be accessed, but not edited and additional annotations can not be made to the semiotic region;

a door object, wherein the door must be opened before any associated content can be accessed;

a wall object, wherein any user may access or add annotated content to the semiotic region;

a television screen object, wherein only video content may be annotated within or accessed from the semiotic region;

a bookshelf object, wherein only books or other similar written content can be annotated within or accessed from the semiotic region;

a newspaper stand object, wherein only news stories comprising one or more of links to online news stories or RSS feeds can be annotated within or accessed from the semiotic region;

a trash bin object, wherein content perceived as garbage can be annotated within or accessed from the semiotic region;

a bus object, wherein content associated with a route which the real-world bus travels may be accessed from the semiotic region;

a game object, wherein a game application may be accessed from the semiotic region; and

a tool object, wherein the tool object may be interacted with and have an effect on other objects in the audio-visual media data.

22. An apparatus comprising:

means for receiving audio-visual media data describing one or more objects having real-world affordances;

means for identifying the one or more objects having real-world affordances; and

means for creating one or more semiotic regions by associating with the one or more objects interaction rules corresponding to the respective real-world affordances of the one or more objects.