METHOD, APPARATUS AND COMPUTER PROGRAM PRODUCT FOR UTILIZING REAL-WORLD AFFORDANCES OF OBJECTS IN AUDIO-VISUAL MEDIA DATA TO DETERMINE INTERACTIONS WITH THE ANNOTATIONS TO THE OBJECTS

• US 20110096992A1
• Filed: 12/30/2010
• Published: 04/28/2011
• Est. Priority Date: 12/20/2007
• Status: Abandoned Application

• Current Assignee: Conversant Wireless Licensing S.à.r.l. (f/k/a Core Wireless Licensing S.a.r.l.) (MOSAID Technologies Inc. (f/k/a Conversant Intellectual Property Management))
• Original Assignee: Nokia Corporation
• Inventors: Uusitalo, Jussi Severi, Arrasvuori, Juha
• Application Number:
  US12/982,234
• Publication Number:
  US 20110096992A1
• Time in Patent Office: Days
• US Class Current: 382/173
• CPC Class Codes: G06F 16/748 Hypervideo linking data to content, e.g. by linking an URL to a video object in the context of video distribution systems H04N21/858
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00

Abstract

An apparatus for determining interactions with annotations to objects based upon real-world affordances of the objects in audio-visual media data may include a processing element configured to receive image data describing one or more objects having real-world affordances, to identify the one or more objects having real-world affordances, and to create one or more semiotic regions by associating with the one or more objects interaction rules corresponding to the respective real-world affordances of the one or more objects.

Claims

1. A method comprising:

receiving audio-visual media data describing one or more objects having real-world affordances;

identifying the one or more objects having real-world affordances; and

in response to the one or more objects having real-world affordances, creating one or more semiotic regions by associating with the one or more objects interaction rules corresponding to one or more actions associated with the respective real-world affordances of the one or more objects.

2. The method of claim 1, wherein identifying the one or more objects having real-world affordances comprises one or more of:

analyzing image data contained within the audio-visual media data with a shape recognition algorithm for identifying the one or more objects having real-world affordances;

comparing the image data to reference image data describing predefined objects having real-world affordances to identify one or more objects having real-world affordances within the image data;

comparing audio data contained within the audio-visual media data to reference audio data corresponding to predefined sound-making objects having real-world affordances to identify one or more objects having real-world affordances within the audio data;

or

receiving an indication identifying one or more objects having real-world affordances.

3. The method of claim 1, further comprising receiving an indication of a location associated with the audio-visual media data and identifying one or more owners associated with an object appearing in the audio-visual media data based upon the indication of a location.

4. The method of claim 3, further comprising receiving indicia defining access rights to the object appearing in the audio-visual media data from the one or more identified owners associated with the object and associating those access rights with the object.

5. The method of claim 1, further comprising identifying one or more objects described by the audio-visual media data in which a third party has intellectual property rights and associating interaction rules with the one or more objects in which a third party has intellectual property rights, wherein the interaction rules prevent the attachment of tags, annotations, or other content to the associated objects without the permission of the third party.

6. The method of claim 1, further comprising identifying one or more transitory objects described by the audio-visual media data and associating interaction rules with the one or more transitory objects, wherein the interaction rules prevent the attachment of tags, annotations, or other content to the associated transitory objects.

7. The method of claim 1, wherein creating one or more semiotic regions by associating interaction rules corresponding to the respective real-world affordances of the one or more objects with the one or more objects comprises determining whether the identified object corresponds to a predefined association selected from the group comprising:

a window object wherein any associated content can be accessed, but not edited and additional annotations can not be made to the semiotic region;

a door object, wherein the door must be opened before any associated content can be accessed;

a wall object, wherein any user may access or add annotated content to the semiotic region;

a television screen object, wherein only video content may be annotated within or accessed from the semiotic region;

a bookshelf object, wherein only books or other similar written content can be annotated within or accessed from the semiotic region;

a newspaper stand object, wherein only news stories comprising one or more of links to online news stories or RSS feeds can be annotated within or accessed from the semiotic region;

a trash bin object, wherein content perceived as garbage can be annotated within or accessed from the semiotic region;

a bus object, wherein content associated with a route which the real-world bus travels may be accessed from the semiotic region;

a game object, wherein a game application may be accessed from the semiotic region; and

a tool object, wherein the tool object may be interacted with and have an effect on other objects in the audio-visual media data.

8. A computer-readable storage medium carrying one or more sequences of one or more instructions which, when executed by one or more processors, cause an apparatus to at least perform the following steps:

receiving audio-visual media data describing one or more objects having real-world affordances;

identifying the one or more objects having real-world affordances; and

creating one or more semiotic regions by associating with the one or more objects interaction rules corresponding to one or more actions associated with the respective real-world affordances of the one or more objects.

9. The computer-readable storage medium of claim 8, wherein the step of identifying one or more objects having real-world affordances comprises one or more of:

analyzing image data contained within the audio-visual media data with a shape recognition algorithm for identifying the one or more objects having real-world affordances;

comparing the image data to reference image data describing predefined objects having real world affordances to identify one or more objects having real-world affordances within the image data;

comparing audio data contained within the audio-visual media data to reference audio data corresponding to predefined sound-making objects having real-world affordances to identify one or more objects having real-world affordances within the audio data;

or

receiving an indication identifying one or more objects having real-world affordances.

10. The computer-readable storage medium of claim 8, wherein the apparatus is caused to further perform:

receiving an indication of a location associated with the audio-visual media data; and

identifying one or more owners associated with an object appearing in the audio-visual media data based upon the indication of a location.

11. The computer-readable storage medium of claim 10, wherein the apparatus is caused to further perform:

further comprising

receiving indicia defining access rights to the object appearing in the audio-visual media data from the one or more identified owners associated with the objects and

associating those access rights with the object.

12. The computer-readable storage medium of claim 8, wherein the apparatus is caused to further perform:

identifying one or more objects described by the audio-visual media data in which a third party has intellectual property rights and associating interaction rules with the one or more objects in which a third party has intellectual property rights, wherein the interaction rules prevent the attachment of tags, annotations, or other content to the associated objects without the permission of the third party.

13. The computer-readable storage medium of claim 8, wherein the apparatus is caused to further perform:

identifying one or more transitory objects described by the audio-visual media data and associating interaction rules with the one or more transitory objects, wherein the interaction rules prevent the attachment of tags, annotations, or other content to the associated transitory objects.

14. The computer-readable storage medium of claim 8, wherein step of creating one or more semiotic regions by associating interaction rules corresponding to the real-world affordances of the one or more objects with the one or more objects by causing the apparatus to further perform:

determining whether the identified object corresponds to a predefined association selected from the group comprising;

a window object wherein any associated content can be accessed, but not edited and additional annotations can not be made to the semiotic region;

a door object, wherein the door must be opened before any associated content can be accessed;

a wall object, wherein any user may access or add annotated content to the semiotic region;

a television screen object, wherein only video content may be annotated within or accessed from the semiotic region;

a bookshelf object, wherein only books or other similar written content can be annotated within or accessed from the semiotic region;

a newspaper stand object, wherein only news stories comprising one or more of links to online news stories or RSS feeds can be annotated within or accessed from the semiotic region;

a trash bin object, wherein content perceived as garbage can be annotated within or accessed from the semiotic region;

a bus object, wherein content associated with a route which the real-world bus travels may be accessed from the semiotic region;

a game object, wherein a game application may be accessed from the semiotic region; and

a tool object, wherein the tool object may be interacted with and have an effect on other objects in the audio-visual media data.

15. An apparatus comprising:

at least one processor; and

at least one memory including computer program code for one or more programs,

the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following

receive audio-visual media data describing one or more objects having real-world affordances;

identify the one or more objects having real-world affordances; and

create one or more semiotic regions by associating with the one or more objects interaction rules corresponding to one or more actions associated with the respective real-world affordances of the one or more objects.

16. The apparatus of claim 15, wherein the apparatus is further caused to:

identify the one or more objects having real-world affordances based upon one or more of;

analyzing image data contained within the audio-visual media data with a shape recognition algorithm for identifying the one or more objects having real-world affordances;

comparing the image data to reference image data describing predefined objects having real-world affordances to identify one or more objects having real-world affordances within the image data;

comparing audio data contained within the audio-visual media data to reference audio data corresponding to predefined sound-making objects having real-world affordances to identify one or more objects having real-world affordances within the audio data;

Or

receiving an indication identifying one or more objects having real-world affordances.

17. The apparatus of claim 15, wherein the apparatus is further caused to:

receive an indication of a location associated with the audio-visual media data and to identify one or more owners associated with an object appearing in the image data based upon the indication of a location.

18. The apparatus of claim 16, wherein the apparatus is further caused to:

receive indicia defining access rights to the object appearing in the audio-visual media data from the one or more identified owners associated with the object and to associate those access rights with the object.

19. The apparatus of claim 15, wherein the apparatus is further caused to:

identify one or more objects described by the audio-visual media data in which a third party has intellectual property rights and to associate interaction rules with the one or more objects in which a third party has intellectual property rights,

wherein the interaction rules prevent the attachment of tags, annotations, or other content to the associated objects without the permission of the third party.

20. The apparatus of claim 15, wherein the apparatus is further caused to:

identify one or more transitory objects described by the audio-visual media dataz and to associate interaction rules with the one or more transitory objects,

wherein the interaction rules prevent the attachment of tags, annotations, or other content to the associated transitory objects.