System and method for interactive projection and playback of relevant media segments onto the facets of three-dimensional shapes

• US 8,762,890 B2
• Filed: 07/27/2011
• Issued: 06/24/2014
• Est. Priority Date: 07/27/2010
• Status: Active Grant

• Current Assignee: Telcordia Technologies Incorporated (Telefonaktiebolaget LM Ericsson)
• Original Assignee: Telcordia Technologies Incorporated (Telefonaktiebolaget LM Ericsson)
• Inventors: Falchuk, Benjamin, Wu, Chung Ying
• Primary Examiner(s): Kim, Matt
• Assistant Examiner(s): Lu, Hua
• Application Number:
  US13/191,773
• Publication Number:
  US 20120192115A1
• Time in Patent Office: 1,063 Days
• Field of Search: None
• US Class Current: 715/848
• CPC Class Codes: G06F 2203/04802 3D-info-object: information is displayed on the internal or external surface of a three dimensional manipulable object, e.g. on the faces of a cube that can be rotated by the user
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

A system for interactive media skimming and search on a device comprises a scene manager building a model of a 3D scene of a multimedia segment of the media visible on a screen of the device, facets of scene objects in the 3D scene are used to dynamically convey visual imagery as a texture from a multimedia source and maintaining a logical navigable relationship between the scene objects, the object facets and the multimedia segments, and further comprising an interaction manager, a user manager securely storing user information and preferences, a playback component initiating rough or high definition playback, a texturizer creating a 2D texture artifact from a set of the multimedia segments in the media for a 3D facet, a transformer transforming and modifying pixels, a cache and scene heuristics maintaining a set of 3D scenes comprising objects, object facets, a virtual camera, and positions of the objects.

Claims

1. A system for interactive media skimming and search on a device, comprising:

a scene manager operable to build a model of a 3D scene of a multimedia segment of the media visible on a screen of the device in which one or more facets of scene objects are used to dynamically convey visual imagery as a texture from a multimedia source and operable to maintain a logical navigable relationship between the scene objects, the object facets and the multimedia segments;

an interaction manager operable to catch and manage interactions of a user with the model of the 3D scene, operable to instruct the scene manager to map screen pixels to the object facets and in turn to the multimedia segments, and operable to simplify user experience by limiting interaction parsing when in particular modes;

a user manager operable to securely store a user name, password, authorizations, and user preferences comprising at least preferred 3D scene types for particular multimedia categories;

a playback component operable to initiate one of rough playback and high definition playback in response to a user interaction and operable to determine an appropriate facet on a 3D shape in the 3D scene and to transform the multimedia such that the multimedia appears to play upon the appropriate facet;

a texturizer operable to create a 2D texture artifact from a specified set of the multimedia segments in the media for a particular target 3D facet such that resulting operations to view the texture as if stretched onto the particular target 3D facet are successful and make one or more perspective of the multimedia clearly visible on the particular target 3D facet;

a transformer operable to transform and modify pixels of one or more of chosen media segments and textures in response to requests such that the resulting 2D texture artifact contains new information via the pixels that are revealed in full only when the resulting 2D texture artifact is projected onto one or more shape facets in the 3D scene;

a cache operable to store and index one or more of the media segments, the textures, scene metadata, and the preferred 3D scene types and objects correlated to the multimedia categories and individual segments; and

scene heuristics operable to maintain a set of 3D scenes comprising one or more objects, object facets, a virtual camera, and positions of the objects relative to the camera and the relationship of each set of the set of 3D scenes to a particular multimedia category or to a specific set of named multimedia resources, wherein when a valid user name and password are determined by the user manager, the scene manager builds the model in accordance with the texturizer, the transformer and the cache;

wherein one or more of the interactions caught and managed by the interaction manager are interpreted as instructions to rotate, scale, and translate 3D shapes whose surfaces have been mapped with cues, summaries, or other information relevant to the interactive media skimming and search.

2. The system according to claim 1, the scene manager further operable to select a 3D shape and a means for ordering visual information upon facets of the 3D shape wherein a series of ordered media segments are ordered such that coarse segment ordering can be visually seen upon one aspect of the 3D shape and segment action can be visible upon another aspect of the 3D shape.

3. The system according to claim 1, the scene manager further operable to perform one or more of dynamically adapting the 3D scene between skimming sessions, and dynamically creating a related media scene for any given media such that 3D shapes are placed in relation to each other and the virtual camera so that relative position of the 3D shapes and the virtual camera to a principle shape convey relevance to the principle shape and each placed 3D shape conveys media information via interactions with the texturizer and the transformer.

4. The system according to claim 1, the scene manager further operable to choose 3D shapes that have initially obscured facets and to use the texturizer and the transformer to create projections of multimedia content onto both obscured and visible facets, making a view of the initially obscured facets visible only via particular predefined user interactions.

5. The system according to claim 1, the interaction manager further operable to perform one or more of decomposing and then limiting and enhancing user interactions with the interaction manager, differentiating exploratory user interactions from user interactions showing interest or disinterest in the visible contents on the 3D scene, and affecting the virtual camera actions based on decomposed and processed user interactions.

6. The system according to claim 1, wherein the high definition playback launches an external player with metadata about a current scene and current location being browsed within the media.

7. The system according to claim 1, the texturizer further operable to perform one or more of confirming a texture file to correspond to a given request for a particular object type, and creating a texture shape for the particular object type from the specified set of media segments.

8. The system according to claim 1, the transformer further operable to receive as input identification of a media resource, and to create as output a handle to the transformed media, wherein the resulting media has one of an advertisement bitmap stitched into a form of the resulting media, the resulting media has a new form making the resulting media more suitable as a texture for a particular 3D shape, and the resulting media includes highlighted information that did not exist in the original.

9. The system according to claim 8, wherein the transformer creates the output by determining semantics of a current view item or texture, determining a preferred cardinality and placement of advertisement bitmaps, determining specific segments having associated advertisements, replacing or augmenting the specific segments with advertisement images, and storing instructions describing how to respond to inputs on the specific segments containing the advertisement images.

10. The system according to claim 8, wherein user input on particular advertisements embedded by the system into the textures results in an optional virtual camera repositioning and new information related to the particular advertisement displaying on a particular set of facets of one or more of the 3D shapes in the 3D scene.

11. The system according to claim 8, wherein a user interaction upon the 3D scene results in an adjustment comprising one or more of rotation, translation, or scale of shapes in the scene, wherein as the adjustment occurs in view of the user, the rapidly changing visible imagery conveys what appears to the user as an animated view of a multimedia segment in playback.

12. A method for interactive media skimming and search on a device, comprising steps of:

building a model of a 3D scene of a multimedia segment of the media visible on a screen of the device in which one or more facets of scene objects are used to dynamically convey visual imagery as a texture from a multimedia source and maintaining a logical navigable relationship between the scene objects, the object facets and the multimedia segments;

catching and managing interactions of a user with the model of the 3D scene, mapping screen pixels to the object facets and in turn to the multimedia segments, and simplifying user experience by limiting interaction parsing when in particular modes;

securely storing a user name, password, authorizations, and user preferences comprising at least preferred 3D scene types for particular multimedia categories;

initiating one of rough playback and high definition playback in response to a user interaction and determining an appropriate facet on a 3D shape in the 3D scene and transforming the multimedia such that the multimedia appears to play upon the appropriate facet;

creating, using a texturizer, a 2D texture artifact from a specified set of the multimedia segments in the media for a particular target 3D facet such that resulting operations to view the texture as if stretched onto the particular target 3D facet are successful and make one or more perspective of the multimedia clearly visible on the particular target 3D facet;

transforming and modifying, using a transformer, pixels of one or more of chosen media segments and textures in response to requests such that the resulting 2D texture artifact contains new information via the pixels that are revealed in full only when the resulting 2D texture artifact is projected onto one or more shape facets in the 3D scene;

storing and indexing one or more of the media segments, the textures, scene metadata, and the preferred 3D scene types and objects correlated to the multimedia categories and individual segments; and

maintaining a set of 3D scenes comprising one or more objects, object facets, a virtual camera, and positions of the objects relative to the camera and the relationship of each set of the set of 3D scenes to a particular multimedia category or to a specific set of named multimedia resources, wherein when a valid user name and password are determined, building the model is performed in accordance with the texturizer, the transformer, and storing and indexing the media segments;

wherein one or more of the interactions are interpreted as instructions to rotate, scale, and translate 3D shapes whose surfaces have been mapped with cues, summaries, or other information relevant to the interactive media skimming and search.

13. The method according to claim 12, the step of building the model further comprising selecting a 3D shape and a means for ordering visual information upon facets of the 3D shape wherein a series of ordered media segments are ordered such that coarse segment ordering can be visually seen upon one aspect of the 3D shape and segment action can be visible upon another aspect of the 3D shape.

14. The method according to claim 12, the step of building the model further comprising performing one or more of dynamically adapting the 3D scene between skimming sessions, and dynamically creating a related media scene for any given media such that 3D shapes are placed in relation to each other and the virtual camera so that relative position of the 3D shapes and the virtual camera to a principle shape convey relevance to the principle shape and each placed 3D shape conveys media information via interactions with the texturizer and the transformer.

15. The method according to claim 12, the step of building the model further comprising choosing 3D shapes that have initially obscured facets and to use the texturizer and the transformer to create projections of multimedia content onto both obscured and visible facets, making a view of the initially obscured facets visible only via particular predefined user interactions.

16. The method according to claim 12, the step of catching and managing interactions further operable to perform one or more of decomposing and then limiting and enhancing user interactions with the interaction manager, differentiating exploratory user interactions from user interactions showing interest or disinterest in the visible contents on the 3D scene, and affecting the virtual camera actions based on decomposed and processed user interactions.

17. The method according to claim 12, wherein the high definition playback launches an external player with metadata about a current scene and current location being browsed within the media.

18. The method according to claim 12, the step of creating the 2D artifact further comprising one or more of confirming a texture file to correspond to a given request for a particular object type, and creating a texture shape for the particular object type from the specified set of media segments.

19. The method according to claim 12, the step of transforming and modifying pixels of the one or more chosen media segments further comprising receiving as input identification of a media resource, and creating as output a handle to the transformed media, wherein the resulting media has one of an advertisement bitmap stitched into a form of the resulting media, the resulting media has a new form making the resulting media more suitable as a texture for a particular 3D shape, and the resulting media includes highlighted information that did not exist in the original.

20. The method according to claim 19, wherein the output is created by determining semantics of a current view item or texture, determining a preferred cardinality and placement of advertisement bitmaps, determining specific segments having associated advertisements, replacing or augmenting the specific segments with advertisement images, and storing instructions describing how to respond to inputs on the specific segments containing the advertisement images.

21. The method according to claim 19, wherein user input on particular advertisements embedded by the system into the textures results in an optional virtual camera repositioning and new information related to the particular advertisement displaying on a particular set of facets of one or more of the 3D shapes in the 3D scene.

22. The method according to claim 19, wherein a user interaction upon the 3D scene results in an adjustment comprising one or more of rotation, translation, or scale of shapes in the scene, wherein as the adjustment occurs in view of the user, the rapidly changing visible imagery conveys what appears to the user as an animated view of a multimedia segment in playback.

23. A non-transitory computer readable storage medium storing a program of instructions executable by a machine to perform a method for interactive media skimming and search on a device, comprising:

building a model of a 3D scene of a multimedia segment of the media visible on a screen of the device in which one or more facets of scene objects are used to dynamically convey visual imagery as a texture from a multimedia source and maintaining a logical navigable relationship between the scene objects, the object facets and the multimedia segments;

catching and managing interactions of a user with the model of the 3D scene, mapping screen pixels to the object facets and in turn to the multimedia segments, and simplifying user experience by limiting interaction parsing when in particular modes;

securely storing a user name, password, authorizations, and user preferences comprising at least preferred 3D scene types for particular multimedia categories;

initiating one of rough playback and high definition playback in response to a user interaction and determining an appropriate facet on a 3D shape in the 3D scene and transforming the multimedia such that the multimedia appears to play upon the appropriate facet;

creating a 2D texture artifact from a specified set of the multimedia segments in the media for a particular target 3D facet such that resulting operations to view the texture as if stretched onto the particular target 3D facet are successful and make one or more perspective of the multimedia clearly visible on the particular target 3D facet;

transforming and modifying pixels of one or more of chosen media segments and textures in response to requests such that the resulting 2D texture artifact contains new information via the pixels that are revealed in full only when the resulting 2D texture artifact is projected onto one or more shape facets in the 3D scene;

storing and indexing one or more of the media segments, the textures, scene metadata, and the preferred 3D scene types and objects correlated to the multimedia categories and individual segments; and

maintaining a set of 3D scenes comprising one or more objects, object facets, a virtual camera, and positions of the objects relative to the camera and the relationship of each set of the set of 3D scenes to a particular multimedia category or to a specific set of named multimedia resources, wherein when a valid user name and password are determined, building the model is performed in accordance with creating the 2D texture artifact, transforming and modifying the pixels of the one or more chosen media segments, and storing and indexing the media segments;

wherein one or more of the interactions caught and managed by the interaction manager are interpreted as instructions to rotate, scale, and translate 3D shapes whose surfaces have been mapped with cues, summaries- or other information relevant to the interactive media skimming and search.