Methods and apparatus for content interaction

• US 9,654,844 B2
• Filed: 09/11/2015
• Issued: 05/16/2017
• Est. Priority Date: 09/12/2014
• Status: Active Grant

• Current Assignee: Kiswe Mobile Inc.
• Original Assignee: Kiswe Mobile Inc.
• Inventors: Kim, Jeong H., Janiszewski, Thomas J., Ho, Yung-Lung, Sweldens, Wim, Cuypers, Tom, Zane, Francis X.
• Primary Examiner(s): Kumar, Pankaj
• Assistant Examiner(s): ANDRAMUNO, FRANKLIN S
• Application Number:
  US14/851,942
• Publication Number:
  US 20160080830A1
• Time in Patent Office: 613 Days
• Field of Search: 725 74, 725133
• CPC Class Codes: H04N 21/21805 enabling multiple viewpoints, e.g. using a plurality of cameras
• CPC Class Codes: H04N 21/2187 Live feed
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/26603 for automatically generating descriptors from content, e.g. when it is not made available by its provider, using content analysis techniques
• CPC Class Codes: H04N 21/2665 Gathering content from different sources, e.g. Internet and satellite
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4622 Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 21/485 End-user interface for client configuration
• CPC Class Codes: H04N 21/4858 for modifying screen layout parameters, e.g. fonts, size of the windows
• CPC Class Codes: H04N 21/8126 involving additional data, e.g. news, sports, stocks, weather forecasts
• CPC Class Codes: H04N 21/8133 specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video program
• CPC Class Codes: H04N 21/8193 dedicated tools, e.g. video decoder software or IPMP tool
• CPC Class Codes: H04N 21/8352 involving content or source identification data, e.g. Unique Material Identifier [UMID]

Abstract

A method comprises receiving content from at least one source, processing the content to identify an object in the content, assigning an identifier to the object based on the processing, permitting viewer selection of the object identifier, and providing additional information for presentation to a viewer of the content responsive to viewer selection of the object identifier. The content in some embodiments comprises live video from at least one live video source. For example, the content may comprise at least a portion of a panoramic video comprising a combination of multiple live videos from respective ones of a plurality of video sources. The panoramic video may be digitally cropped and scaled before a resulting version of the content is provided for presentation to the viewer. A plurality of different digitally cropped and scaled versions of the content from the panoramic video can be generated and made available to the viewer.

Claims

1. A method comprising:

receiving content from at least one source;

processing the content to identify an object in the content;

assigning an identifier to the object based on the processing;

permitting viewer selection of the object identifier; and

providing additional information for presentation to a viewer of the content responsive to viewer selection of the object identifier;

wherein the content comprises at least a portion of a panoramic video comprising a combination of multiple live videos from respective ones of a plurality of video sources;

the method further comprising digitally cropping and scaling the panoramic video before providing a resulting version of the content for presentation to the viewer;

wherein the receiving, processing, assigning, permitting and providing are implemented in at least one processing device comprising a processor coupled to a memory;

wherein said at least one processing device is further configured to generate a plurality of different digitally cropped and scaled versions of the content from the panoramic video; and

wherein said at least one processing device is further configured to provide a first one of the plurality of different digitally cropped and scaled versions of the content for presentation to the viewer, and responsive to input from the viewer, to provide a second one of the plurality of different digitally cropped and scaled versions of the content for presentation to the viewer, wherein the second one of the plurality of different digitally cropped and scaled versions of the content is provided for presentation from a rewind point specified by the viewer.

2. The method of claim 1 wherein the content comprises live video from at least one live video source.

3. The method of claim 1 further comprising generating a plurality of different digitally cropped and scaled versions of the content from the panoramic video.

4. The method of claim 3 further comprising:

providing a first one of the plurality of different digitally cropped and scaled versions of the content for presentation to the viewer; and

responsive to input from the viewer, providing a second one of the plurality of different digitally cropped and scaled versions of the content for presentation to the viewer;

wherein the second one of the plurality of different digitally cropped and scaled versions of the content is provided for presentation from a rewind point specified by the viewer.

5. The method of claim 1 wherein assigning an identifier to the object comprises assigning a player identifier to the object.

6. The method of claim 5 wherein providing additional information comprises providing the viewer with social media information associated with the player identifier.

7. The method of claim 5 wherein providing additional information comprises providing the viewer with a particular camera view associated with the object identifier.

8. The method of claim 7 wherein the particular camera view comprises a tracked player view in which a player corresponding to the player identifier is tracked over a plurality of frames and the camera view is cropped and scaled to increase an apparent size of the player in the tracked player view.

9. The method of claim 1 wherein the processing and assigning comprise:

obtaining image coordinates for the object in each of a plurality of frames;

determining a trajectory for the object over the frames based at least in part on the obtained image coordinates;

classifying the object in each of at least a subset of the frames based at least in part on the trajectory; and

assigning a player identifier to the classified object.

10. The method of claim 9 wherein obtaining image coordinates for the object in a given one of the frames comprises performing color blob detection based at least in part on a known team uniform color of one or more players.

11. The method of claim 1 wherein assigning an identifier to the object comprises:

receiving input from the viewer associating the identified object with a particular player; and

assigning a player identifier of the particular player to the identified object based at least in part on the received input.

12. A method comprising:

receiving content from at least one source;

processing the content to identify an object in the content;

assigning an identifier to the object based on the processing;

permitting viewer selection of the object identifier; and

providing additional information for presentation to a viewer of the content responsive to viewer selection of the object identifier;

wherein the receiving, processing, assigning, permitting and providing are implemented in at least one processing device comprising a processor coupled to a memory;

wherein the processing and assigning comprise;

obtaining image coordinates for the object in each of a plurality of frames;

determining a trajectory for the object over the frames based at least in part on the obtained image coordinates;

classifying the object in each of at least a subset of the frames based at least in part on the trajectory; and

assigning a player identifier to the classified object; and

wherein classifying the object comprises;

defining an area of a given one of the frames using the obtained image coordinates for that frame;

generating descriptors for the defined area using a dense feature set;

applying the descriptors to a Gaussian mixture model to generate Fisher vectors;

pooling the descriptors and associated Fisher vectors to form a spatial pyramid; and

utilizing the spatial pyramid to classify the object within the given frame as corresponding to the player identifier.

13. A method comprising:

receiving content from at least one source;

processing the content to identify an object in the content;

assigning an identifier to the object based on the processing;

permitting viewer selection of the object identifier; and

providing additional information for presentation to a viewer of the content responsive to viewer selection of the object identifier;

wherein the receiving, processing, assigning, permitting and providing are implemented in at least one processing device comprising a processor coupled to a memory;

wherein the processing and assigning comprise;

detecting a preassigned player indicator on a uniform of a player corresponding to an identified object;

assigning a player identifier to the identified object based at least in part on the detected preassigned player indicator; and

processing the content to substantially remove the preassigned player indicator from the content before providing the content for presentation to the viewer.

14. A method comprising:

receiving content from at least one source;

processing the content to identify an object in the content;

assigning an identifier to the object based on the processing;

permitting viewer selection of the object identifier; and

providing additional information for presentation to a viewer of the content responsive to viewer selection of the object identifier;

wherein the receiving, processing, assigning, permitting and providing are implemented in at least one processing device comprising a processor coupled to a memory;

the method further comprising;

identifying players in the content;

separating the content into a first portion comprising image portions associated with the players and a second portion comprising image portions associated with background;

applying an anti-aliasing process to the second portion; and

recombining the first and second portions;

wherein the recombined first and second portions are provided for presentation to the viewer.

15. The method of claim 14 further comprising altering one or more advertisements of the second portion prior to recombining the first and second portions.

16. An article of manufacture comprising a non-transitory computer-readable storage medium having computer program code embodied therein, wherein the computer program code when executed in at least one processing device causes said at least one processing device:

to receive content from at least one source;

to process the content to identify an object in the content;

to assign an identifier to the object based on the processing;

to permit viewer selection of the object identifier; and

to provide additional information for presentation to a viewer of the content responsive to viewer selection of the object identifier;

wherein the content comprises at least a portion of a panoramic video comprising a combination of multiple live videos from respective ones of a plurality of video sources;

wherein the computer program code when executed in said at least one processing device further causes said at least one processing device to digitally crop and scale the panoramic video before providing a resulting version of the content for presentation to the viewer;

wherein said at least one processing device is further configured to generate a plurality of different digitally cropped and scaled versions of the content from the panoramic video; and

wherein said at least one processing device is further configured to provide a first one of the plurality of different digitally cropped and scaled versions of the content for presentation to the viewer, and responsive to input from the viewer, to provide a second one of the plurality of different digitally cropped and scaled versions of the content for presentation to the viewer, wherein the second one of the plurality of different digitally cropped and scaled versions of the content is provided for presentation from a rewind point specified by the viewer.

17. An apparatus comprising:

at least one processing device comprising a processor coupled to a memory;

wherein said at least one processing device is configured to receive content from at least one source, to process the content to identify an object in the content, to assign an identifier to the object based on the processing, to permit viewer selection of the object identifier, and to provide additional information for presentation to a viewer of the content responsive to viewer selection of the object identifier;

wherein the content comprises at least a portion of a panoramic video comprising a combination of multiple live videos from respective ones of a plurality of video sources;

wherein said at least one processing device is further configured to digitally crop and scale the panoramic video before providing a resulting version of the content for presentation to the viewer;

wherein said at least one processing device is further configured to generate a plurality of different digitally cropped and scaled versions of the content from the panoramic video; and

wherein said at least one processing device is further configured to provide a first one of the plurality of different digitally cropped and scaled versions of the content for presentation to the viewer, and responsive to input from the viewer, to provide a second one of the plurality of different digitally cropped and scaled versions of the content for presentation to the viewer, wherein the second one of the plurality of different digitally cropped and scaled versions of the content is provided for presentation from a rewind point specified by the viewer.

18. The apparatus of claim 17 wherein said at least one processing device comprises at least one server.

19. A content delivery system comprising the apparatus of claim 17.

20. The apparatus of claim 17 wherein the multiple live videos correspond to respective multiple live video feeds provided by respective ones of a plurality of fixed view ultra-high resolution video cameras and wherein the panoramic video is generated by stitching together the multiple live video feeds from the respective ones of the plurality of fixed view ultra-high resolution video cameras so as to thereby combine the multiple live video feeds into a single panoramic view.