Methods and apparatus for content interaction

• US 10,182,270 B2
• Filed: 04/11/2017
• Issued: 01/15/2019
• Est. Priority Date: 09/12/2014
• Status: Active Grant

• Current Assignee: Kiswe Mobile Inc.
• Original Assignee: Kiswe Mobile Inc.
• Inventors: Kim, Jeong H., Zane, Francis X., Janiszewski, Thomas J., Sweldens, Wim, Ho, Yung-Lung, Cuypers, Tom
• Primary Examiner(s): Kumar, Pankaj
• Assistant Examiner(s): Andramuno, Franklin S
• Application Number:
  US15/484,843
• Publication Number:
  US 20170223430A1
• Time in Patent Office: 644 Days
• Field of Search: 725116, 725133
• CPC Class Codes: H04N 21/21805 enabling multiple viewpoints, e.g. using a plurality of cameras
• CPC Class Codes: H04N 21/2187 Live feed
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/26603 for automatically generating descriptors from content, e.g. when it is not made available by its provider, using content analysis techniques
• CPC Class Codes: H04N 21/2665 Gathering content from different sources, e.g. Internet and satellite
• CPC Class Codes: H04N 21/41407 embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/4622 Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet web site content organization and management for information retrieval from the Internet G06F16/958; transmission by internet of broadcast information H04H60/82; stock exchange data over packet-switching network H04L12/1804; push services including data channel over packet-switching network H04L12/1859
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 21/485 End-user interface for client configuration
• CPC Class Codes: H04N 21/4858 for modifying screen layout parameters, e.g. fonts, size of the windows
• CPC Class Codes: H04N 21/8126 involving additional data, e.g. news, sports, stocks, weather forecasts
• CPC Class Codes: H04N 21/8133 specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video program
• CPC Class Codes: H04N 21/8193 dedicated tools, e.g. video decoder software or IPMP tool
• CPC Class Codes: H04N 21/8352 involving content or source identification data, e.g. Unique Material Identifier [UMID]

Abstract

A method comprises receiving content from at least one source, processing the content to identify an object in the content, assigning an identifier to the object based on the processing, permitting viewer selection of the object identifier, and providing additional information for presentation to a viewer of the content responsive to viewer selection of the object identifier. The content in some embodiments comprises live video from at least one live video source. For example, the content may comprise at least a portion of a panoramic video comprising a combination of multiple live videos from respective ones of a plurality of video sources. The panoramic video may be digitally cropped and scaled before a resulting version of the content is provided for presentation to the viewer. A plurality of different digitally cropped and scaled versions of the content from the panoramic video can be generated and made available to the viewer.

Claims

1. A method comprising:

generating panoramic video for a live event;

performing different sets of digital cropping and scaling operations on the panoramic video to generate respective distinct videos corresponding to respective distinct views of the live event;

storing the distinct videos on one or more web servers accessible to mobile devices over a wireless network;

receiving requests from a given one of the mobile devices over the wireless network for respective particular ones of the distinct videos; and

delivering the particular ones of the distinct videos over the wireless network to the given mobile device responsive to the corresponding requests;

the method further comprising;

processing the panoramic video to identify an object in the panoramic video;

assigning an identifier to the object based on the processing; and

configuring one or more of the different sets of digital cropping and scaling operations based at least in part on tracking of the identified object in the panoramic video;

wherein the processing, assigning and configuring comprise;

detecting a preassigned object indicator corresponding to an identified object;

assigning the identifier to the identified object based at least in part on the detected preassigned object indicator; and

processing at least one of the distinct videos to substantially remove the preassigned object indicator from the distinct video before delivering the distinct video for presentation on the given mobile device; and

wherein the method is implemented in at least one processing device comprising a processor coupled to a memory.

2. The method of claim 1 wherein generating the panoramic video comprises combining multiple live videos from respective ones of a plurality of video sources.

3. The method of claim 2 wherein the plurality of video sources comprise respective fixed-view ultrahigh resolution cameras.

4. The method of claim 2 wherein the plurality of video sources are distributed throughout an arena in which the live event is taking place and collectively provide a complete view of the live event.

5. The method of claim 1 further comprising:

providing a first one of the distinct videos for presentation on the given mobile device; and

responsive to one of the requests from the given mobile device, providing a second one of the distinct videos for presentation on the given mobile device;

wherein the second one of the distinct videos is provided for presentation from a point that encompasses a specified rewind point.

6. The method of claim 1 wherein a given one of the requests includes the assigned identifier and further wherein a particular one of the distinct videos that includes the identified object is delivered in response to the given request.

7. The method of claim 6 further comprising providing additional information for presentation on the given mobile device in conjunction with the particular one of the distinct videos that includes the identified object.

8. The method of claim 1 wherein assigning an identifier to the object comprises assigning a player identifier to the object.

9. The method of claim 8 wherein a particular one of the distinct videos provides a tracked player view in which a player corresponding to the player identifier is tracked over a plurality of frames and the set of cropping and scaling operations for the particular distinct video is configured to increase an apparent size of the player in the distinct video relative to its size in the panoramic video.

10. The method of claim 1 wherein the processing, assigning and configuring comprise:

obtaining image coordinates for the object in each of a plurality of frames;

determining a trajectory for the object over the frames based at least in part on the obtained image coordinates;

classifying the object in each of at least a subset of the frames based at least in part on the trajectory; and

assigning the identifier to the classified object.

11. The method of claim 10 wherein obtaining image coordinates for the object in a given one of the frames comprises performing color blob detection based at least in part on a known color characteristic of the object.

12. A method comprising:

generating panoramic video for a live event;

performing different sets of digital cropping and scaling operations on the panoramic video to generate respective distinct videos corresponding to respective distinct views of the live event;

storing the distinct videos on one or more web servers accessible to mobile devices over a wireless network;

receiving requests from a given one of the mobile devices over the wireless network for respective particular ones of the distinct videos; and

delivering the particular ones of the distinct videos over the wireless network to the given mobile device responsive to the corresponding requests;

the method further comprising;

processing the panoramic video to identify an object in the panoramic video;

assigning an identifier to the object based on the processing; and

configuring one or more of the different sets of digital cropping and scaling operations based at least in part on tracking of the identified object in the panoramic video;

wherein the processing, assigning and configuring comprise;

obtaining image coordinates for the object in each of a plurality of frames;

determining a trajectory for the object over the frames based at least in part on the obtained image coordinates;

classifying the object in each of at least a subset of the frames based at least in part on the trajectory; and

assigning the identifier to the classified object;

wherein classifying the object comprises;

defining an area of a given one of the frames using the obtained image coordinates for that frame;

generating descriptors for the defined area using a dense feature set;

applying the descriptors to a Gaussian mixture model to generate Fisher vectors;

pooling the descriptors and associated Fisher vectors to form a spatial pyramid; and

utilizing the spatial pyramid to classify the object within the given frame as corresponding to the identifier; and

wherein the method is implemented in at least one processing device comprising a processor coupled to a memory.

13. A method comprising:

generating panoramic video for a live event

performing different sets of digital cropping and scaling operations on the panoramic video to generate respective distinct videos corresponding to respective distinct views of the live event;

storing the distinct videos on one or more web servers accessible to mobile devices over a wireless network;

receiving requests from a given one of the mobile devices over the wireless network for respective particular ones of the distinct videos; and

delivering the particular ones of the distinct videos over the wireless network to the given mobile device responsive to the corresponding requests;

the method further comprising;

processing the panoramic video to identify an object in the panoramic video;

assigning an identifier to the object based on the processing; and

configuring one or more of the different sets of digital cropping and scaling operations based at least in part on tracking of the identified object in the panoramic video;

the method further comprising;

identifying one or more foreground objects in at least one of the distinct videos;

separating the distinct video into a first portion comprising image portions associated with the one or more foreground objects and a second portion comprising image portions associated with background;

applying an anti-aliasing process to the second portion; and

recombining the first and second portions;

wherein the distinct video comprising the recombined first and second portions is delivered for presentation on the given mobile device; and

wherein the method is implemented in at least one processing device comprising a processor coupled to a memory.

14. The method of claim 13 further comprising altering one or more advertisements of the second portion prior to recombining the first and second portions.

15. An article of manufacture comprising a non-transitory computer-readable storage medium having computer program code embodied therein, wherein the computer program code when executed in at least one processing device causes said at least one processing device:

to generate panoramic video for a live event;

to perform different sets of digital cropping and scaling operations on the panoramic video to generate respective distinct videos corresponding to respective distinct views of the live event;

to store the distinct videos on one or more web servers accessible to mobile devices over a wireless network;

to receive requests from a given one of the mobile devices over the wireless network for respective particular ones of the distinct videos;

to deliver the particular ones of the distinct videos over the wireless network to the given mobile device responsive to the corresponding requests;

to process the panoramic video to identify an object in the panoramic video;

to assign an identifier to the object based on the processing; and

to configure one or more of the different sets of digital cropping and scaling operations based at least in part on tracking of the identified object in the panoramic video;

wherein the processing, assigning and configuring comprise;

detecting a preassigned object indicator corresponding to an identified object;

assigning the identifier to the identified object based at least in part on the detected preassigned object indicator; and

processing at least one of the distinct videos to substantially remove the preassigned object indicator from the distinct video before delivering the distinct video for presentation on the given mobile device.

16. An apparatus comprising:

at least one processing device comprising a processor coupled to a memory;

wherein said at least one processing device is configured;

to generate panoramic video for a live event;

to perform different sets of digital cropping and scaling operations on the panoramic video to generate respective distinct videos corresponding to respective distinct views of the live event;

to store the distinct videos on one or more web servers accessible to mobile devices over a wireless network;

to receive requests from a given one of the mobile devices over the wireless network for respective particular ones of the distinct videos;

to deliver the particular ones of the distinct videos over the wireless network to the given mobile device responsive to the corresponding requests;

to process the panoramic video to identify an object in the panoramic video;

to assign an identifier to the object based on the processing; and

to configure one or more of the different sets of digital cropping and scaling operations based at least in part on tracking of the identified object in the panoramic video;

wherein the processing, assigning and configuring comprise;

detecting a preassigned object indicator corresponding to an identified object;

assigning the identifier to the identified object based at least in part on the detected preassigned object indicator; and

processing at least one of the distinct videos to substantially remove the preassigned object indicator from the distinct video before delivering the distinct video for presentation on the given mobile device.

17. The apparatus of claim 16 wherein said at least one processing device comprises at least one server.

18. A content delivery system comprising the apparatus of claim 16.

19. The apparatus of claim 16 wherein said at least one processing device is further configured:

to obtain image coordinates for the object in each of a plurality of frames;

to determine a trajectory for the object over the frames based at least in part on the obtained image coordinates;

to classify the object in each of at least a subset of the frames based at least in part on the trajectory; and

to assign the identifier to the classified object.