METHOD AND COMPUTING DEVICE FOR PERFORMING VIRTUAL CAMERA FUNCTIONS DURING PLAYBACK OF MEDIA CONTENT

• US 20140199043A1
• Filed: 01/15/2014
• Published: 07/17/2014
• Est. Priority Date: 01/15/2013
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Guntur, Ravindra, Patil, Ravikiran, Prabhu, Mahesh Krishnananda
• Granted Patent: US 9,826,276 B2
• Time in Patent Office: Days
• US Class Current: 386/230
• CPC Class Codes: H04N 21/21805 enabling multiple viewpoints, e.g. using a plurality of cameras
• CPC Class Codes: H04N 21/4728 for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region

Abstract

A method and computing device are provided for performing virtual camera functions during playback of media content. In the method, the computing device selects a region of interest (RoI) during a playback of media content, and represents the RoI as an ensemble of disparate hypotheses. Then the computing device identifies the RoI in a frame sequence sampled according to the hypotheses, performs a virtual camera action on the identified RoI, and plays the RoI with a virtual camera.

Claims

1. A method for performing a virtual camera function, the method comprising:

selecting a region of interest (RoI) during a playback of media content;

representing the RoI as an ensemble of disparate hypotheses;

identifying the RoI in a frame sequence sampled according to the hypotheses;

performing a virtual camera action on the identified RoI; and

playing the RoI with a virtual camera.

2. The method of claim 1, wherein the RoI contains an object of interest (OoI) which is one of a fixed frame and a varying frame.

3. The method of claim 2, wherein the selecting of the RoI includes:

detecting a gesture for selecting the RoI during the playback of media content;

upon detection of the gesture, slowing down the playback of media content; and

temporarily pausing the playback of media content when the OoI contained in the RoI is moving,

wherein the RoI is selected by at least one of spot-zoom, pinch zoom, and stylus pen.

4. The method of claim 1, wherein the representing of the RoI as an ensemble of disparate hypothesis includes:

representing the selected RoI as independent color and local pixel variation patterns derived from multiple image channels; and

creating a representation of the RoI by selecting pixels from within the RoI.

5. The method of claim 1, wherein the identifying of the RoI includes:

determining a best matched RoI by combining the disparate hypotheses in the frame sequence; and

setting the determined RoI as a viewport and determining a position and size of the viewport,

wherein the RoI is determined based on a divergence measure that weighs different histogram bins and different cross bins comparisons differentially.

6. The method of claim 1, wherein the performing of the virtual camera action includes:

determining a position and size of a new viewport according to both information about a position and size of a past viewport and information about a position and size of a current viewport;

displaying the new viewport;

resizing and repositioning the new viewport by estimating trajectory when a motion of an object of interest (OoI) contained in the new viewport is detected.

7. The method of claim 6, wherein the resizing and repositioning includes zooming-in, zooming-out, and panning.

8. The method of claim 1, wherein the frame sequence is displayed in at least one of split-screen, picture-in-picture, and thumbnail view.

9. A computing device for performing a virtual camera function, the computing device comprising:

a virtual camera control module configured to render media content; and

a processor configured to select a region of interest (RoI) during a playback of media content, to represent the RoI as an ensemble of disparate hypotheses, to identify the RoI in a frame sequence sampled according to the hypotheses, to perform a virtual camera action on the identified RoI, and to play the RoI with a virtual camera.

10. The computing device of claim 9, wherein the processor is further configured to detect a gesture for selecting the RoI during the playback of media content, to slow down the playback of media content upon detection of the gesture, and to temporarily pause the playback of media content when the OoI contained in the RoI is moving.

11. The computing device of claim 9, wherein the processor is further configured to represent the selected RoI as independent color and local pixel variation patterns derived from multiple image channels, and to represent the RoI as the ensemble of disparate hypotheses by creating a representation of the RoI by selecting pixels from within the RoI.

12. The computing device of claim 9, wherein the processor is further configured to determine a best matched RoI by combining the disparate hypotheses in the frame sequence, to set the determined RoI as a viewport, to determine a position and size of the viewport, and determine the RoI based on a divergence measure that weighs different histogram bins and different cross bins comparisons differentially.

13. The computing device of claim 9, wherein the processor is further configured to determine a position and size of a new viewport according to both information about a position and size of a past viewport and information about a position and size of a current viewport, to display the new viewport, and to resize and reposition the new viewport by estimating trajectory when a motion of an object of interest (OoI) contained in the new viewport is detected.

14. The computing device of claim 9, wherein the virtual camera control module is further configured to zoom-in, zoom-out, and pan the RoI.

15. A non-transitory computer readable medium comprising a program stored thereon which causes a processor to:

select a region of interest (RoI) during a playback of media content, to represent the RoI as an ensemble of disparate hypotheses, to identify the RoI in a frame sequence sampled according to the hypotheses, to perform a virtual camera action on the identified RoI, and to play the RoI with a virtual camera.

16. The non-transitory computer readable medium of claim 15, wherein the processor is further configured to detect a gesture for selecting the RoI during the playback of media content, to slow down the playback of media content upon detection of the gesture, and to temporarily pause the playback of media content when the OoI contained in the RoI is moving.

17. The non-transitory computer readable medium of claim 15, wherein the processor is further configured to represent the selected RoI as independent color and local pixel variation patterns derived from multiple image channels, and to represent the RoI as the ensemble of disparate hypotheses by creating a representation of the RoI by selecting pixels from within the RoI.

18. The non-transitory computer readable medium of claim 15, wherein the processor is further configured to determine a best matched RoI by combining the disparate hypotheses in the frame sequence, to set the determined RoI as a viewport, to determine a position and size of the viewport, and determine the RoI based on a divergence measure that weighs different histogram bins and different cross bins comparisons differentially.

19. The non-transitory computer readable medium of claim 15, wherein the processor is further configured to determine a position and size of a new viewport according to both information about a position and size of a past viewport and information about a position and size of a current viewport, to display the new viewport, and to resize and reposition the new viewport by estimating trajectory when a motion of an object of interest (OoI) contained in the new viewport is detected.

20. The non-transitory computer readable medium of claim 15, wherein the processor is further configured to zoom-in, zoom-out, and pan the RoI.