Method and computing device for performing virtual camera functions during playback of media content

• US 9,826,276 B2
• Filed: 01/15/2014
• Issued: 11/21/2017
• Est. Priority Date: 01/15/2013
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd. (Samsung Group)
• Original Assignee: Samsung Electronics Co. Ltd. (Samsung Group)
• Inventors: Guntur, Ravindra, Patil, Ravikiran, Prabhu, Mahesh Krishnananda
• Primary Examiner(s): Adams, Eileen
• Application Number:
  US14/156,353
• Publication Number:
  US 20140199043A1
• Time in Patent Office: 1,406 Days
• Field of Search: None
• CPC Class Codes: H04N 21/21805 enabling multiple viewpoints, e.g. using a plurality of cameras
• CPC Class Codes: H04N 21/4728 for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region

Abstract

A method and computing device are provided for performing virtual camera functions during playback of media content. In the method, the computing device selects a region of interest (RoI) during a playback of media content, and represents the RoI as an ensemble of disparate hypotheses. Then the computing device identifies the RoI in a frame sequence sampled according to the hypotheses, performs a virtual camera action on the identified RoI, and plays the RoI with a virtual camera.

Claims

1. A method for playing a video on a screen of a computing device, the method comprising:

detecting a gesture on the screen of the computing device for setting a region comprising at least one object included in the video that is playing on the screen of the computing device;

identifying a plurality of frame sequences sampled from the video in response to the detected gesture, wherein the plurality of frame sequences comprises the at least one object;

enlarging the at least one object in response to detecting a motion of the at least one object included in the region; and

playing the video comprising the plurality of frame sequences that includes the enlarged at least one object on the screen of the computing device.

2. The method of claim 1, wherein the motion of the at least one object is detected within the plurality of frame sequences including at least one of a fixed background or a varying background.

3. The method of claim 1, wherein detecting the gesture further comprises:

slowly playing the video when the gesture is detected; and

temporarily pausing playing the video when the motion of the at least one object included in the region is detected within the plurality of frame sequences included in the video, wherein the region is set by at least one of a spot-zooming, a pinch zooming, or a stylus pen.

4. The method of claim 1, further comprising:

representing, when the region is set in response to the detected gesture, the set region as an ensemble of disparate hypotheses;

representing the set region as independent color and local pixel variation patterns derived from multiple image channels; and

representing the region as the ensemble of disparate hypotheses by creating a representation of the region by selecting pixels from the region.

5. The method of claim 4, wherein identifying the at least one object comprises:

identifying the region in the plurality of frame sequences sampled from the video according to the disparate hypotheses;

determining a matched region by combining the disparate hypotheses in the plurality of frame sequences;

setting the determined matched region as a viewport; and

determining a position and size of the viewport, wherein the region is determined based on a divergence measurement that weighs different histogram bins and different cross bins comparisons differentially.

6. The method of claim 1, wherein playing the video comprises:

estimating a moving trajectory of the at least one object when the motion of the at least one object is detected; and

playing the video based on the estimated moving trajectory of the at least one object, wherein estimating the moving trajectory of the at least one object comprises;

determining a position and size of a new viewport according to both information associated with a position and size of a past viewport and information associated with a position and size of a current viewport;

displaying the new viewport; and

resizing and repositioning the new viewport by the estimated moving trajectory when the motion of the at least one object included in the new viewport is detected.

7. The method of claim 6, wherein resizing and repositioning includes at least one of a zooming-in, a zooming-out, or a panning.

8. The method of claim 1, wherein the plurality of frame sequences is displayed on at least one of a split-screen, a picture-in-picture, or a thumbnail view.

9. A computing device for playing a video on a screen of the computing device, the computing device comprising:

a virtual camera controller configured to render the video; and

a processor operably connected to the virtual camera controller, the processor is configured to;

detect a gesture on the screen of the computing device for setting a region comprising at least one object included in the video that is playing on the screen of the computing device;

identify a plurality of frame sequences sampled from the video in response to the detected gesture, wherein the plurality of frame sequences comprises the at least one object;

enlarge the at least one object in response to detecting a motion of the at least one object included in the region; and

play the video comprising the plurality of frame sequences that includes the enlarged at least one object on the screen of the computing device.

10. The computing device of claim 9, wherein the processor is further configured to:

slowly play the video when the gesture is detected; and

temporarily pause playing the video when the motion of the at least one object included in the region is detected within the plurality of frame sequences included in the video, wherein the region is set by at least one of a spot-zooming, a pinch-zooming, or a stylus pen.

11. The computing device of claim 9, wherein the processor is further configured to:

represent, when the region is set in response to the detected gesture, the set region as an ensemble of disparate hypotheses;

represent the set region as independent color and local pixel variation patterns derived from multiple image channels; and

represent the region as the ensemble of disparate hypotheses by creating a representation of the region by selecting pixels from the region.

12. The computing device of claim 11, wherein the processor is further configured to:

identify the region in the plurality of frame sequences sampled from the video according to the disparate hypotheses;

determine a matched region by combining the disparate hypotheses in the plurality of frame sequences;

set the determined matched region as a viewport; and

determine a position and size of the viewport, wherein the region is determined based on a divergence measurement that weighs different histogram bins and different cross bins comparisons differentially.

13. The computing device of claim 9, wherein the processor is further configured to:

estimate a moving trajectory of the at least one object when the motion of the at least one object is detected; and

play the video based on the estimated moving trajectory of the at least one object, wherein the estimation of the moving trajectory of the at least one object comprises;

determine a position and size of a new viewport according to both information associated with a position and size of a past viewport and information associated with a position and size of a current viewport;

display the new viewport; and

resize and reposition the new viewport by the estimated moving trajectory when the motion of the at least one object included in the new viewport is detected.

14. The computing device of claim 9, wherein the virtual camera controller is further configured to determine at least one of zoom-in, zoom-out, or pan the set region.

15. A non-transitory computer readable medium comprising a program code, that when executed by a processor, cause the processor to:

detect a gesture on a screen of a computing device for setting a region comprising at least one object included in a video that is playing on the screen of the computing device;

identify a plurality of frame sequences sampled from the video in response to the detected gesture, wherein the plurality of frame sequences comprises the at least one object;

enlarge the at least one object in response to detecting a motion of the at least one object included in the region; and

play the video comprising the plurality of frame sequences that includes the enlarged at least one object on the screen of the computing device.

16. The non-transitory computer readable medium of claim 15, further comprising the program code, that when executed by the processor, cause the processor to:

slowly play the video when the gesture is detected; and

temporarily pause playing the video when the motion of the at least one object included in the region is detected within the plurality of frame sequences included in the video, wherein the region is set by at least one of a spot-zooming, a pinch-zooming, or a stylus pen.

17. The non-transitory computer readable medium of claim 15, further comprising the program code, that when executed by the processor, cause the processor to:

represent, when the region is set in response to the detected gesture, the set region as an ensemble of disparate hypotheses;

represent the set region as independent color and local pixel variation patterns derived from multiple image channels; and

represent the region as the ensemble of disparate hypotheses by creating a representation of the region by selecting pixels from the region.

18. The non-transitory computer readable medium of claim 17, further comprising the program code, that when executed by the processor, cause the processor to:

identify the region in the plurality of frame sequences sampled from the video according to the disparate hypotheses;

determine a matched region by combining the disparate hypotheses in the plurality of frame sequences;

set the determined matched region as a viewport; and

determine a position and size of the viewport, wherein the region is determined based on a divergence measurement that weighs different histogram bins and different cross bins comparisons differentially.

19. The non-transitory computer readable medium of claim 15, further comprising the program code, that when executed by the processor, cause the processor to:

estimate a moving trajectory of the at least one object when the motion of the at least one object is detected;

play the video based on the estimated moving trajectory of the at least one object, wherein the estimation of the moving trajectory of the at least one object comprises;

determine a position and size of a new viewport according to both information associated with a position and size of a past viewport and information associated with a position and size of a current viewport;

display the new viewport; and

resize and reposition the new viewport by the estimated moving trajectory when the motion of the at least one object included in the new viewport is detected.

20. The non-transitory computer readable medium of claim 15, further comprising the program code, that when executed by the processor, cause the processor to determine at least one of zoom-in, zoom-out, or pan the set region.