Multi-view scene segmentation and propagation
First Claim
1. A method for applying an effect to a multi-view video stream, the method comprising:
- in a data store, receiving the video stream;
in an input device, receiving first user input designating a boundary between a foreground region and a background region, at a different depth from the foreground region, of a reference image of the video stream;
in a processor, based on the first user input, generating a reference mask that indicates the foreground region and the background region;
in the processor, using the reference mask to generate a first non-reference mask that indicates the foreground region and the background region for a first non-reference image, different from the reference image, of the video stream;
in the processor, generating a modified video stream by using the reference mask to apply the effect to the reference image and using the first non-reference mask to apply the effect to the first non-reference image; and
in a display device, displaying the modified video stream as a virtual reality or augmented reality experience.
2 Assignments
0 Petitions
Accused Products
Abstract
A depth-based effect may be applied to a multi-view video stream to generate a modified multi-view video stream. User input may designate a boundary between a foreground region and a background region, at a different depth from the foreground region, of a reference image of the video stream. Based on the user input, a reference mask may be generated to indicate the foreground region and the background region. The reference mask may be used to generate one or more other masks that indicate the foreground and background regions for one or more different images, from different frames and/or different views from the reference image. The reference mask and other mask(s) may be used to apply the effect to the multi-view video stream to generate the modified multi-view video stream.
-
Citations
34 Claims
-
1. A method for applying an effect to a multi-view video stream, the method comprising:
-
in a data store, receiving the video stream; in an input device, receiving first user input designating a boundary between a foreground region and a background region, at a different depth from the foreground region, of a reference image of the video stream; in a processor, based on the first user input, generating a reference mask that indicates the foreground region and the background region; in the processor, using the reference mask to generate a first non-reference mask that indicates the foreground region and the background region for a first non-reference image, different from the reference image, of the video stream; in the processor, generating a modified video stream by using the reference mask to apply the effect to the reference image and using the first non-reference mask to apply the effect to the first non-reference image; and in a display device, displaying the modified video stream as a virtual reality or augmented reality experience. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14)
-
-
15. A non-transitory computer-readable medium for applying an effect to a multi-view video stream, comprising instructions stored thereon, that when executed by a processor, perform the steps of:
-
causing a data store to receive the video stream; causing an input device to receive first user input designating a boundary between a foreground region and a background region, at a different depth from the foreground region, of a reference image of the video stream; based on the first user input, generating a reference mask that indicates the foreground region and the background region; using the reference mask to generate a first non-reference mask that indicates the foreground region and the background region for a first non-reference image, different from the reference image, of the video stream; generating a modified video stream by using the reference mask to apply the effect to the reference image and using the first non-reference mask to apply the effect to the first non-reference image; and causing a display device to display the modified video stream as a virtual reality or augmented reality experience. - View Dependent Claims (16, 17, 18, 19, 20, 21, 22, 23, 24)
-
-
25. A system for applying an effect to a multi-view video stream, the system comprising:
-
a data store configured to receive the video stream; an input device configured to receive first user input designating a boundary between a foreground region and a background region, at a different depth from the foreground region, of a reference image of the video stream; a processor, communicatively coupled to the data store and the input device, configured to; based on the first user input, generate a reference mask that indicates the foreground region and the background region; use the reference mask to generate a first non-reference mask that indicates the foreground region and the background region for a first non-reference image, different from the reference image, of the video stream; and generate a modified video stream by using the reference mask to apply the effect to the reference image and using the first non-reference mask to apply the effect to the first non-reference image; and a display device configured to display the modified video stream as a virtual reality or augmented reality experience. - View Dependent Claims (26, 27, 28, 29, 30, 31, 32, 33, 34)
-
Specification