Streaming video navigation systems and methods

• US 8,787,726 B2
• Filed: 02/25/2013
• Issued: 07/22/2014
• Est. Priority Date: 02/26/2012
• Status: Active Grant

• Current Assignee: Antonio Rossi
• Original Assignee: Antonio Rossi
• Inventors: Rossi, Antonio
• Primary Examiner(s): Tran, Thai
• Assistant Examiner(s): Tran, William
• Application Number:
  US13/776,609
• Publication Number:
  US 20130223812A1
• Time in Patent Office: 512 Days
• Field of Search: 386/224
• US Class Current: 386/224
• CPC Class Codes: H04N 13/315 the parallax barriers being time-variant
• CPC Class Codes: H04N 13/354 for displaying sequentially
• CPC Class Codes: H04N 13/398 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 21/21805 enabling multiple viewpoints, e.g. using a plurality of cameras
• CPC Class Codes: H04N 21/2187 Live feed
• CPC Class Codes: H04N 21/23424 involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/2393 involving handling client requests scheduling or organising the servicing of application requests H04L67/60
• CPC Class Codes: H04N 21/47202 for requesting content on demand, e.g. video on demand
• CPC Class Codes: H04N 21/47217 for controlling playback functions for recorded or on-demand content, e.g. using progress bars, mode or play-point indicators or bookmarks specific graphical features in visual interfaces H04N21/4312
• CPC Class Codes: H04N 21/6587 Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 5/765 Interface circuits between an apparatus for recording and another apparatus
• CPC Class Codes: H04N 5/772 the recording apparatus and the television camera being placed in the same enclosure
• CPC Class Codes: H04N 5/783 Adaptations for reproducing at a rate different from the recording rate
• CPC Class Codes: H04N 9/87 Regeneration of colour television signals H04N9/80 takes precedence

Abstract

The multimedia systems and methods disclosed herein permit rapid switching from one video stream to another in order to present similar images at different camera angles. The systems and methods further describe a virtual joystick and a moviola system. The virtual joystick allows a viewer to select for viewing in a main portion of a display, a sequence of images at a different camera angle than what was previously rendered on the main portion. The different camera angle is intuitively obvious based on a spatial relationship between two or more thumbnail display windows displayed in the virtual joystick. The moviola system allows rapid backwards navigation of a video stream that has already been received in a video receiver, and also allows slow motion backwards/forwards viewing starting from a paused image.

Claims

1. A multimedia system comprising:

a first video buffer configured to accept a first video stream comprising a first plurality of differentially compressed image frames and a first plurality of key frames, the first video stream generated from a first sequence of images of an event perceived at a first video camera angle;

a second video buffer configured to accept a second video stream comprising a second plurality of differentially compressed image frames and a second plurality of key frames, the second video stream generated from a second sequence of images of the event perceived at a second video camera angle, wherein the first and the second video streams are perceivably aligned in time;

a multimedia processing circuit configured to receive one of the first or the second video stream from the first or the second video buffer respectively, and transmit the received one of the first or the second video stream through a multimedia transmission link;

a communication interface configured to receive via a communications link at an arbitrary instant in time, a request to switch from the transmitted one of the first or the second video stream to the other one of the first or the second video stream, the arbitrary instant in time occurring after a first key frame has been transmitted out of the multimedia processing circuit and before a next key frame is transmitted out of the multimedia processing circuit; and

a first processor configured to respond to the request by executing steps comprising;

using a backwards seek procedure for identifying in the other one of the first or the second video stream, a second key frame that precedes the arbitrary instant in time;

using the identified second key frame to execute a forward seek procedure for identifying in the other one of the first or the second video stream, a compressed image frame corresponding to the arbitrary instant in time;

configuring the multimedia processing circuit to switch from the transmitted one of the first or the second video stream to the other one of the first or the second video stream; and

configuring the multimedia processing circuit to transmit the other one of the first or the second video stream starting from the compressed image frame corresponding to the arbitrary instant in time.

2. The system of claim 1, wherein the multimedia processing circuit is a part of a server device configured to provide at least one of the first video stream or the second video stream to one or more client devices.

3. The system of claim 2, wherein the server device is configured to provide at least one of a first or a second audio stream to the one or more client devices.

4. The system of claim 3, wherein the at least one of a first or a second audio stream is provided substantially concurrent with providing a respective one of the at least one of the first video stream or the second video stream.

5. The system of claim 4, wherein the first processor is further configured to respond to the request by executing additional steps comprising:

configuring the streaming multimedia server to continue to transmit the at least one of a first or a second audio stream without interruption, upon switching from the transmitted one of the first or the second video stream to the other one of the first or the second video stream.

6. The system of claim 1, wherein the first video buffer, the second video buffer, the multimedia processing circuit, the communication interface, and the first processor are housed in a transmitter unit that is communicatively coupled to a receiver unit, the receiver unit comprising:

a display device; and

a second processor configured to execute steps comprising;

recovering from a received one of the first or the second video streams, an uncompressed video signal;

providing the uncompressed video signal to a display driver for displaying on the display device;

detecting initiation of the request; and

transmitting the request to the transmitter unit via the communications link.

7. The system of claim 6, wherein initiation of the request is carried out via a graphical user interface displayed on the display device.

8. The system of claim 6, wherein at least a portion of the transmitter unit is integrated into the receiver unit.

9. A method comprising:

storing in a first buffer, a first video stream comprising a first plurality of differentially compressed image frames and a first plurality of key frames, the first video stream generated from a first sequence of images of an event perceived at a first video camera angle;

storing in a second buffer, a second video stream comprising a second plurality of differentially compressed image frames and a second plurality of key frames, the second video stream generated from a second sequence of images of the event perceived at a second video camera angle, wherein the first and the second video streams are perceivably aligned in time;

receiving at an arbitrary instant in time after transmission of a first key frame and during transmission of one of the compressed image frames in the first plurality of differentially compressed image frames, a request to transmit the second video stream in place of the first video stream;

using a backwards seek procedure to identify in the buffered second video stream, a second key frame that precedes the arbitrary instant in time;

using the identified second key frame to execute a forward seek for identifying in the buffered second video stream, a compressed image frame corresponding to the arbitrary instant in time; and

transmitting the second video stream starting from the compressed image frame corresponding to the arbitrary instant in time.

10. The method of claim 9, wherein the first and second buffers are contained in a part of a server device configured to provide at least one of the first video stream or the second video stream to one or more client devices.

11. The method of claim 10, further comprising:

providing from the server device, at least one of a first or a second audio stream to the one or more client devices.

12. The method of claim 11, wherein the at least one of a first or a second audio stream is provided substantially concurrent with providing a respective one of the at least one of the first video stream or the second video stream.

13. The method of claim 9, wherein the backwards seek procedure comprises ignoring a set of compressed image frames in the buffered second video stream that correspond to a set of compressed image frames in the first video stream, wherein the set of compressed image frames in the first video stream has been transmitted after transmission of the first key frame.

14. The method of claim 13, wherein using the forward seek comprises ignoring the set of compressed image frames in the buffered second video stream that correspond to the set of compressed image frames in the first video stream that has been transmitted after transmission of the first key frame.

15. The method of claim 14, wherein the first sequence of images of the real-time event is captured by orienting a first video camera at a first angle, and wherein the second sequence of images of the real-time event is captured by orienting a second video camera at a second angle that is different than the first angle.

16. The method of claim 14, wherein at least one of the first or the second sequence of images of the real-time event is a computer-generated sequence of images.

17. The method of claim 14, wherein the first sequence of images of the real-time event is captured by a first video camera, and the second sequence of images is computer-generated from the first sequence of images.

18. The method of claim 9, further comprising:

transmitting an audio signal along with the first video stream; and

continuing transmitting of the audio signal without interruption when transmitting the second video stream starting from the compressed image frame corresponding to the arbitrary instant in time.

19. A non-transitory computer-readable storage medium having stored thereon, instructions that when executed by a processor implement a method comprising the steps of:

storing in a first buffer, a first video stream comprising a first plurality of differentially compressed image frames and a first plurality of key frames, the first video stream generated from a first sequence of images of an event perceived at a first video camera angle;

storing in a second buffer, a second video stream comprising a second plurality of differentially compressed image frames and a second plurality of key frames, the second video stream generated from a second sequence of images of the event perceived at a second video camera angle, wherein the first and the second video streams are perceivably aligned in time;

detecting the receiving of a message at an arbitrary instant in time after transmission of a first key frame and during transmission of one of the compressed image frames in the first plurality of differentially compressed image frames, the message comprising a request to transmit the second video stream starting from the arbitrary instant in time;

using a backwards seek procedure to identify in the buffered second video stream, a second key frame preceding the arbitrary instant in time;

using the identified second key frame to execute a forward seek procedure to identify in the buffered second video stream, a compressed image frame that corresponds to the arbitrary instant in time; and

transmitting the second video stream starting from the compressed image frame that corresponds to the arbitrary instant in time.

20. The non-transitory computer-readable storage medium of claim 19, wherein one or more software players is used for executing one or more of the method steps.

21. The non-transitory computer-readable storage medium of claim 19, further comprising instructions for generating the second sequence of images from the first sequence of images, wherein the first sequence of images is captured by a first video camera.

22. A method comprising:

storing in a first video buffer of a multimedia transmitter, a first video stream comprising a first plurality of differentially compressed image frames and a first plurality of key frames, the first video stream generated from a first sequence of images of an event perceived at a first video camera angle;

storing in a second video buffer of the multimedia transmitter, a second video stream comprising a second plurality of differentially compressed image frames and a second plurality of key frames, the second video stream generated from a second sequence of images of the event perceived at a second video camera angle, wherein the first and the second video streams are perceivably aligned in time;

receiving in the multimedia transmitter, at an arbitrary instant in time after transmission of a first key frame and during transmission of one of the compressed image frames in the first plurality of differentially compressed image frames, a first request to replace a previously transmitted image perceived at the first video camera angle, with a corresponding image perceived at the second video camera angle, wherein the previously transmitted image has been placed in a paused state in a receiver communicatively coupled to the multimedia transmitter;

using a backwards seek procedure to identify in the buffered second video stream, a second key frame that corresponds to the transmitted first key frame;

using the identified second key frame to execute a forward seek procedure for identifying in the buffered second video stream, the corresponding image perceived at the second video camera angle; and

transmitting to the receiver, an image frame containing the corresponding image perceived at the second video camera angle.

23. The method of claim 22, further comprising:

receiving in the multi-media transmitter, following the first request, a second request to provide support for performing one of the following operations in the receiver;

a) a frame-by-frame forward-play operation starting at the corresponding image perceived at the second video camera angle, b) a frame-by-frame backwards-play operation starting at the corresponding image perceived at the second video camera angle, c) a slow-motion forward-play operation starting at the corresponding image perceived at the second video camera angle, d) a slow-motion backwards-play operation starting at the corresponding image perceived at the second video camera angle; and

in response to the second request, transmitting from the transmitter a corresponding one of;

e) a first set of image frames succeeding the corresponding image perceived at the second video camera angle, f) a second set of image frames preceding the corresponding image perceived at the second video camera angle, g) a third set of image frames succeeding the corresponding image perceived at the second video camera angle, and h) a fourth set of image frames preceding the corresponding image perceived at the second video camera angle.

24. The method of claim 23, wherein at least one of the first, second, third, or fourth set of images is transmitted from the transmitter at a first frame rate that is different than a second frame rate used for transmitting another one of the first, second, third, or fourth set of images.

25. The method of claim 22, further comprising:

transmitting an audio signal; and

continuing transmitting of the audio signal without interruption when transmitting the image frame containing the corresponding image perceived at the second video camera angle.

26. The method of claim 25, wherein the audio signal is transmitted along with the first video stream.

27. The method of claim 25, wherein the audio signal is transmitted independent of the first and the second video streams.