Interactive user selected video/audio views by real time stitching and selective delivery of multiple video/audio sources

• US 20140098185A1
• Filed: 10/09/2012
• Published: 04/10/2014
• Est. Priority Date: 10/09/2012
• Status: Abandoned Application

• Current Assignee: Behnam Salemi, Shahram Davari
• Original Assignee: Behnam Salemi, Shahram Davari
• Inventors: Davari, Shahram, Salemi, Behnam
• Application Number:
  US13/573,820
• Publication Number:
  US 20140098185A1
• Time in Patent Office: Days
• US Class Current: 348/36
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 7/00 Television systems details H04N3/00, H04N5/00; methods or arrangements, for coding, decoding, compressing or decompressing digital video signals H04N19/00; selective content distribution H04N21/00
• CPC Class Codes: H04N 7/002 Special television systems not provided for by H04N7/007 - H04N7/18 still pictures via a television channel H04N1/00098

Abstract

This invention describes how a panoramic view can be created in real-time using multiple ordinary video cameras by splicing the video frames in real-time. It also describes how a subset of that panoramic view can be viewed on customer screen and how a customer can smoothly shift and scroll or zoom the customer view in real-time to view the other parts of the panoramic view using a remote control device. This invention also describes how all this can be achieved economically by using a cloud service such as assigning a Virtual Machine to each customer and using hardware acceleration engines in the data center such as high-end video cards.

Claims

1. A system capturing multiple discrete audio/video streams in a camera assembly and transmitting the said discrete audio/video streams to an audio/video server, said system comprising:

a camera assembly;

a local storage;

a audio/video encoder;

an audio/video multiplexer;

a switch/router;

a synchronization clock;

a data network; and

an audio/video server;

wherein said camera assembly comprising multiple video cameras, each producing a discrete audio/video stream;

wherein said video cameras are Standard Definition cameras or High Definition cameras;

wherein said video cameras are 2 dimensional cameras or 3 dimensional cameras;

wherein a first group of said video cameras are installed on a horizontal plane;

wherein a second group of said video cameras are installed on a vertical plane;

wherein said video cameras are synchronized via said synchronization clock;

wherein said discrete audio/video stream is in RAW format or is in encoded format;

wherein said discrete audio/video streams are multiplexed by said audio/video multiplexer to create an aggregate audio/video stream;

wherein said aggregate audio/video streams is transmitted to said audio/video server through said network via said switch/router.

2. The system as recited in claim 1, wherein said discrete audio/video streams are stored locally in said local storage.

3. The system as recited in claim 1, wherein said discrete audio/video streams are encoded by said audio/video encoder and multiplexed by said audio/video multiplexer to create said aggregate audio/video stream.

4. The system as recited in claim 1, wherein said video cameras receive commands from said data network and adjust their Contrast, Brightness and Color based on said received commands.

5. A system that combines multiple discrete audio/video streams and creates a panoramic Master view in real-time and streams a subset of the said Master view to user, said system comprising:

a computer server;

a local storage;

a audio/video decoder;

an audio/video encoder;

a audio/video de-multiplexer;

a video processing card;

a switch/router; and

a data network;

wherein said switch/router receives an aggregate audio/video stream from said data network;

wherein said audio/video de-multiplexer de-multiplexes said aggregate audio/video stream and recovers the comprising discrete audio/video streams;

wherein said audio/video decoder, decodes said discrete audio/video streams and creates RAW audio/video streams;

wherein said computer server calibrates the frames of said RAW audio/video streams in horizontal and vertical axis;

wherein said computer server splices said RAW audio/video streams to create a Master view;

wherein said computer server creates a user audio/video stream from said Master view for transmission to a user based on said user requested view;

where is said computer server changes said user audio/video stream based on commands received from said user over said data network;

wherein said computer server encodes said user audio/video stream and sends it to said user over said data network using said switch/router.

6. The system as recited in claim 5, wherein said discrete audio/video streams and/or said RAW audio/video streams and/or said user audio/video streams are stored in local storage.

7. The system as recited in claim 5, wherein said computer server uses said graphic card as hardware assist for said audio/video decoding, and said audio/video encoding.

8. The system as recited in claim 5, wherein said computer server uses said graphic card as hardware assist for said stitching operation.

9. The system as recited in claim 5, wherein said computer server calibrates said RAW audio/video streams in Contrast, Brightness and Color.

10. The system as recited in claim 5, wherein said computer server de-skews the overlap section of said RAW audio/video streams.

11. The system as recited in claim 5, wherein said computer server creates said user audio/video stream that corresponds to a wider view than the said user requested view, in order to offset the delay between said user and said computer server.

12. The system as recited in claim 5, wherein said discrete audio/video streams from 2 dimensional sources are combined to create 3 dimensional view.

13. The system as recited in claim 5, wherein said computer server sends augmented information to said user upon said user'"'"'s request.

14. The system as recited in claim 9, wherein said computer servers sends results of said calibration of said RAW audio/video streams in Contrast, Brightness and Color to source of said discrete audio/video streams.

16. The system as recited in claim 14, wherein said set-top box is a computer or laptop or tablet or smart phone.

17. The system as recited in claim 14, wherein said remote commander is a remote control with button, or a remote control with motion sensor, or a sensor that senses head, eye, hand or other body part movements to detect said user commands.

18. The system as recited in claim 14, wherein said set-top box properly crops said user audio/video stream using a cropping window and creates a smaller display format for displaying over said display.

19. The system as recited in claim 14, wherein said user can send request to said data network to display augmented information on said display.

20. The system as recited in claim 18 wherein said set-top box can adjust said cropping window in response to said user commands.

15. A system that sends a user request to change the received audio/video stream from a video server, said system comprising:

a set-top box;

a remote commander;

a display;

a data network; and

a switch/router;

wherein said set-top box receives user audio/video streams from said switch/router over said network;

wherein said set-top box displays said user audio/video stream over said display;

wherein said user sends user commands to said data network via said remote commander.