System and Method for Client Services for Interactive Multi-View Video

• US 20060023066A1
• Filed: 08/19/2004
• Published: 02/02/2006
• Est. Priority Date: 07/27/2004
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Li, Jiang, Lou, Jianguang, Xu, Yihua, Cai, Hua
• Application Number:
  US10/923,107
• Publication Number:
  US 20060023066A1
• Time in Patent Office: Days
• US Class Current: 348/159
• CPC Class Codes: H04N 21/21805 enabling multiple viewpoints, e.g. using a plurality of cameras
• CPC Class Codes: H04N 21/2353 specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata
• CPC Class Codes: H04N 21/2365 Multiplexing of several video streams
• CPC Class Codes: H04N 21/4334 Recording operations recording of a television signal H04N5/76; arrangements for recording or accumulating broadcast information or broadcast-related information H04H60/27
• CPC Class Codes: H04N 21/4347 Demultiplexing of several video streams
• CPC Class Codes: H04N 21/47202 for requesting content on demand, e.g. video on demand
• CPC Class Codes: H04N 21/4821 using a grid, e.g. sorted out by channel and broadcast time
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

Interactive multi-view video presents new types of video capture systems, video formats, video compression algorithms, and services. Many video cameras are allocated to capture an event from various related locations and directions. The captured videos are compressed and are sent to a server in real-time. Users can subscribe to a new type of service that allows users to connect to the servers and receive multi-view videos interactively. In addition to conventional playing controls, users can manipulate controls to the camera positions and orientations, select viewing directions and enjoy special effects such as a sweeping effect and a freeze-and-rotate effect, and so on. Interactive multi-view video provides a brand-new experience in watching events.

Claims

18. A computer-implemented process for viewing video by a user, comprising the process actions of:

capturing different streams of images of the same event space from different view points simultaneously using two or more cameras;

receiving user-requested ones of the captured different streams of images at a computer for a user to view.

19. The computer-implemented process of claim 18 wherein the user-requested ones of the captured different streams of images are provided to the computer in real-time as the images are captured.

20. The computer-implemented process of claim 18 wherein captured different streams of images are stored to a storage medium and the user-requested ones of the captured different streams of images are provided to the computer from the storage medium.

21. The computer-implemented process of claim 20 wherein the captured different streams of images are stored on a server.

22. The computer-implemented process of claim 21 wherein the captured different streams of images are stored on the user'"'"'s computer.

24. The computer-implemented process of claim 18 wherein the user can control the viewpoint from which the different streams of images are captured.

25. A system for providing video for viewing by a user, comprising:

a general purpose computing device;

a computer program comprising program modules executable by the general purpose computing device, wherein the computing device is directed by the program modules of the computer program to,

input multiple video streams of the same event captured by multiple cameras simultaneously from different viewpoints,

provide user-requested ones of the input multiple video streams from a server to a client for a user at the client to view.

1. A computer-implemented process for providing video viewing services to a user, comprising the process actions of:

capturing different streams of images of the same event from different views simultaneously using two or more cameras;

a client interactively requesting specific streams of the captured images; and

providing the specific requested streams of the captured images from a server to said client.

2. The computer-implemented process of claim 1 wherein the specific streams of captured images are provided in real-time.

3. The computer-implemented process of claim 2 wherein the client can control the view point of at least one camera in real-time.

4. The computer-implemented process of claim 3 wherein the client can control the pan-tilt value of at least one camera.

5. The computer-implemented process of claim 1 wherein the captured streams of video are stored for later viewing.

6. The computer-implemented process of claim 5 wherein the user can switch from viewing the captured video stream from one camera viewpoint to another captured video stream from another video viewpoint as the video stream continues in time by accessing the video streams from different cameras that provide the desired point of view.

7. The computer-implemented process of claim 5 wherein the user can sweep through viewing the captured video stream from adjacent camera viewpoints as the video stream continues in time by accessing the video streams from adjacent cameras that provide the desired point of view.

8. The computer-implemented process of claim 5 wherein time is frozen and the camera view point rotates about a given point.

9. The computer-implemented process of claim 5 wherein the user can play back a previously viewed or created video sequence.

10. The computer-implemented process of claim 5 wherein a user can create a script of a set of views and special effects that can be played on demand.

11. The computer-implemented process of claim 10 wherein the user can send the script to other users who can, when the script is activated, observe the same scripted video events.

12. The computer-implemented process of claim 1 wherein a server generates a video stream from the captured videos and sends it to the client based on the client'"'"'s request.

13. The computer-implemented process of claim 12 wherein there are two communication channels for one client.

14. The computer-implemented process of claim 13 wherein one communication channel is used to transmit audio/video data to reduce latency.

15. The computer-implemented process of claim 14 wherein the communication channel is a User Datagram Protocol (UDP) channel.

16. The computer-implemented process of claim 13 wherein one communication channel is used to transmit command and control data.

17. The computer-implemented process of claim 16 wherein the communication channel is a Transmission Control Protocol (TCP) channel.

26. The system of claim 25 where user-requested one of the multiple video streams are provided to the client in real-time.

27. The system of claim 25 wherein the user is sent a video stream captured from one camera viewpoint switching to at least one video stream captured from another camera viewpoint as the video streams continue in time.

26-1. The system of claim 25 wherein the user is sent video streams from adjacent camera viewpoints as the video stream continues in time.

27-2. The system of claim 25 wherein time is frozen and the user is sent video streams from a camera view point that rotates about a given point.