Streaming panoramic video

• US 20020021353A1
• Filed: 06/08/2001
• Published: 02/21/2002
• Est. Priority Date: 06/09/2000
• Status: Abandoned Application

• Current Assignee: iMove Inc. (Digital Domain Holdings Ltd.)
• Original Assignee: iMove Inc. (Digital Domain Holdings Ltd.)
• Inventors: DeNies, Mark
• Application Number:
  US09/877,166
• Publication Number:
  US 20020021353A1
• Time in Patent Office: Days
• US Class Current: 348/36
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04N 19/23 with coding of regions that are present throughout a whole video segment, e.g. sprites, background or mosaic
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 21/21805 enabling multiple viewpoints, e.g. using a plurality of cameras
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/44224 Monitoring of user activity on external systems, e.g. Internet browsing
• CPC Class Codes: H04N 21/4621 Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen arrangements for using the results of monitoring on user's side in broadcast systems H04H60/65; flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/4728 for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
• CPC Class Codes: H04N 21/643 Communication protocols network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP], H04L65/65
• CPC Class Codes: H04N 21/845 Structuring of content, e.g. decomposing content into time segments

Abstract

Streaming panoramic images from a server to a client. The system utilizes a special program at the client and a special program in the server. The special program at the client communicates with the special program at the server to direct which portion of the panorama should be streamed to the client. The special program at the client has the ability to accept data that represents a portion of a series of panoramic frames, to decompress the data, to select the data that constitutes an appropriate view window and to render a portion of each frame on a screen or display. The special program at the server selects particular slices that constitute a region of interest in the panorama and these slices are sent to the client. When the location of the view window is changed by more than a threshold amount, the client sends a command back to the web server to adjusts the selection of the slices that are streamed from the server to the client.

Claims

1. A method of streaming a panorama from a server to a client, wherein a user can only see the portion of the panorama in a view window and the user can move the location of the view window in the panorama, said method comprising the steps of

dividing the panorama into slices,

transmitting from the server to the client slices of said panorama that contain the view window plus a guard band surrounding the view window,

transmitting from the client to the server instructions to change the location of said guard band as said user moves said view window.

2. The method recited in claim 1 wherein said slices are the slices defined in the MPEG standard.

3. The method recited in claim 1 where the streaming from the server to the client is handled by a streaming server and a plug-in to said server provides the slices in said guard band.

4. The method recited in claim 1 where the client and the server are located on the same physical machine.

5. The method of streaming data relative to a series of panoramic images from a server to a client, whereby a view window of said client can be displayed to a user, said method comprising the steps of:

dividing each of said panoramic images into areas,

streaming a plurality of said areas from each area from said server to said client, said plurality of areas including said view window and a guard band around said view window,

displaying said view window portion of said panorama at said client,

accepting user directions to change the location of said view window,

sending commands to said server to change said plurality of areas being streamed to said server when said view window is changed more than a threshold amount, and

changing the areas streamed from said server to said client in response to said commands.

6. The method recited in claim 5 wherein said areas are MPEG slices.

7. The method recited in claim 5 wherein said server is a Real Networks server.

8. The method recited in claim 5 wherein said panorama is displayed to said user in a perspectively correct manner.

9. The method recited in claim 5 wherein said server simultaneously streams portions of different panoramas to different clients.

10. The method recited in claim 5 wherein said server and said client are on the same physical machine.

11. A system for transmitting panoramic images from a server to a client,

means at said server for dividing each panorama into areas, a plurality of said areas forming a region of interest of said panorama, said region of interest including a view window and a guard band around said view window,

means for transmitting a region of interest from each panorama in a series of panoramas from said server to said client,

means at said client for moving the location of said view window in said panorama,

means for transmitting from said client to said server commands to change the location of said region of interest, and

means at said server for changing the location of said region of interest which is streamed to said client.

12. The system recited in claim 11 where each of said areas comprise a plurality of MPEG slices.

13. The system recited in claim 12 wherein said server simultaneously streams portions of different panoramas to different clients.

14. The system recited in claim 12 wherein said server and said client are on the same physical machine.

15. The system recited in claim 12 wherein all said means are physically located on one physical system.

17. The system recited in claim 16 including a user input device whereby said user can move said view window in said area of interest, and a communication path from said client to said server whereby said client can instruct said server to change the location of said view window.

18. The system recited in claim 17 wherein said user input device is a computer mouse.

19. The system recited in claim 15 wherein said panoramic images are stored at said server using MPEG compression forming “

I” and

“

B”

or “

P”

frames.

20. The system recited in claim 19 wherein the region of interest from “

I”

frames and entire “

B”

or “

P”

frames are transmitted from said server to said client.

16. A system for allowing a series of panoramic images stored at a server to be viewed by a user at a client, said system including,

a streaming server at said server for streaming data to said client,

a program at said server for providing said streaming server with an area of interest from each panorama to be streamed to said client, said area of interest including a view window and a guard band around said view window, and

a program at said client for receiving said data and for selecting the data representing said view window and for displaying said view window to said user.