Method of accessing a spatio-temporal part of a video sequence of images

• US 9,258,622 B2
• Filed: 04/26/2011
• Issued: 02/09/2016
• Est. Priority Date: 04/28/2010
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Youenn, Fablet, Franck, Denoual
• Primary Examiner(s): Czekaj, Dave
• Assistant Examiner(s): Werner, David N
• Application Number:
  US13/094,778
• Publication Number:
  US 20110305278A1
• Time in Patent Office: 1,750 Days
• US Class Current: 1/1
• CPC Class Codes: H04N 19/167 Position within a video image, e.g. region of interest [ROI]
• CPC Class Codes: H04N 21/23412 for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/2343 involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/234318 by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/234345 the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment
• CPC Class Codes: H04N 21/234363 by altering the spatial resolution, e.g. for clients with a lower screen resolution
• CPC Class Codes: H04N 21/2387 Stream processing in response to a playback request from an end-user, e.g. for trick-play
• CPC Class Codes: H04N 21/2668 Creating a channel for a dedicated end-user group, e.g. insertion of targeted commercials based on end-user profiles information retrieval from the Internet by querying with filtering and personalisation G06F16/9535; arrangements for replacing or switching information during the broadcast H04H20/10; push services over packet-switching network H04L12/1859; adaptation of message content in packet-switching networks H04L51/063
• CPC Class Codes: H04N 21/45455 applied to a region of the image
• CPC Class Codes: H04N 21/4728 for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
• CPC Class Codes: H04N 21/6373 for rate control , e.g. request to the server to modify its transmission rate flow control in packet networks H04L47/10
• CPC Class Codes: H04N 21/6375 for requesting retransmission , e.g. of data packets lost or corrupted during transmission from serverARQ protocols H04L1/18; implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP] H04L69/16
• CPC Class Codes: H04N 21/6581 Reference data, e.g. a movie identifier for ordering a movie or a product identifier in a home shopping application
• CPC Class Codes: H04N 7/17336 Handling of requests in head-ends

Abstract

The invention concerns a method and a device for accessing a spatio-temporal part of a video sequence of images compressed using a coding format implementing a prediction.

The method comprises the steps of:

• • defining a target spatial region corresponding to said spatio-temporal part to access;
  • obtaining compressed data corresponding to a selection spatial region in at least one image of the video sequence, said selection spatial region including said target spatial region;
  • obtaining at least one item of updating information for updating the selection spatial region that is provided for selecting compressed data in a following image of the video sequence, said updating information depending on the decoding of the compressed data corresponding to said target spatial region.

Claims

1. A method of accessing a spatio-temporal part of a video sequence of images compressed using a coding format implementing a spatial-unit-based prediction, the method comprising, at a client terminal of a communication network, the steps of:

defining a target spatial region corresponding to said spatio-temporal part to access;

obtaining compressed data corresponding to a selection spatial region in at least one first image of the video sequence, said selection spatial region including said target spatial region;

decoding the compressed data corresponding to the selection spatial region in the first image to obtain a decoded part of the first image corresponding to the target spatial region in the first image;

identifying, based on said decoding step, spatial unit or units missing in the compressed data to decode the whole target spatial region in the first image, and projecting the location of the missing spatial unit or units on the same image;

identifying, based on said decoding step, non-useful spatial unit or units of the decompressed data that are not used to obtain the decoded part corresponding to the target spatial region in the first image;

generating, based on the identified missing and non-useful spatial unit or units, at least one item of updating information for updating the selection spatial region before it is used for at least one next image of the video sequence, the item of updating information being configured to add the projected missing spatial unit or units to the selection spatial region and to delete the identified non-useful spatial unit or units from the selection spatial region;

transmitting the item of updating information to a remote server storing the compressed video sequence, for the client terminal to obtain next compressed data corresponding to the updated selection spatial region in the next image of the video sequence.

2. A method according to claim 1, comprising the steps of:

decoding said obtained compressed data which correspond to the target spatial region; and

classifying spatial units of images of the video sequence, according their utility on said decoding, so as to obtain an item of updating information on the basis of said classifying.

3. A method according to claim 2, wherein a spatial unit is useful on decoding compressed data corresponding to the target spatial region, when said spatial unit is included in the target spatial region or when said spatial unit comprises reference data used in the predictive coding of said compressed data.

4. A method according to claim 2, wherein said classifying creates a group of selected spatial units which are useful for the decoding of said compressed data corresponding to the target spatial region;

a group of selected spatial units which are not useful for the decoding of said compressed data corresponding to the target spatial region; and

a group of non-selected spatial units which are useful for the decoding of said compressed data corresponding to the target spatial region.

5. A method according to claim 2, wherein said selection spatial region is defined by a set of sides, and the classifying of a spatial unit leads to the updating of at least one parameter for moving a side of the selection spatial region.

6. A method according to claim 5, wherein said updating information comprises the parameters for movement relative to each side of the selection spatial region.

7. A method according to claim 1, comprising the steps of:

storing the obtained compressed data;

identifying spatial units that are non-selected and useful for the decoding of said compressed data corresponding to the target spatial region, so as to form said transmitted updating information;

providing corrections to said decoding of the compressed data corresponding to the target spatial region to compensate for the absence of the identified spatial units in order to display a decoded version of said compressed data;

requesting and receiving, from said remote server, said identified spatial units; and

updating the stored compressed data using said received identified spatial units.

8. A method according to claim 1, in which said updating information depends on at least one spatial unit that is not selected and is useful for the decoding of said compressed data corresponding to the target spatial region.

9. A method according to claim 1, in which said updating information depends on at least one spatial unit that is selected and not useful for the decoding of said compressed data corresponding to the target spatial region.

10. A method according to claim 1, wherein said updating information depends on an average estimation of movement of the target spatial region between two images.

11. The method according to claim 1, wherein the updating information depends on a spatial unit comprising reference data used in predictive coding of the compressed data.

12. A device for accessing a spatio-temporal part of a video sequence of image compressed using a coding format implementing a spatial-unit-based prediction, comprising, at a client terminal of a communication network, the steps of:

a means for defining a target spatial region corresponding to said spatio-temporal part to access;

a means for obtaining compressed data corresponding to a selection spatial region in at least one first image of the video sequence, said selection spatial region including said target spatial region;

a means for decoding the compressed data corresponding to the selection spatial region in the first image to obtain a decoded part of the first image corresponding to the target spatial region in the first image;

a means for identifying, based on said decoding step, spatial unit or units missing in the compressed data to decode the whole target spatial region in the first image, and projecting the location of the missing spatial unit or units on the same image;

a means for identifying, based on said decoding step, non-useful spatial unit or units of the compressed data that are not used to obtain the decoded part corresponding to the target spatial region in the first image;

a means for generating, based on the identified missing and non-useful spatial unit or units, at least one item of updating information for updating the selection spatial region before it is used for at least one next image of the video sequence, the item of updating information being configured to add the projected missing spatial unit or units to the selection spatial region and to delete the identified non-useful spatial unit or units from the selection spatial region;

a means for transmitting the item of updating information to a remote server storing the compressed video sequence, for the client terminal to obtain next compressed data corresponding to the updated selection spatial region in the next image of the video sequence.

13. A non-transitory storage medium storing a computer-executable program causing a computer to implement a method of accessing a spatio-temporal part of a video sequence of images compressed using a coding format implementing a spatial-unit-based prediction, the method comprising, at a client terminal of a communication network, the steps of:

a step of defining a target spatial region corresponding to said spatio-temporal part to access, and further comprising the steps of;

obtaining compressed data corresponding to a selection spatial region in at least one first image of the video sequence, said selection spatial region including said target spatial region;

decoding the compressed data corresponding to the selection spatial region in the first image to obtain a decoded part of the first image corresponding to the target spatial region in the first image;

identifying, based on said decoding step, spatial unit or units missing in the compressed data to decode the whole target spatial region in the first image, and projecting the location of the missing spatial unit or units on the same image;

identifying, based on said decoding step, non-useful spatial unit or units of the compressed data that are not used to obtain the decoded part corresponding to the target spatial region in the first image;

generating, based on the identified missing and non-useful spatial unit or units, at least one item of updating information for updating the selection spatial region before it is used for at least one next image of the video sequence, the item of updating information being configured to add the projected missing spatial unit or units to the selection spatial region and to delete the identified non-useful spatial unit or units from the selection spatial region;

transmitting the item of updating information to a remote server storing the compressed video sequence, for the client terminal to obtain next compressed data corresponding to the updated selection spatial region in the next image of the video sequence.

14. A method of accessing a spatio-temporal part of a video sequence of images compressed using a coding format implementing a spatial-unit based prediction, the method comprising, at a server device of a communication network, the steps of:

receiving, from a remote client terminal, a request for access to said spatio-temporal part;

defining a target spatial region corresponding to said spatio-temporal part identified in said request;

obtaining compressed data corresponding to a selection spatial region in at least one first image of the video sequence, said selection spatial region including said target spatial region;

sending the obtained compressed data to the remote client terminal;

receiving, from the remote client terminal, at least one item of updating information for updating the selection spatial region based on the decoding of the compressed data so obtained which correspond to said target spatial region;

identifying, based on the item of updating information, spatial unit or units missing in the sent compressed data to decode the whole target spatial region in the first image, and projecting the location of the missing spatial unit or units on the same image;

identifying, based on the item of updating information, non-useful spatial unit or units of the compressed data that are not used to obtain the decoded part corresponding to the target spatial region in the first image;

updating the selection spatial region before it is used for at least one next image of the video sequence, where the updating adds the projected missing spatial unit or units to the selection spatial region and deletes the identified non-useful spatial unit or units from the selection spatial region;

obtaining next compressed data corresponding to the updated selection spatial region in the next image of the video sequence; and

sending the obtained next compressed data to the remote client terminal.

15. A method according to claim 14, comprising a step of storing, by the server device, a processing history of a plurality of access requests, said updating of the selection spatial region for a later access request depending on said history.

16. A method according to claim 14, wherein said obtainment of the compressed data comprises selecting compressed data corresponding to a group of several images, and the updating information depends on the decoding of the compressed data obtained in those several images.

17. A method according to claim 16, wherein the selection of the compressed data corresponding to an image of said group of images comprises the steps of:

adjusting the dimensions of the selection spatial region depending on the type of image concerned; and

selecting the compressed data corresponding to said image, which are included in said adjusted selection spatial region.

18. A method according to claim 14, in which said updating information depends on at least one spatial unit that is not selected and is useful for the decoding of said compressed data corresponding to the target spatial region.

19. A method according to claim 14, in which said updating information depends on at least one spatial unit that is selected and not useful for the decoding of said compressed data corresponding to the target spatial region.

20. A method according to claim 14, comprising a step of updating said selection spatial region, said updating consisting in identifying a spatial region encompassing a set of non-selected spatial units which are useful for the coding of said compressed data corresponding to the target spatial region.

21. A method according to claim 14, comprising a step of updating said selection spatial region, said updating comprising a step of reducing said selection spatial region on the basis of an item of congestion information relative to a communication network on which said obtained compressed data are transmitted.

22. A method according to claim 14, wherein said updating information depends on an average estimation of movement of the target spatial region between two images.

23. The method according to claim 14, wherein the updating information depends on a spatial unit comprising reference data used in predictive coding of the compressed data.