METHOD OF PROCESSING A VIEWPORT WITHIN LARGE FORMAT IMAGERY

• US 20140240345A1
• Filed: 05/08/2014
• Published: 08/28/2014
• Est. Priority Date: 09/16/2010
• Status: Active Grant

• Current Assignee: Cubic Digital Intelligence, Inc. (Cubic Corporation)
• Original Assignee: Pixia Corp. (Cubic Corporation)
• Inventors: Thakkar, Rahul C., PAKULA, Scott L., ERNST, Rudolf O.
• Granted Patent: US 9,058,642 B2
• Time in Patent Office: Days
• US Class Current: 345/619
• CPC Class Codes: G06F 16/2291 User-Defined Types; Storage management thereof
• CPC Class Codes: G06F 16/2343 Locking methods, e.g. distributed locking or locking implementation details
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06T 1/00 General purpose image data processing
• CPC Class Codes: G06T 1/60 Memory management
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: H04N 21/23424 involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/2353 specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata
• CPC Class Codes: H04N 21/2383 Channel coding or modulation of digital bit-stream, e.g. QPSK modulation arrangements for detecting or preventing errors in the information received by adapting the channel coding H04L1/0009; analogue front ends or means for connecting modulators, demodulators or transceivers to a transmission line H04L27/0002
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/2665 Gathering content from different sources, e.g. Internet and satellite
• CPC Class Codes: H04N 21/6125 involving transmission via Internet transmission by internet of broadcast information H04H60/82

Abstract

A method of processing a viewport within an image arranged as a matrix of tiles from a container file is provided. The method includes receiving a request to read data of the viewport starting from a point of origin of the viewport, the viewport having a pixel width and a pixel height, the viewport being a portion of an image that is stored in a record within the file container; computing column numbers or row numbers, or both, of the tiles containing the viewport; and reading in parallel or asynchronously each row or each column of tiles containing the viewport.

Claims

1. A method of processing a viewport within an image arranged as a matrix of tiles from a container file, the method being implemented by a computer system that includes one or more processors configured to execute processing operations, the method comprising:

receiving, by the one or more processors, a request to read data of the viewport starting from a point of origin of the viewport, the viewport having a pixel width and a pixel height, the viewport being a portion of an image that is stored in a record within the file container;

computing, by the one or more processors, column numbers or row numbers, or both, of the tiles containing the viewport; and

reading, by the one or more processors, in parallel or asynchronously each row or each column of tiles containing the viewport.

2. The method according to claim 1, further comprising reading, by the one or more processors, record metadata of the record containing the image that includes the viewport, wherein reading the record metadata includes reading a record metadata section at absolute container file offset O_rm, where O_rm=S_fcm+(S_r×

N), where S_fcm corresponds to a size of the record metadata section and S_r corresponds to a size of the record, and N is a record number.

3. The method according to claim 1, further comprising determining, by the one or more processors, whether the image containing the viewport is stored:

uncompressed, tiled arranged in a column major order, or

compressed, tiled and arranged in column major order, or

compressed, tiled and arranged in a row-major order, or

in a third party file format.

4. The method according to claim 3, wherein if the image is uncompressed, tiled arranged in a column major order,

computing top left and bottom right column numbers of the tiles containing the viewport, the viewport having a top left corner at X₀, Y₀, and having a pixel width V_w and pixel height V_h,

for each column between the top left column number and the bottom right column number, computing an absolute aligned offset from a start of the container file, and an aligned size of a strip starting from line Y₀ to and including line Y₀+V_h−

1, and

collecting and processing only pixels that reside within bounds of the viewport to generate the output viewport in a single raster.

5. The method according to claim 3, wherein if the record contains an image compressed, tiled, with the tiles arranged in row-major order,

computing top left and bottom right column and row numbers of the tiles containing the viewport, the viewport having a top left corner at X₀, Y₀, and having a pixel width V_w and pixel height V_h,

for each row between the top left row number and the bottom right row number, launching parallel or asynchronous operations to read tiles for each row.

6. The method according to claim 5, further comprising reading an entire tile strip for each row in a single read operation or reading one or more individual tiles in parallel or in asynchronous operation.

7. The method according to claim 6, wherein reading the tile strip in parallel or in asynchronous operation comprises reading starting from a storage device block size (SDBS) aligned location prior to or equal to a start of an initial tile in the tile strip and ending at a storage device block size (SDBS) aligned location after or equal to an end of a last tile in the tile strip.

8. The method according to claim 6, further comprising reading multiple individual tiles in parallel or asynchronously, wherein each tile read starts from a storage device block size (SDBS) aligned location prior to or equal to the start of a tile and ends at a storage device block size (SDBS) aligned location after or equal to the end of the same tile.

9. The method according to claim 8, further comprising collecting and processing only pixels that reside within viewport bounds to generate an output viewport in a single raster.

10. The method according to claim 3, wherein if the record contains an image stored compressed, tiled, with the tiles arranged in column-major order,

computing top left and bottom right column and row numbers of the tiles containing the viewport, the viewport having a top left corner at X₀, Y₀, and having a pixel width V_w and pixel height V_h, and

for each column, launching parallel or asynchronous operations to read tiles for each column between the top left column number and the bottom right column number.

11. The method according to claim 10, reading an entire tile strip for the column in a single read operation or reading one or more individual tiles in parallel or in asynchronous operation.

12. The method according to claim 11, wherein reading the entire tile strip comprises reading starting from a storage device block size (SDBS) aligned location prior to or equal to a start of an initial tile and ending at a storage device block size (SDBS) aligned location after or equal to an end of a last tile.

13. The method according to claim 11, further comprising reading multiple individual tiles in parallel or asynchronously, wherein each tile read starts from a storage device block size (SDBS) aligned location prior to or equal to a start of a tile and ends at a storage device block size (SDBS) aligned location after or equal to the end of the same tile.

14. The method according to claim 13, further comprising collecting and processing only pixels that reside within the viewport bounds to generate the output viewport in a single raster.

15. The method according to claim 1, wherein if the record contains an image stored in a third party tiled file format,

computing a top left and bottom right column and row numbers of the tiles containing the viewport, the viewport having a top left corner at X₀, Y₀, and having a pixel width V_w, and pixel height V_h,

determining that record metadata of the record containing the image that includes the viewport holds a table of offsets and sizes of each tile in the image that exists in a known third-party file format, and

determining whether data in the image is stored in row-major order or column-major order.

16. The method according to claim 15, wherein if the image data is stored in the column-major order,

computing a top left and bottom right column and row numbers of the tiles containing the viewport,

for each column between the top left column number and the bottom right column number, launching parallel or asynchronous operations to read tiles for each column, and

reading an entire tile strip for each column in a single read operation or reading one or more individual tiles in parallel or in asynchronous operation.

17. The method according to claim 15, if the image data is stored in row-major order,

computing a top left and bottom right column and row numbers of the tiles containing the viewport,

for each row between the top left row number and the bottom right row number, launching parallel or asynchronous operations to read tiles for each row, and

reading the entire tile strip for each row in a single read operation or reading one or more individual tiles in parallel or in asynchronous operation.

18. The method according to claim 1, wherein receiving the request comprises receiving a hypertext transfer protocol (HTTP) request.

19. The method according to claim 1, further comprising outputting the data of the viewport.

20. The method according to claim 19, wherein the outputting comprises transmitting the data of the viewport using hypertext transfer protocol (HTTP) or displaying the data of the viewport on a display device.

21. A method of processing a plurality of viewports in a plurality of images, each viewport being wholly contained within an image in the plurality of images, each image in the plurality of images being arranged as a matrix of tiles, the plurality of images being stored in a container file, the method being implemented by a computer system that includes one or more processors configured to execute processing operations, the method comprising:

receiving, by the one or more processors, a request to read data of each of the plurality of viewports starting from a point of origin of each of the plurality of viewports, each viewport having a pixel width and a pixel height, and each viewport being a portion of an image in the plurality of images, each image is stored in a record in a plurality of records within the file container;

computing, by the one or more processors, column numbers or row numbers, or both, of the tiles containing each viewport; and

reading, by the one or more processors, in parallel or asynchronously each row or each column of tiles containing each viewport.

22. The method according to claim 21, further comprising outputting each of the plurality of viewports.

23. The method according to claim 21, wherein the point of origin of each of the plurality of viewports is fixed from one viewport contained in one image to another viewport contained in another image.

24. The method according to claim 21, wherein the point of origin of each of the plurality of viewports is different from one viewport to another viewport.

25. The method according to claim 21, wherein the pixel width and the pixel height of each viewport in the plurality of viewports is the same for all the viewports.

26. The method according to claim 21, further comprising reading, by the one or more processors, record metadata of each record containing a corresponding image that contains each viewport.