Optimized image delivery over limited bandwidth communication channels
First Claim
1. A method of communicating images for display on a wireless mobile device, the method being performed by one or more servers, the method comprising steps of:
- employing a predetermined pyramid grid structure arranged in a hierarchy comprising a plurality of levels such that each level of the hierarchy includes a regular grid, the regular grid at each lower level of the hierarchy having more elements than at a higher level in the hierarchy, wherein each element of each regular grid defines a discrete spatial area, and the discrete spatial areas at any given level cover, in a substantially continuous and non-overlapping manner, an area that is substantially the same across the plurality of levels;
providing client software to the wireless mobile device;
using data of a source image processed with respect to the plurality of levels to obtain a series of related images of progressively lower detail level, wherein each related image of the series of related images comprises image data and is subdivided into a regular array of substantially non-overlapping image parcels, wherein image parcels of each regular array of the image parcels form a plurality of discrete portions of a related image of the series of related images according to discrete spatial areas associated with a corresponding level of the predetermined pyramid grid structure;
receiving a first request at the one or more servers from the wireless mobile device over a network communication channel, the first request being specifically for a first image parcel of the series of related images, the first image parcel corresponding to a first grid element of the predetermined pyramid grid structure, the first grid element corresponding to a first discrete spatial area;
sending the first image parcel from the one or more servers to the wireless mobile device over the network communication channel, in response to the first request;
receiving a second request at the one or more servers from the wireless mobile device over the network communication channel, the second request being specifically for a second image parcel of the series of related images, the second image parcel corresponding to a second grid element of the predetermined pyramid grid structure, the second grid element corresponding to a second discrete spatial area, the step of receiving the second request being performed after the step of receiving the first request, wherein a first non-empty overlap area exists between the first discrete spatial area and the second discrete spatial area; and
sending the second image parcel from the one or more servers to the wireless mobile device over the network communication channel, in response to the second request;
wherein the client software comprises instructions to configure the wireless mobile device to;
employ the predetermined pyramid grid structure;
request and receive image parcels through a local embedded server, the local embedded server being configured to use a local image repository to store image parcels using a local image repository, the local embedded server providing access to at least some image parcels;
determine a first user-controlled viewing frustum based on a navigational input to the wireless mobile device;
select the first image parcel of the series of related images based on the predetermined pyramid grid structure and the first user-controlled viewing frustum, and send the first request, automatically without specific user input, in response to the navigational input;
select the second image parcel of the series of related images based on the predetermined pyramid grid structure and the first user-controlled viewing frustum, and send the second request, automatically without specific user input, in response to the navigational input;
the first request and the second request being sent according to a priority order based at least in part on corresponding levels in the predetermined pyramid grid structure, the first image parcel having an earlier priority than the second image parcel according to the priority order;
automatically, without further user input after the navigational input, render a view for the first user-controlled viewing frustum comprising rendering at least a portion of the first image parcel before finishing receiving and rendering the second image parcel, to thereby enable a regional enhancement of level of details for the first user-controlled viewing frustum in at least the first non-empty overlap area; and
store the first image parcel and the second image parcel received by the wireless mobile device.
0 Assignments
0 Petitions
Accused Products
Abstract
Large-scale images are retrieved over network communications channels for display on a client device by selecting an update image parcel relative to an operator controlled image viewpoint to display via the client device. A request is prepared for the update image parcel and associated with a request queue for subsequent issuance over a communications channel. The update image parcel is received from the communications channel and displayed as a discrete portion of the predetermined image. The update image parcel optimally has a fixed pixel array size, is received in a single and or plurality of network data packets, and were the fixed pixel array may be constrained to a resolution less than or equal to the resolution of the client device display.
34 Citations
49 Claims
-
1. A method of communicating images for display on a wireless mobile device, the method being performed by one or more servers, the method comprising steps of:
-
employing a predetermined pyramid grid structure arranged in a hierarchy comprising a plurality of levels such that each level of the hierarchy includes a regular grid, the regular grid at each lower level of the hierarchy having more elements than at a higher level in the hierarchy, wherein each element of each regular grid defines a discrete spatial area, and the discrete spatial areas at any given level cover, in a substantially continuous and non-overlapping manner, an area that is substantially the same across the plurality of levels; providing client software to the wireless mobile device; using data of a source image processed with respect to the plurality of levels to obtain a series of related images of progressively lower detail level, wherein each related image of the series of related images comprises image data and is subdivided into a regular array of substantially non-overlapping image parcels, wherein image parcels of each regular array of the image parcels form a plurality of discrete portions of a related image of the series of related images according to discrete spatial areas associated with a corresponding level of the predetermined pyramid grid structure; receiving a first request at the one or more servers from the wireless mobile device over a network communication channel, the first request being specifically for a first image parcel of the series of related images, the first image parcel corresponding to a first grid element of the predetermined pyramid grid structure, the first grid element corresponding to a first discrete spatial area; sending the first image parcel from the one or more servers to the wireless mobile device over the network communication channel, in response to the first request; receiving a second request at the one or more servers from the wireless mobile device over the network communication channel, the second request being specifically for a second image parcel of the series of related images, the second image parcel corresponding to a second grid element of the predetermined pyramid grid structure, the second grid element corresponding to a second discrete spatial area, the step of receiving the second request being performed after the step of receiving the first request, wherein a first non-empty overlap area exists between the first discrete spatial area and the second discrete spatial area; and sending the second image parcel from the one or more servers to the wireless mobile device over the network communication channel, in response to the second request; wherein the client software comprises instructions to configure the wireless mobile device to; employ the predetermined pyramid grid structure; request and receive image parcels through a local embedded server, the local embedded server being configured to use a local image repository to store image parcels using a local image repository, the local embedded server providing access to at least some image parcels; determine a first user-controlled viewing frustum based on a navigational input to the wireless mobile device; select the first image parcel of the series of related images based on the predetermined pyramid grid structure and the first user-controlled viewing frustum, and send the first request, automatically without specific user input, in response to the navigational input; select the second image parcel of the series of related images based on the predetermined pyramid grid structure and the first user-controlled viewing frustum, and send the second request, automatically without specific user input, in response to the navigational input; the first request and the second request being sent according to a priority order based at least in part on corresponding levels in the predetermined pyramid grid structure, the first image parcel having an earlier priority than the second image parcel according to the priority order; automatically, without further user input after the navigational input, render a view for the first user-controlled viewing frustum comprising rendering at least a portion of the first image parcel before finishing receiving and rendering the second image parcel, to thereby enable a regional enhancement of level of details for the first user-controlled viewing frustum in at least the first non-empty overlap area; and store the first image parcel and the second image parcel received by the wireless mobile device. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
-
-
13. A computing system comprising one or more servers, wherein:
-
the one or more servers provide client software for a wireless mobile device; the computing system and the wireless mobile device both use a substantially same predetermined pyramid grid structure arranged in a hierarchy comprising a plurality of levels such that each level of the hierarchy includes a regular grid, the regular grid at each lower level of the hierarchy having more elements than at a higher level in the hierarchy, wherein each element of each regular grid defines a discrete spatial area, and the discrete spatial areas at any given level cover, in a substantially continuous and non-overlapping manner, an area that is substantially the same across the plurality of levels; the one or more servers are coupled to the wireless mobile device by a network communication channel; the one or more servers are configured to; use data of a source image processed with respect to the plurality of levels to obtain a series of related images of progressively lower detail level, wherein each related image of the series of related images comprises image data and is subdivided into a regular array of substantially non-overlapping image parcels, each image parcel of each regular array of the image parcels forming a plurality of discrete portions of a related image of the series of related images according to discrete spatial areas associated with a corresponding level of the predetermined pyramid grid structure; receive a first request at the one or more servers from the wireless mobile device over the network communication channel, the first request being specifically for a first image parcel of the series of related images, the first image parcel corresponding to a first grid element of the predetermined pyramid grid structure, the first grid element corresponding to a first discrete spatial area; send the first image parcel from the one or more servers to the wireless mobile device over the network communication channel, in response to the first request; receive a second request at the one or more servers from the wireless mobile device over the network communication channel, the second request being specifically for a second image parcel of the series of related images, the second image parcel corresponding to a second grid element of the predetermined pyramid grid structure, the second grid element corresponding to a second discrete spatial area, the second request being received by the one or more servers after the first request, wherein a first non-empty overlap area exists between the first discrete spatial area and the second discrete spatial area; and send the second image parcel from the one or more servers to the wireless mobile device over the network communication channel, in response to the second request; wherein the client software comprises instructions to configure the wireless mobile device to; request and receive image parcels through a local embedded server, the local embedded server being configured to use a local image repository to store image parcels, the local embedded server providing access to at least some image parcels, the first request and the second request being sent to the one or more servers according to a priority order based at least in part on corresponding levels of the first and second grid elements in the predetermined pyramid grid structure, the first image parcel having an earlier priority than the second image parcel according to the priority order; determine a first user-controlled viewing frustum based on a navigational input to the wireless mobile device; select the first image parcel of the series of related images based on the predetermined pyramid grid structure and the first user-controlled viewing frustum and send the first request, automatically without specific user input, in response to the navigational input; select the second image parcel of the series of related images based on the predetermined pyramid grid structure and the first user-controlled viewing frustum and send the second request, automatically without specific user input, in response to the navigational input; automatically, without further user input after the navigational input, render a view for the first user-controlled viewing frustum comprising rendering at least a portion of the first image parcel before finishing receiving and rendering the second image parcel to thereby enable a regional enhancement of level of details for the first user-controlled viewing frustum in at least the first non-empty overlap area; and store the first image parcel and the second image parcel received by the wireless mobile device. - View Dependent Claims (14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24)
-
-
25. A method of receiving images for display by a wireless mobile device configured by client software, the method comprising the following steps performed by the wireless mobile device:
-
employing a predetermined pyramid grid structure arranged in a hierarchy comprising a plurality of levels such that each level of the hierarchy includes a regular grid, the regular grid at each lower level of the hierarchy having more elements than at a higher level in the hierarchy, wherein each element of each regular grid of the predetermined pyramid grid structure defines a discrete spatial area, and the discrete spatial areas at any given level cover, in a substantially continuous and non-overlapping manner, an area that is substantially the same across the plurality of levels; determining a first user-controlled viewing frustum based on a navigational input to the wireless mobile device; selecting a first image parcel associated with a first discrete spatial area of the predetermined pyramid grid structure based on the predetermined pyramid grid structure and the first user-controlled viewing frustum automatically without specific user input, in response to the navigational input; sending a first request specifically for the first image automatically without specific user input, in response to the navigational input; receiving the first image parcel; selecting a second image parcel and a plurality of subsequent image parcels for the first user-controlled viewing frustum based on the predetermined pyramid grid structure and the first user-controlled viewing frustum automatically without specific user input, in response to the navigational input; sending a second request specifically for the second image parcel automatically without specific user input, in response to the navigational input, the second image parcel being associated with a second discrete spatial area of the predetermined pyramid grid structure; the sending of the first request and the second request being performed according to a priority order based at least in part on corresponding levels of the first and second grid elements in the predetermined pyramid grid structure, the first image parcel having an earlier priority than the second image parcel according to the priority order, a first non-empty overlap area existing between the first discrete spatial area and the second discrete spatial area; receiving the second image parcel; automatically, without further user input after the navigational input, rendering a view for the first user-controlled viewing frustum comprising rendering at least a portion of the first image parcel before finishing receiving and rendering the second image parcel to thereby enable a regional enhancement of level of details for the first user-controlled viewing frustum in at least the first non-empty overlap area; storing the first image parcel and the second image parcel received by the wireless mobile device; and providing access to at least some image parcels stored in a local image repository, through a local embedded server. - View Dependent Claims (26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36)
-
-
37. A wireless mobile device comprising client software, the client software comprising instructions to configure the wireless mobile device to:
-
employ a predetermined pyramid grid structure arranged in a hierarchy comprising a plurality of levels such that each level of the hierarchy includes a regular grid, the regular grid at each lower level of the hierarchy having more elements than at a higher level in the hierarchy, wherein each element of each regular grid of the predetermined pyramid grid structure defines a discrete spatial area, and the discrete spatial areas at any given level cover, in a substantially continuous and non-overlapping manner, an area that is substantially the same across the plurality of levels; determine a first user-controlled viewing frustum based on navigational input to the wireless mobile device; select a first image parcel associated with a first discrete spatial area of the predetermined pyramid grid structure based on the predetermined pyramid grid structure and the first user-controlled viewing frustum automatically without specific user input, in response to the navigational input; send a first request specifically for the first image parcel automatically without specific user input, in response to the navigational input; receive the first image parcel over a network communication channel; select a second image parcel and a plurality of subsequent image parcels for the first user-controlled viewing frustum based on the predetermined pyramid grid structure and the first user-controlled viewing frustum, the second image parcel being associated with a second discrete spatial area of the predetermined pyramid grid structure automatically without specific user input, in response to the navigational input, wherein a first non-empty overlap area exists between the first discrete spatial area and the second discrete spatial area; send a second request specifically for the second image parcel automatically without specific user input, in response to the navigational input; the first request and the second request being sent according to a priority order based at least in part on viewable areas of the first and the second image parcels corresponding to the first user-controlled viewing frustum; receive the second image parcel over the network communication channel; automatically, without further user input after the navigational input, render a view for the first user-controlled viewing frustum comprising rendering at least a portion of the first image parcel before finishing receiving and rendering the second image parcel to thereby enable a regional enhancement of level of details for the first user-controlled viewing frustum in at least the first non-empty overlap area; store the first image parcel and the second image parcel; and provide access to image parcels stored in a local image repository through a local embedded server. - View Dependent Claims (38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48)
-
-
49. A method of communicating images for display on a wireless mobile device, performed by one or more servers, comprising:
-
preprocessing source image data to generate image parcels according to a predetermined subdivision structure, comprising a plurality of levels, such that each level of the predetermined subdivision structure includes a regular array, the regular array at each subsequent level of the predetermined subdivision structure having fewer elements than at a previous level, wherein each image parcel occupies an area defined by a respective element of a respective array, and the areas at any given level forming a regular array that covers, in a substantially continuous and non-overlapping manner, an region that is substantially the same across the plurality of levels, providing client software to the wireless mobile device, the client software comprising instructions to configure the wireless mobile device to; request and receive image parcels through a local embedded server, the local embedded server being configured to use a local image repository to store image parcels using a local image repository, the local embedded server providing access to at least some image parcels; determine a first user-controlled viewing frustum based on a navigational input to the wireless mobile device; select a first image parcel and a second image parcel which incompletely overlaps the first image parcel, the level of the first image parcel being subsequent to the level of the second image parcel, based on the first user-controlled viewing frustum; send a first request and a second request, automatically without specific user input, in response to the navigational input according to a priority order based at least in part on corresponding levels in the predetermined subdivision structure of the first and second image parcels; automatically, without further user input after the navigational input, render a view for the first user-controlled viewing frustum comprising rendering at least a portion of the first image parcel before finishing receiving and rendering the second image parcel thereby enabling a regional enhancement of a level of detail for the first user-controlled viewing frustum at least in the second area; and store the first image parcel and the second image parcel received by the wireless mobile device; receiving the first request specifically for the first image parcel at the one or more servers from the wireless mobile device; sending the first image parcel from the one or more servers to the wireless mobile device, in response to the first request; receiving the second request specifically for the second image parcel at the one or more servers from the wireless mobile device; and sending the second image parcel from the one or more servers to the wireless mobile device, in response to the second request.
-
Specification