Optimized image delivery over limited bandwidth communication channels

• US 9,635,136 B2
• Filed: 11/03/2016
• Issued: 04/25/2017
• Est. Priority Date: 12/27/2000
• Status: Expired due to Term

• Current Assignee: Bradium Technologies LLC (General Patent Corporation)
• Original Assignee: Bradium Technologies LLC (General Patent Corporation)
• Inventors: Levanon, Isaac, Lavi, Yonatan
• Primary Examiner(s): LAZARO, DAVID R
• Application Number:
  US15/343,052
• Publication Number:
  US 20170054830A1
• Time in Patent Office: 173 Days
• Field of Search: 709202, 709203, 709217, 709218, 709230, 709231, 382305, 382232, 345428, 345581, 345625
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 3/4092 Image resolution transcoding, e.g. by using client-server architectures
• CPC Class Codes: G09G 2340/02 Handling of images in compressed format, e.g. JPEG, MPEG
• CPC Class Codes: G09G 2350/00 Solving problems of bandwidth in display systems
• CPC Class Codes: G09G 2370/02 Networking aspects
• CPC Class Codes: G09G 2370/16 Use of wireless transmission of display information
• CPC Class Codes: G09G 5/003 Details of a display terminal, the details relating to the control arrangement of the display terminal and to the interfaces thereto specific for a CRT G09G1/165; for a flat panel G09G3/2092
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/63 Routing a service request depending on the request content or context

Abstract

Large-scale images are retrieved over network communications channels for display on a client device by selecting an update image parcel relative to an operator controlled image viewpoint to display via the client device. A request is prepared for the update image parcel and associated with a request queue for subsequent issuance over a communications channel. The update image parcel is received from the communications channel and displayed as a discrete portion of the predetermined image. The update image parcel optimally has a fixed pixel array size, is received in a single and or plurality of network data packets, and were the fixed pixel array may be constrained to a resolution less than or equal to the resolution of the client device display.

Claims

1. A method of communicating images for display to a plurality of wireless portable devices, the method comprising steps of:

pre-processing a source image to obtain a series (K₀, K<sub>−

1</sub> . . . K_1-N) of related images of progressively lower image resolution, wherein each related image of the series (K₀, K<sub>−

1</sub> . . . K_1-N) comprises pixel data and is subdivided into a regular array of image parcels, and each image parcel of each regular array of the image parcels forms a discrete portion of the source image;

storing the image parcels of the series for serving by one or more servers;

providing client software to a wireless portable device;

receiving a first request at the one or more servers from the first wireless portable device over a network communication channel, wherein the network communication channel is at least in part wireless, and the first request being for a first image parcel of the series, wherein the first image parcel is selected based on a first user-controlled image viewpoint on the first wireless portable device relative to the source image;

sending the first image parcel from the one or more servers to the first wireless portable device over the network communication channel, in response to the first request;

receiving a second request at the one or more servers from the first wireless portable device over the network communication channel, the second request being for a second image parcel of the series, wherein the second image parcel is selected based on the first user-controlled image viewpoint on the first wireless portable device relative to the source image, the step of receiving the second request being performed after the step of receiving the first request; and

sending the second image parcel from the one or more servers to the first wireless portable device over the network communication channel, in response to the second request;

wherein;

the first wireless portable device renders at least a portion of the first image parcel before finishing receiving the second image parcel;

the first wireless portable device issues the first request and the second request according to a priority order;

priority of the second request in the priority order is not higher than priority of the first request in the priority order;

the first wireless portable device stores the first image parcel and the second image parcel in a local parcel storage at the first wireless portable device;

before issuing the first request, the first wireless portable device determines that a third image parcel is usable for the first user-controlled image viewpoint and the third image parcel is already stored in the local parcel storage, and the first wireless portable device renders the third image parcel before issuing the first request without requesting the third image parcel over the network communication channel;

the first wireless portable device handles download operations of at least the first image parcel and the second image parcel in parallel, thereby enabling efficient use of network bandwidth in conditions of network latency; and

the local parcel store is configured as a server to provide access to the at least some image parcels received by the first wireless portable device.

2. The method of claim 1, wherein the first user-controlled image viewpoint is determined based on a navigational input of the first wireless portable device.

3. The method of claim 2 wherein the navigational input comprises three-dimensional positional coordinate data and rotational positional data.

4. The method of claim 2, wherein the first wireless portable device stores requests for image parcels to be downloaded from the one or more servers in a priority queue, and wherein responsive to a change in the navigational input, a request is removed from the priority queue.

5. The method of claim 1, wherein number of parallel image parcel download operations by the first wireless portable device for image parcels of the series is determined based at least in part on network response latency and available system resources.

6. The method of claim 1, further comprising sending overlay data by the one or more servers to the first wireless portable device over the network communication channel.

7. The method according to claim 6, wherein the overlay data comprises text annotations relating to at least one item selected from the group consisting of:

one or more street names, one or more building names, and one or more landmarks.

8. The method of claim 1, wherein the wireless portable device issues the first request and the second request according to a priority order based at least in part on viewable areas corresponding to the first user-controlled image viewpoint.

9. The method of claim 1, wherein the first wireless portable device issues the first request and the second request according to a priority order based at least in part on resolution of the first image parcel and resolution of the second image parcel.

10. A computing system comprising one or more servers, wherein the one or more servers are coupled to a wireless portable device by a network communication channel, the network communication channel being at least in part wireless, the one or more servers being configured to:

pre-process a source image to obtain a series (K₀, K<sub>−

1</sub> . . . K_1-N) of related images of progressively lower image resolution, wherein each related image of the series (K₀, K<sub>−

1</sub> . . . K_1-N) comprises pixel data and is subdivided into a regular array of image parcels, and each image parcel of each regular array of the image parcels forms a discrete portion of the source image;

store the image parcels of the series for serving by the one or more servers;

provide client software to the wireless portable device;

receive a first request from the wireless portable device over the network communication channel, the first request being for a first image parcel of the series, wherein the first image parcel is selected based on a first user-controlled image viewpoint on the wireless portable device relative to the source image;

send the first image parcel from the one or more servers to the wireless portable device over the network communication channel, in response to the first request;

receive a second request from the wireless portable device over the network communication channel, the second request being for a second image parcel of the series, wherein the second image parcel is selected based on the first user-controlled image viewpoint on the wireless portable device relative to the source image, the second request being received by the one or more servers after the first request is received by the one or more servers; and

send the second image parcel from the one or more servers to the wireless portable device over the network communication channel, in response to the second request;

wherein;

the wireless portable device renders at least a portion of the first image parcel before finishing receiving the second image parcel;

the wireless portable device issues the first request and the second request according to a priority order;

priority of the second request in the priority order is not higher than priority of the first request in the priority order;

the wireless portable device stores the first image parcel and the second image parcel in a local parcel storage at the first wireless portable device;

before issuing the first request, the first wireless portable device determines that a third image parcel is usable for the first user-controlled image viewpoint and that the third image parcel is already stored in the local parcel storage, and the wireless portable device renders the third image parcel before issuing the first request without requesting the third image parcel over the network communication channel;

the wireless portable device handles download operations of at least the first image parcel and the second image parcel in parallel, thereby enabling efficient use of network bandwidth in conditions of network latency; and

the local parcel store is configured as a server to provide access to the at least some image parcels received by the wireless portable device.

11. The computing system of claim 10, wherein the first user-controlled image viewpoint is determined based on a navigational input of the wireless portable device.

12. The computing system of claim 11, wherein the navigational input comprises three-dimensional positional coordinate data and rotational positional data.

13. The computing system of claim 11, wherein the wireless portable device stores requests for the image parcels of the series to be downloaded from the one or more servers in a priority queue, and wherein responsive to a change in the navigational input, a request is removed from the priority queue.

14. The computing system of claim 10, wherein number of parallel image parcel download operations by the wireless portable device for the image parcels of the series is determined based at least in part on network response latency and available system resources.

15. The computing system of claim 10, wherein the one or more servers are further configured to send overlay data to the wireless portable device over the network communication channel.

16. The computing system of claim 15, wherein the overlay data comprises text annotations relating to at least one item selected from the group consisting of:

one or more street names, one or more building names, and one or more landmarks.

17. The computing system of claim 10, wherein the wireless portable device issues the first request and the second request according to a priority order based at least in part on viewable areas corresponding to the first user-controlled image viewpoint.

18. The computing system of claim 10, wherein the wireless portable device issues the first request and the second request according to a priority order based at least in part on resolution of the first image parcel and resolution of the second image parcel.

19. A wireless portable device, wherein:

the wireless portable device is coupled to one or more servers over a network communication channel, the network communication channel being at least in part wireless, the one or more servers storing a series (K₀, K<sub>−

1</sub> . . . K_1-N) of related images of progressively lower image resolution, each related image of the series (K₀, K<sub>−

1</sub> . . . K_1-N) comprising pixel data and being subdivided into a regular array of image parcels, each image parcel of each regular array of the image parcels of the series forming a discrete portion of a source image, the series being obtained by processing the source image, the one or more servers being configured to serve the related images of the series;

the wireless portable device is configured to;

determine a first image parcel of the series based on a first user-controlled image viewpoint on the wireless portable device relative to the source image;

send a first request for the first image parcel of the series to the one or more servers over the network communication channel;

receive the first image parcel from the one or more servers over the network communication channel, in response to the first request;

determine a second image parcel based on the first user-controlled image viewpoint on the wireless portable device relative to the source image;

send a second request for the second image parcel to the one or more servers, the second request being sent after the first request; and

receive the second image parcel from the one or more servers to the wireless portable device over the network communication channel, in response to the second request;

render at least a portion of the first image parcel before finishing receiving the second image parcel;

store the first image parcel and the second image parcel in a local parcel storage at the first wireless portable device;

before sending the first request, determine that a third image parcel is usable for the first user-controlled image viewpoint and the third image parcel is already stored in the local parcel storage, and render the third image parcel without requesting the third image parcel over the network communication channel;

handle download operations of at least the first image parcel and the second image parcel in parallel, thereby enabling efficient use of network bandwidth in conditions of network latency;

configure the local parcel store as a server to provide access to at least some image parcels received by the wireless portable device;

the first request and the second request are issued according to a priority order;

priority of the second request in the priority order is not higher than priority of the first request in the priority order.

20. The wireless portable device of claim 19, further configured to determine the first user-controlled image viewpoint based on a navigational input of the wireless portable device.

21. The wireless portable device of claim 20, wherein the navigational input comprises three-dimensional positional coordinate data and rotational positional data.

22. The wireless portable device of claim 20, further configured to store requests for image parcels to be downloaded from the one or more servers in a priority queue, and, responsive to a change in the navigational input, to remove a request from the priority queue.

23. The wireless portable device of claim 19, wherein number of parallel image parcel download operations by the wireless portable device for image parcels of the series is determined based at least in part on network response latency and available system resources.

24. The wireless portable device of claim 19, further configured to receive overlay data sent by the one or more servers to the wireless portable device over the network communication channel, and to render the overlay data when rendering parcel images of the series.

25. The wireless portable device of claim 24, wherein the overlay data comprises text annotations relating to at least one item selected from the group consisting of:

one or more street names, one or more building names, and one or more landmarks.

26. The wireless portable device of claim 19, further configured to send the first request and the second request according to a priority order based at least in part on viewable areas corresponding to the first user-controlled image viewpoint.

27. The wireless portable device of claim 19, further configured to send the first request and the second request according to a priority order based at least in part on resolution of the first image parcel and resolution of the second image parcel.