Optimized image delivery over limited bandwidth communication channels

• US 9,641,645 B2
• Filed: 09/29/2016
• Issued: 05/02/2017
• Est. Priority Date: 12/27/2000
• Status: Expired due to Fees

• Current Assignee: Bradium Technologies LLC (General Patent Corporation)
• Original Assignee: Bradium Technologies LLC (General Patent Corporation)
• Inventors: Levanon, Isaac, Lavi, Yonatan
• Primary Examiner(s): LAZARO, DAVID R
• Application Number:
  US15/281,037
• Publication Number:
  US 20170019501A1
• Time in Patent Office: 215 Days
• Field of Search: 709202, 709203, 709217, 709218, 709230, 709231, 382305, 382232, 345428, 345581, 345625
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 3/4092 Image resolution transcoding, e.g. by using client-server architectures
• CPC Class Codes: G09G 2340/02 Handling of images in compressed format, e.g. JPEG, MPEG
• CPC Class Codes: G09G 2350/00 Solving problems of bandwidth in display systems
• CPC Class Codes: G09G 2370/02 Networking aspects
• CPC Class Codes: G09G 2370/16 Use of wireless transmission of display information
• CPC Class Codes: G09G 5/003 Details of a display terminal, the details relating to the control arrangement of the display terminal and to the interfaces thereto specific for a CRT G09G1/165; for a flat panel G09G3/2092
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/63 Routing a service request depending on the request content or context

Abstract

Large-scale images are retrieved over network communications channels for display on a client device by selecting an update image parcel relative to an operator controlled image viewpoint to display via the client device. A request is prepared for the update image parcel and associated with a request queue for subsequent issuance over a communications channel. The update image parcel is received from the communications channel and displayed as a discrete portion of the predetermined image. The update image parcel optimally has a fixed pixel array size, is received in a single and or plurality of network data packets, and were the fixed pixel array may be constrained to a resolution less than or equal to the resolution of the client device display.

Claims

1. A method of communicating images for display, the method comprising steps of:

processing data of a source image to obtain a series (K₀, K₁ . . . K_1-N) of related images of progressively lower image resolution, wherein each related image of the series (K₀, K₁ . . . K_1-N) comprises image data and is subdivided into a regular array of image parcels, each image parcel of each regular array of the image parcels forming a discrete portion of the source image and having same predetermined pixel number and same predetermined color or bit per pixel depth, the step of processing being performed by one or more servers;

receiving a first request at the one or more servers from a wireless portable device over a network communication channel, the first request being for a first image parcel of the series, wherein the first image parcel is selected based on a first user-controlled image viewpoint on the wireless portable device relative to the source image;

sending the first image parcel from the one or more servers to the wireless portable device over the network communication channel, in response to the first request;

receiving a second request at the one or more servers from the wireless portable device over the network communication channel, the second request being for a second image parcel of the series, wherein the second image parcel is selected based on the first user-controlled image viewpoint or on a second user-controlled image viewpoint on the wireless portable device relative to the source image, the step of receiving the second request being performed after the step of receiving the first request; and

sending the second image parcel from the one or more servers to the wireless portable device over the network communication channel, in response to the second request.

2. The method of claim 1, further comprising:

providing client software to the wireless portable device;

wherein the wireless portable device renders at least a portion of the first image parcel before finishing receiving the second image parcel.

3. The method of claim 2, wherein:

the wireless portable device issues the first request and the second request according to a priority order;

priority of the second request in the priority order is not higher than priority of the first request in the priority order; and

wherein the wireless portable device stores the first image parcel and the second image parcel in a local parcel storage.

4. The method of claim 2, wherein the wireless portable device stores the first image parcel and the second image parcel received by the wireless portable device in a local store of the wireless portable device.

5. The method of claim 4, wherein the first user-controlled image viewpoint is determined based on navigational input of the wireless portable device.

6. The method of claim 5, wherein the navigational input comprises three-dimensional positional coordinate data and rotational positional data.

7. The method of claim 5, wherein the client software configures the local store as a server to provide access to at least some image parcels received by the wireless portable device, the at least some image parcels comprising the first image parcel and the second image parcel.

8. The method of claim 1, wherein the second image parcel is selected based on the first user-controlled image viewpoint, and number of parallel requests by the wireless portable device for image parcels of the series is determined based at least in part on network response latency and available system resources, thereby enabling efficient use of network bandwidth in conditions of network latency.

9. The method of claim 1, further comprising sending overlay data by the one or more servers to the wireless portable device over the network communication channel.

10. The method according to claim 9, wherein the overlay data comprises text annotations relating to at least one item selected from the group consisting of:

one or more street names, one or more building names, and one or more landmarks.

11. The method of claim 1, wherein the wireless portable device issues the first request and the second request according to a priority order based at least in part on viewable areas corresponding to the first user-controlled image viewpoint.

12. The method of claim 1, wherein the wireless portable device issues the first request and the second request according to a priority order based at least in part on resolutions of the first image parcel and the second image parcel.

13. A computing system comprising one or more servers, wherein the one or more servers are coupled to a wireless portable device by a network communication channel, the one or more servers being configured to:

process data of a source image to obtain a series (K₀, K₁ . . . K_1-N) of related images of progressively lower image resolution, wherein each related image of the series (K₀, K₁ . . . K_1-N) comprises image data and is subdivided into a regular array of image parcels, each image parcel of each regular array of the image parcels forming a discrete portion of the source image and having same predetermined pixel number and same predetermined color or bit per pixel depth, resolution of each related image of the series except initial of the related images in the series being related to resolution of the immediately preceding related image in the series by a factor of four, number of image parcels into which each related image of the series except the initial of the related images is subdivided being related by a factor of four to number of image parcels into which the immediately preceding related image in the series is subdivided;

receive a first request from the wireless portable device over the network communication channel, the first request being for a first image parcel of the series, wherein the first image parcel is selected based on a first user-controlled image viewpoint on the wireless portable device relative to the source image;

send the first image parcel from the one or more servers to the wireless portable device over the network communication channel, in response to the first request;

receive a second request at the one or more servers from the wireless portable device over the network communication channel, the second request being for a second image parcel of the series, wherein the second image parcel is selected based on the first user-controlled image viewpoint or on a second user-controlled image viewpoint on the wireless portable device relative to the source image, wherein the second request is received after the first request; and

send the second image parcel to the wireless portable device over the network communication channel, in response to the second request.

14. The computing system of claim 13, wherein the one or more servers are further configured to

provide client software to the wireless portable device;

and

wherein the wireless portable device renders at least a portion of the first image parcel before finishing receiving the second image parcel.

15. The computing system of claim 14, wherein:

the wireless portable device issues the first request and the second request according to a priority order;

priority of the second request in the priority order is not higher than priority of the first request in the priority order; and

wherein the wireless portable device stores the first image parcel and the second image parcel in a local parcel storage.

16. The computing system of claim 14, wherein the wireless portable device stores at least some image parcels received by the wireless portable device in a local store of the wireless portable device.

17. The computing system of claim 16, wherein the first user-controlled image viewpoint is determined based on navigational input of the wireless portable device.

18. The computing system of claim 17, wherein the navigational input comprises three-dimensional positional coordinate data and rotational positional data.

19. The computing system of claim 17, wherein the client software configures the local store as a server to provide access to the at least some image parcels received by the wireless portable device, the at least some image parcels comprising the first image parcel and the second image parcel received by the wireless portable device.

20. The computing system of claim 13, wherein number of parallel requests by the wireless portable device for image parcels of the series is determined based at least in part on network response latency and available system resources, thereby enabling efficient use of network bandwidth in conditions of network latency.

21. The computing system of claim 13, wherein the one or more servers are further configured to send overlay data to the wireless portable device over the network communication channel.

22. The computing system according to claim 21, wherein the overlay data comprises text annotations relating to at least one item selected from the group consisting of:

one or more street names, one or more building names, and one or more landmarks.

23. The computing system of claim 13, wherein the wireless portable device issues the first request and the second request according to a priority order based at least in part on viewable areas corresponding to the first user-controlled image viewpoint.

24. The computing system of claim 13, wherein the wireless portable device issues the first request and the second request according to a priority order based at least in part on resolutions of the first image parcel and the second image parcel.

25. A method of communicating images for display, the method comprising steps of:

sending a first request from a wireless portable device to one or more servers over a network communication channel, the first request being for a first image parcel, the first image parcel being selected based on a first user-controlled image viewpoint on the wireless portable device relative to a source image;

receiving the first image parcel by the wireless portable device from the one or more servers over the network communication channel, in response to the first request;

sending a second request from the wireless portable device to the one or more servers over the network communication channel, the second request being for a second image parcel, the second image parcel being selected based on the first user-controlled image viewpoint or on a second user-controlled image viewpoint on the wireless portable device relative to the source image, the step of sending the second request being performed after the step of sending the first request; and

receiving the second image parcel by the wireless portable device from the one or more servers over the network communication channel, in response to the second request;

wherein the source image is processed by one or more servers to obtain a series (K₀, K₁ . . . K_1-N) of related images of progressively lower image resolution, wherein each related image of the series (K₀, K₁ . . . K_1-N) comprises image data and is subdivided into a regular array of image parcels, each image parcel of each regular array of the image parcels forming a discrete portion of the source image and having same predetermined pixel number and same predetermined color or bit per pixel depth, resolution of each related image of the series except initial of the related images in the series being related to resolution of the immediately preceding related image in the series by a first predetermined factor, number of image parcels into which each related image of the series except the initial of the related images is subdivided being related by a second predetermined factor to number of image parcels into which the immediately preceding related image in the series is subdivided, and wherein the series comprises the first image parcel and the second image parcel.

26. The method of claim 25, further comprising rendering by the wireless portable device at least a portion of the first image parcel before finishing receiving the second image parcel.

27. The method of claim 26, wherein:

the steps of sending the first request and sending the second request are performed according to a priority order; and

priority of the second request in the priority order is not higher than priority of the first request in the priority order;

the method further comprising storing the first image parcel and the second image parcel in a local parcel storage of the wireless portable device.

28. The method of claim 26, further comprising storing the first image parcel and the second image parcel received by the wireless portable device in a local store of the wireless portable device.

29. The method of claim 28, further comprising determining the first user-controlled image viewpoint based on navigational input of the wireless portable device.

30. The method of claim 29, wherein the navigational input comprises three-dimensional positional coordinate data and rotational positional data.

31. The method of claim 28, further comprising receiving client software by the wireless portable device from the one or more servers, wherein the step of storing comprises storing at least some image parcels received by the wireless portable device in the local store configured by the client software as a server to provide access to the at least some image parcels, the at least some image parcels comprising the first image parcel and the second image parcel received by the wireless portable device.

32. The method of claim 25, further comprising determining number of parallel requests by the wireless portable device for image parcels of the series based at least in part on network response latency and available system resources, to enable efficient use of network bandwidth in conditions of network latency.

33. The method of claim 25, further comprising receiving by the wireless portable overlay data sent by the one or more servers to the wireless portable device over the network communication channel.

34. The method according to claim 33, wherein the overlay data comprises text annotations relating to at least one item selected from the group consisting of:

one or more street names, one or more building names, and one or more landmarks.

35. The method of claim 25, wherein the steps of sending the first request and sending the second request are performed according to a priority order based at least in part on viewable areas corresponding to the first user-controlled image viewpoint.

36. The method of claim 25, wherein the steps of sending the first request and sending the second request are performed according to a priority order based at least in part on resolutions of the first image parcel and the second image parcel.