SYSTEMS AND METHODS FOR NETWORKED, IN-CONTEXT, COMPOSED, HIGH RESOLUTION IMAGE VIEWING

• US 20120007866A1
• Filed: 07/12/2010
• Published: 01/12/2012
• Est. Priority Date: 07/12/2010
• Status: Active Grant

• Current Assignee: Opus Medical, Inc. (Smith & Nephew PLC)
• Original Assignee: CME Advantage, Inc.
• Inventors: Tahan, Thomas
• Granted Patent: US 8,667,054 B2
• Time in Patent Office: Days
• US Class Current: 345/428
• CPC Class Codes: G06T 3/053 Detail-in-context presentations fisheye or wide-angle transformations G06T3/047
• CPC Class Codes: G16H 30/20 for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing
• CPC Class Codes: G16H 40/67 for remote operation
• CPC Class Codes: H04N 19/164 Feedback from the receiver or from the transmission channel
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/33 in the spatial domain
• CPC Class Codes: H04N 21/23439 for generating different versions
• CPC Class Codes: H04N 21/25816 involving client authentication restricting access to computer systems by authenticating users using a predetermined code G06F21/33; cryptographic authentication protocols H04L9/32; networks authentication protocols H04L63/08; authentication in wireless network security H04W12/06
• CPC Class Codes: H04N 21/25875 involving end-user authentication restricting access to computer systems by authenticating users using a predetermined code G06F21/33; arrangements for secret or secure communication including means for verifying the identity or authority of a user of the system H04L9/32; networks authentication protocols H04L63/08; authentication in wireless network security H04W12/06
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/4312 involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
• CPC Class Codes: H04N 21/440245 the reformatting operation being performed only on part of the stream, e.g. a region of the image or a time segment
• CPC Class Codes: H04N 21/4728 for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
• CPC Class Codes: H04N 21/6379 directed to encoder , e.g. for requesting a lower encoding rate
• CPC Class Codes: H04N 21/6582 Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number arrangements where receivers interact with the broadcast H04H20/38
• CPC Class Codes: H04N 21/6587 Control parameters, e.g. trick play commands, viewpoint selection

Abstract

Systems and methods are provided for viewing portions of an image in high resolution and in context with a full image, which is displayed at a base resolution that is lower resolution than the resolution of the high-resolution image. A client device can send a request to a server for a composed image comprised of the base-resolution image overlaid at the area of interest with a virtual lens showing the area of interest in high-resolution image. The request can include the coordinates of the area of interest relative to a reference point on the image, the desired resolution level for the high-resolution image, and the virtual lens size and shape. The desired resolution can fall into a range above the resolution of the base-resolution image and up to the resolution of the maximum-resolution image stored on, generated on, or input to the server.

Claims

1. A computer-implemented method for displaying a portion of a high-resolution image in the context of a base-resolution image, wherein one or more processors are programmed to perform the steps comprising:

receiving a request to view an image at a client device;

sending a request to a server over a network for a base-resolution image that corresponds to the requested image;

receiving the base-resolution image from the server over the network;

displaying the base-resolution image on a display of the client device;

receiving a request to display an area of interest of the base-resolution image in high resolution;

sending a request to the server over the network for a composed image that corresponds to the base-resolution image;

receiving the composed image from the server over the network; and

displaying the composed image on the display of the client device.

16. The method of claim 1, wherein the selections of the client device'"'"'s user are saved in the memory of the client device for later use.

2. The method of claim 2 wherein the resolution of the base-resolution image is less than the resolution of a maximum-resolution image generated on, input to, or stored on the server.

3. The method of claim 2 wherein the composed image is comprised of the base resolution image with a portion of a high-resolution image corresponding to an area of interest as an overlay of the base-resolution image at a location that corresponds to the area of interest.

4. The method of claim 3 wherein the resolution of the high-resolution image is less than or equal to the resolution of the maximum-resolution image, and wherein the resolution of the high-resolution image is greater than the resolution of the base-resolution image.

5. The method of claim 2 wherein the resolution of the base-resolution image is selected at least in part based on the capabilities of the client device or network.

6. The method of claim 5 wherein the displayed base-resolution image is generated from a received image of greater than or equal resolution.

7. The method of claim 3 wherein the overlay comprises a virtual lens.

8. The method of claim 7 further comprising:

receiving a request at the client device to resize the virtual lens;

sending a request to the server for a composed image with the virtual lens having the requested size;

receiving the composed image from the server; and

displaying the composed image on the client device display.

9. The method of claim 7 further comprising:

receiving a request at the client device to change a shape of the virtual lens to a requested shape; and

sending a request to the server for a composed image with the virtual lens having requested shape;

receiving the composed image from the server; and

displaying the composed image on the client device display.

10. The method of claim 7 further comprising:

receiving a request at the client device to change resolution of the high-resolution image displayed by the virtual lens;

sending a request to the server for a composed image with a portion of the image corresponding to the area of interest at the requested high resolution as an overlay of the base-resolution image at a location that corresponds to the area of interest;

receiving the composed image from the server; and

displaying the composed image on the client device display.

11. The method of claim 7, further comprising:

tracking the movements of a cursor controlled by a pointing device over the base-resolution image at the client device;

sending a request to the server for a composed image comprised of a portion of the image corresponding to the area of interest at the requested high resolution as an overlay of the base-resolution image at a location that corresponds to the area of interest;

receiving the composed image from the server; and

displaying the composed image on the client device display.

12. The method of claim 11, further comprising:

freezing the position of the virtual lens with respect to the base-resolution image in response to a user command.

13. The method of claim 12, further comprising:

unfreezing the position of the previously frozen virtual lens with respect to the base-resolution image in response to a user command.

14. The method of claim 7, further comprising:

displaying the base-resolution image in response to a user command to hide the virtual lens.

15. The method of claim 14, further comprising:

displaying the composed image in response to a user command to show a previously hidden virtual lens.

17. A computer-implemented method for providing high-resolution imagery, wherein one or more processors are programmed to perform the steps comprising:

receiving a request from a client device for a base-resolution image;

verifying that the client device or a user of the client device has access rights to the base-resolution image per an access control policy;

generating a base-resolution image based on an image of greater than or equal resolution generated on, input to, or stored on the server and associated with the requested base-resolution image;

transmitting the base-resolution image to the client device via a network connection;

receiving a request for a composed image associated with the base-resolution image and area of interest;

verifying that the client device or the user of the client device has access rights to the composed image per the access control policy;

generating the composed image; and

transmitting the composed image to the client device.

18. The method of claim 17, wherein the composed image is comprised of the base-resolution image with a portion of the image corresponding to the area of interest at the requested high resolution as an overlay of the base-resolution image at a location that corresponds to the area of interest.

19. The method of claim 18 wherein the base-resolution image is generated from an image of greater than or equal resolution generated on, input to, or stored on the server.

20. The method of claim 18 wherein the high-resolution image is generated from an image of greater than or equal resolution generated on, input to, or stored on the server.

21. The method of claim 18 wherein the overlay comprises a virtual lens.

22. The method of claim 21, wherein generating the composed image containing the virtual lens further comprises:

rendering a border around the virtual lens to demarcate the virtual lens from the base-resolution image.

23. The method of claim 21, wherein generating the composed image containing the virtual lens further comprises:

rendering the virtual lens such that the virtual lens is centered over the area of interest.

24. The method of claim 23, wherein rendering the virtual lens such that the virtual lens is centered over the area of interest further comprises:

determining whether at least a portion of the virtual lens would be outside a border of the base-resolution image if the virtual lens were rendered centered over the area of interest;

if at least a portion of the virtual lens would be outside a border of the base-resolution image, determining an offset location for the virtual lens where the virtual lens would not be outside of the border of the base-resolution image; and

rendering the virtual lens such that the virtual lens and the portion of the high-resolution image displayed within it is centered over the offset location.

25. The method of claim 21, further comprising:

adjusting the size and shape of the virtual lens to improve the function of an application in which the virtual lens is being used.

26. The method of claim 18 wherein the image is video.

27. The method of claim 26, wherein displaying a portion of the high-resolution image corresponding to the area of interest as an overlay of the base-resolution image content at a location that corresponds to the area of interest further comprises:

synchronizing a frame rate and a currently displayed frame of the high-resolution image with a frame-rate and currently displayed frame of the base-resolution video.

28. The method of claim 17, wherein the selections of the client device'"'"'s user are saved in the memory of the server for later use.

29. A technical system for providing high-resolution imagery, the system comprising:

a non-transitory computer readable medium for storing computer executable programmed modules;

a processor communicatively coupled with the non-transitory computer readable medium for executing programmed modules stored therein;

a network interface module stored in the non-transitory computer readable medium and configured to

receive over a network a request for an image at a selected resolution from a client device,

transmit the image to the client device over the network if the client device or a user of the client device has access rights to the image;

receive over a network from a client device a request for a composed image comprised of a portion of a high-resolution image corresponding to an area of interest overlaid over a base-resolution image at a location corresponding to the area of interest, and

transmit the composed image to the client device over the network if the client device or a user of the client device has access rights to the image;

a request processing module stored in the non-transitory computer readable medium and in communication with the network interface module, the request processing module being configured to

receive the request for an image at the selected resolution from the network interface module,

process the request for the image at the selected resolution,

send the requested image at the requested resolution to the network interface module for transmission to the client device;

receive the request for the composed image from the network interface module,

process the request for the composed image, and

send the composed image to the network interface module for transmission to the client device;

an access control module stored in the non-transitory computer readable medium and in communication with the request processing module, the access control module being configured to

receive an authorization request from the request processing module to verify access rights of the client device or a user of the client device to the requested image,

verify the access rights per an access control policy, and

send an authorization response to the request processing module indicating whether the client device or the user has access rights to the image;

an image management module stored in the non-transitory computer readable medium in communication with the request processing module, and the image management module being configured to process an image management request using a data store.

30. The system of claim 29, wherein the request processing module in processing the request is further configured to:

determine whether to send the requested image to the network interface module for transmission to the requesting client device based on the authorization response from the access control module indicating that the client device or the user of the client device has access rights.

31. The system of claim 29, wherein the request processing module in processing the request for the image at the selected resolution is further configured to:

send a request to the image management module for an image at a resolution greater than or equal to the base resolution specified in the request;

receive from the image management module the image at the resolution greater than or equal to the base resolution specified in the request; and

generate the base-resolution image based on the received image.

32. The system of claim 29, wherein the request processing module in processing the request for the composed image is further configured to:

send a request to the image management module for an image at a resolution greater than or equal to the base resolution specified in the request;

receive from the image management module the image at the resolution greater than or equal to the base resolution specified in the request;

generate the base-resolution image based on the received image;

send a request to the image management module for an image at a resolution greater than or equal to the high resolution specified in the request;

receive from the image management module the image at the resolution greater than or equal to the high resolution specified in the request;

generate the high-resolution image based on the received image; and

generate a composed image by overlaying a portion of the high-resolution image corresponding to the area of interest specified in the request at a location of the base-resolution image that corresponds to the area of interest.

33. The system of claim 29, wherein the image management module is further configured to

input an image from an image input device or system;

and

store the image in the data store.

34. The system of claim 29, wherein the image management module is further configured to

generate an image;

and

store the image in the data store.

35. The system of claim 29, wherein the image management module is further configured to:

generate a plurality of images from an image of greater than or equal resolution, each of the plurality of images having a different resolution level and generated from an image of greater than or equal resolution generated on, input to, or stored on the server, and

store the plurality of images in the data store.

36. The system of claim 29, wherein the request processing module in processing the request is further configured to:

store a set of user selections in the data store for later use.

37. A technical system for viewing high-resolution imagery, the system comprising:

a non-transitory computer readable medium for storing computer executable programmed modules;

a processor communicatively coupled with the non-transitory computer readable medium for executing programmed modules stored therein;

a network interface module stored in the non-transitory computer readable medium and configured to

transmit a request for an image to a server over a network,

receive the requested image from the server over the network;

transmit a request for a composed image to the server over the network, and

receive the composed image from the server over the network;

a user interface module stored in the non-transitory computer readable medium and configured to

receive a request from a user to view an image,

receive a base-resolution image corresponding to the requested image,

display the base-resolution image on a display of the system,

receive a request from the user to display an area of interest of the base-resolution image in high resolution,

send a request to the request processing module for a composed image comprised of a portion of a high-resolution image corresponding to the area of interest overlaid on the base-resolution image at a location that corresponds to the area of interest,

receive the composed image from the request processing module, and

display the composed image on a display of the system; and

a request processing module stored in the non-transitory computer readable medium and in communication with the user interface module and the network interface module, the request processing module being configured to

process a request for a base-resolution image from the user interface module, and

process a request for a composed image from the user interface module; and

an image management module stored in the non-transitory computer readable medium and in communication with the request processing module and network interface module, the image management module being configured to process an image management request using a data store.

38. The system of claim 37, wherein the request processing module in processing a request for a base-resolution image is further configured to:

receive a request for a base-resolution image from the user interface module;

send a request for the base-resolution image to the image management module;

receive the base-resolution image from the image management module; and

send the base-resolution image to the user interface module for display.

39. The system of claim 37, wherein the request processing module in processing a request for a composed image is further configured to:

receive a request from the user interface module for a composed image comprised of a portion of the high-resolution image corresponding to the area of interest overlaid over the base-resolution image at a location that corresponds to the area of interest,

send a request to the image management module for the composed image;

receive the composed image from the image management module; and

send the requested composed image to the user interface module.

40. The system of claim 37, wherein the image management module in processing the image management request is further configured to:

receive a request for an image from the request processing module;

retrieve the requested image; and

send the requested image to the request processing module.

41. The system of claim 40, wherein the image management module in retrieving the requested image is further configured to:

check whether the requested image is cached in the data store; and

retrieve the requested image from the data store if it is cached.

42. The system of claim 40, wherein the image management module in retrieving the requested image is further configured to:

send a request for the image to the network interface module;

receive the requested image from the network interface module; and

cache the received image in the data store.

43. The system of claim 37, wherein the request processing module in processing the request is further configured to:

store a set of user selections in the data store for later use.