SYSTEMS AND METHODS TO SUPER RESOLVE A USER-SELECTED REGION OF INTEREST

• US 20140375865A1
• Filed: 06/19/2014
• Published: 12/25/2014
• Est. Priority Date: 06/21/2013
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Rezaiifar, Ramin, Sung, Chiachi, SHROFF, Nitesh
• Granted Patent: US 9,635,246 B2
• Time in Patent Office: Days
• US Class Current: 348/333.12
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces

Abstract

Systems, devices, and methods are described for efficiently super resolving a portion of an image. One embodiment involves capturing, using a camera module of a device, at least one image of a scene, and creating a higher resolution image of a user-selected region of interest. The super resolution of the region of interest may be performed by matching a high resolution grid with a grid that is at the resolution of a device camera, populating the high resolution grid with information from an image from the camera, and then populating the remaining points of the grid that are not yet populated.

Claims

1. A method of improving image quality for a user-selected region of interest comprising:

capturing, using a camera module of a device, at least one image of a scene;

receiving, at the device, a user input identifying a region of interest, wherein the region of interest identifies a portion of the at least one image of the scene that is less than a complete area of the at least one image; and

creating a higher resolution image of the region of interest using the at least one image of the scene by;

creating a high resolution grid associated with the higher resolution image of the region of interest;

identifying one or more low resolution grids associated with the portion of the at least one image of the scene, wherein each low resolution grid is associated with one image, and wherein each point of each low resolution grid comprises information captured by the camera module;

determining an alignment between the high resolution grid and each of the one or more low resolution grids; and

populating points of the high resolution grid with information from a corresponding aligned pixel from the one or more low resolution grids.

2. The method of claim 1 wherein the at least one image of the scene is captured prior to receipt of the user input identifying the region of interest;

wherein the user input identifying the region of interest is a touch screen input on a display output identifying the portion of the at least one image when the scene is displayed on the display output.

3. The method of claim 1 wherein the at least one image of the scene is captured after receipt of the user input identifying the region of interest.

4. The method of claim 1 wherein the at least one image of the scene consists of a single image of the scene, and wherein creating the higher resolution image of the region of interest comprises using patch redundancies identified in the single image of the scene to create the higher resolution image of the region of interest from the single image of the scene.

5. The method of claim 1 further comprising:

receiving at the device, prior to the capturing of the at least one image of the scene, a user input selecting a number of images to capture for use in creating the higher resolution image of the region of interest,

wherein capturing the at least one image comprises automatically capturing the number of images of the scene; and

wherein creating the higher resolution image of the region of interest comprises identifying the region of interest in each image of the number of images and creating the higher resolution image of the region of interest using said identified region of interest in each image of the number of images.

6. The method of claim 1 further comprising:

displaying the higher resolution image of the region of interest on a display output of the device.

7. The method of claim 6 further comprising:

creating a second higher resolution image of at least a portion of the at least one image outside of the region of interest.

8. The method of claim 1 further comprising:

capturing sensor data for at least one sensor on the device; and

associating the sensor data with the at least one image,

wherein the sensor data is used to create the higher resolution image of the region of interest.

9. The method of claim 8 wherein the sensor data comprises motion data from an accelerometer of the device.

10. The method of claim 1 further comprising:

displaying the higher resolution image of the region of interest as a preview image;

receiving a user input in response to the displaying of the preview image requesting an additional increase in resolution of the higher resolution image of the region of interest; and

creating a second higher resolution image of the region of interest using the at least one image of the scene by;

creating a second higher resolution grid associated with the additional higher resolution image of the region of interest;

identifying one or more high resolution grids;

determining an alignment between the second higher resolution grid and each of the one or more high resolution grids; and

populating each point of the second higher resolution grid with information from the corresponding aligned pixel from the one or more high resolution grids.

11. A device comprising:

a memory;

a user input module;

a camera module comprising a sensor; and

a processor coupled to the memory and the camera module, the processor configured to;

capture, using the camera module of the device, at least one image of a scene;

receive, at the user input module, a user input identifying a region of interest, wherein the region of interest identifies a portion of the at least one image of the scene that is less than a complete area of the at least one image; and

create a higher resolution image of the region of interest using the at least one image of the scene by;

creating a high resolution grid associated with the higher resolution image of the region of interest;

identifying one or more low resolution grids associated with the portion of the at least one image of the scene, wherein each low resolution grid is associated with one image, and wherein each point of each low resolution grid comprises information captured by the camera module;

determining an alignment between the high resolution grid and each of the one or more low resolution grids; and

populating points of the high resolution grid with information from a corresponding aligned pixel from the one or more low resolution grids.

12. The device of claim 11 further comprising:

a display output;

wherein the at least one image of the scene is captured prior to receipt of the user input identifying the region of interest; and

wherein the user input module a touch screen input of the display output.

13. The device of claim 12 wherein the processor is further configured to:

display the higher resolution image of the region of interest on the display output of the device.

14. The device of claim 13 wherein the processor is further configured to:

identify free computer processing unit cycles available on the processor while the higher resolution image of the region of interest is displayed on the display output of the device; and

create a second higher resolution image of at least a portion of the at least one image outside of the region of interest using the at least one image of the scene while the higher resolution image of the region of interest is displayed on the display output.

15. The device of claim 11 wherein the at least one image of the scene consists of a single image of the scene, and wherein creating the higher resolution image of the region of interest comprises using patch redundancies identified in the single image of the scene to create the higher resolution image of the region of interest from the single image of the scene.

16. The device of claim 11 wherein the processor is further configured to:

receive at the device, prior to the capturing of the at least one image of the scene, a user input selecting a number of images to capture for use in creating the higher resolution image of the region of interest.

17. The device of claim 11 further comprising an accelerometer coupled to the processor, wherein the processor is further configured to:

capture motion data from the accelerometer;

associate the motion data with the at least one image; and

create the higher resolution image of the region of interest using the motion data.

18. A non-transitory computer readable storage medium comprising a set of instructions that, when executed by a processor coupled to the storage medium, cause a device improve an image quality for a user-selected region of interest, the instructions comprising:

receiving, at the device, at least one image of a scene;

receiving, at the device, a user input identifying a region of interest, wherein the region of interest identifies a portion of the at least one image of the scene that is less than a complete area of the at least one image; and

creating a higher resolution image of the region of interest using the at least one image of the scene by;

creating a high resolution grid associated with the higher resolution image of the region of interest;

identifying one or more low resolution grids associated with the portion of the at least one image of the scene, wherein each low resolution grid is associated with one image, and wherein each point of each low resolution grid comprises information captured by a camera module;

determining an alignment between the high resolution grid and each of the one or more low resolution grids; and

populating points of the high resolution grid with information from a corresponding aligned pixel from the one or more low resolution grids.

19. The non-transitory computer readable medium of claim 18 wherein the instructions further comprise:

identifying free computer processing unit cycles available on the device while the higher resolution image of the region of interest is displayed on a display output of the device; and

creating a second higher resolution image of at least a portion of the at least one image that is outside of the region of interest while the higher resolution image of the region of interest is displayed on the display output.

20. The non-transitory computer readable medium of claim 18 wherein the instructions further comprise:

displaying the higher resolution image of the region of interest as a preview image;

receiving a user input in response to the displaying of the preview image requesting an additional increase in resolution of the higher resolution image of the region of interest; and

creating an additional higher resolution image of the region of interest using the at least one image of the scene by;

creating an additional higher resolution grid associated with the additional higher resolution image of the region of interest;

identifying one or more high resolution grids comprising the high resolution grid;

determining an alignment between the additional higher resolution grid and each of the one or more high resolution grids; and

populating each point of the additional higher resolution grid with information from the corresponding aligned pixel from the one or more high resolution grids.

21. The non-transitory computer readable medium of claim 18 wherein the instructions further comprise:

displaying the higher resolution image of the region of interest as a preview image with a user interface for accepting or rejecting the higher resolution image of the region of interest.

22. A device for improving image quality for a user-selected region of interest comprising:

means for receiving at least one image of a scene;

means for receiving a user input identifying a region of interest, wherein the region of interest identifies a portion of the at least one image of the scene that is less than a complete area of the at least one image; and

means for creating a high resolution grid associated with the higher resolution image of the region of interest;

means for identifying one or more low resolution grids associated with the portion of the at least one image of the scene, wherein each low resolution grid is associated with one image, and wherein each point of each low resolution grid comprises information captured by the camera module;

means for determining an alignment between the high resolution grid and each of the one or more low resolution grids; and

means for populating points of the high resolution grid with information from a corresponding aligned pixel from the one or more low resolution grids to create a super resolved image of the region of interest.

23. The device of claim 22 further comprising:

means for displaying the at least one image of the scene and for displaying the super resolved image of the region of interest.

24. The method of claim 22 wherein the means for receiving at least one image of the scene comprises a wireless transceiver.

25. The method of claim 22 further comprising:

means for updating the super resolved image of the region of interest in response to a user input.

26. The method of claim 22 further comprising:

means for displaying a user interface for accepting or rejecting the super resolved image of the region of interest; and

means for storing the super resolution image of the region of interest in response to a user input accepting the super resolved image of the region of interest.