Content-aware image resizing using superpixels

• US 9,824,421 B2
• Filed: 01/25/2012
• Issued: 11/21/2017
• Est. Priority Date: 01/25/2012
• Status: Active Grant

• Current Assignee: Tahoe Research Limited (f/k/a Learndale Limited) (Vector Capital Corporation)
• Original Assignee: Intel Corporation
• Inventors: Ragozin, Dmitry, Sapatov, Alexander
• Primary Examiner(s): WU, XIAO MIN
• Assistant Examiner(s): Lhymn, Sarah
• Application Number:
  US13/977,279
• Publication Number:
  US 20140028727A1
• Time in Patent Office: 2,127 Days
• Field of Search: 345619, 345621, 345625, 345627, 345628, 345639, 345642, 345643, 345645, 345660, 345661, 345664, 345665, 345666, 345668, 345670, 345671, 345682, 382173, 382276, 382277, 382282
• CPC Class Codes: G06T 3/04 Context-preserving transformations, e.g. by using an importance map panospheric to cylindrical image transformations G06T3/12
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting

Abstract

Illustrative embodiments of methods, machine-readable media, and computing devices that provide content-aware image resizing using superpixels are disclosed. In some embodiments, a method may include segmenting a machine-readable image file into a plurality of superpixels, where each of the plurality of superpixels corresponds to a plurality of pixels of the machine-readable image file. The method may also include selecting a target region in the machine-readable image file, where the target region includes the pixels corresponding to a seam of superpixels extending across a dimension of the machine-readable image file. The method may further include selecting a seam of pixels in the target region, where the seam of pixels extends across the dimension of the machine-readable image file, and resizing the machine-readable image file by removing or augmenting the seam of pixels.

Claims

1. A computing device comprising:

a camera; and

an image co-processor to (i) segment an image captured by the camera into a plurality of superpixels, each of the plurality of superpixels corresponding to a plurality of pixels of the image, (ii) select a seam of superpixels in the image extending across a dimension of the image, (iii) select a seam of pixels within the seam of superpixels, the seam of pixels extending across the dimension of the image, and (iv) resize the image by removing or augmenting the seam of pixels.

2. The computing device of claim 1, wherein to segment the image into the plurality of superpixels comprises to:

generate a point cloud comprising a plurality of points that each correspond to a plurality of pixels of the image; and

apply a watershed algorithm to the point cloud to generate the plurality of superpixels.

3. The computing device of claim 2, wherein to generate the point cloud comprises to generate a plurality of points that each correspond to ten or more pixels of the image.

4. The computing device of claim 1, wherein the image co-processor is further to re-segment the image into a new plurality of superpixels, after resizing the image.

5. The computing device of claim 1, wherein to select the seam of superpixels in the image comprises to:

assign an energy value to each of the plurality of superpixels; and

determine a seam of superpixels having a minimal seam energy using the assigned energy values.

6. The computing device of claim 5, wherein to assign an energy value to each of the plurality of superpixels comprises to:

assign a low energy value to superpixels that are similar to neighboring superpixels; and

assign a high energy value to superpixels that are dissimilar to neighboring superpixels.

7. The computing device of claim 5, wherein the image co-processor is further to assign a new energy value to each superpixel belonging to the seam of superpixels and to each superpixel adjacent to the seam of superpixels, after resizing the image.

8. The computing device of claim 1, wherein to select the seam of pixels within the seam of superpixels comprises to:

assign an energy value to each of the pixels within the seam of superpixels; and

determine a seam of pixels within the seam of superpixels having a minimal seam energy using the assigned energy values.

9. The computing device of claim 7, wherein to assign an energy value to each of the pixels within the seam of superpixels comprises to:

assign a low energy value to pixels that are similar to neighboring pixels; and

assign a high energy value to pixels that are dissimilar to neighboring pixels.

10. A method comprising:

segmenting a machine-readable image file into a plurality of superpixels, each of the plurality of superpixels corresponding to a plurality of pixels of the machine-readable image file;

selecting a seam of superpixels in the machine-readable image file extending across a dimension of the machine-readable image file;

selecting a seam of pixels within the seam of superpixels, the seam of pixels extending across the dimension of the machine-readable image file; and

resizing the machine-readable image file by removing or augmenting the seam of pixels.

11. The method of claim 10, wherein segmenting the machine-readable image file into the plurality of superpixels comprises:

generating a point cloud comprising a plurality of points that each correspond to a plurality of pixels of the machine-readable image file; and

applying a watershed algorithm to the point cloud to generate the plurality of superpixels.

12. The method of claim 11, wherein generating the point cloud comprises generating a plurality of points that each correspond to ten or more pixels of the machine-readable image file.

13. The method of claim 10, further comprising re-segmenting the machine-readable image file into a new plurality of superpixels, after resizing the machine-readable image file.

14. The method of claim 10, wherein selecting the seam of superpixels in the machine-readable image file comprises:

assigning an energy value to each of the plurality of superpixels; and

determining a seam of superpixels having a minimal seam energy using the assigned energy values.

15. The method of claim 14, further comprising assigning a new energy value to each superpixel belonging to the seam of superpixels and to each superpixel adjacent to the seam of superpixels, after resizing the machine-readable image file.

16. The method of claim 10, wherein selecting the seam of pixels within the seam of superpixels comprises:

assigning an energy value to each of the pixels within the seam of superpixels; and

determining a seam of pixels within the seam of superpixels having a minimal seam energy using the assigned energy values.

17. One or more non-transitory machine-readable media comprising a plurality of instructions that, in response to being executed, cause a processor of a computing device to:

segment a machine-readable image file into a plurality of superpixels, each of the plurality of superpixels corresponding to a plurality of pixels of the machine-readable image file;

select a seam of superpixels in the machine-readable image file extending across a dimension of the machine-readable image file;

select a seam of pixels within the seam of superpixels, the seam of pixels extending across the dimension of the machine-readable image file; and

resize the machine-readable image file by removing or augmenting the seam of pixels.

18. The one or more non-transitory machine-readable media of claim 17, wherein the plurality of instructions further cause the processor to re-segment the machine-readable image file into a new plurality of superpixels, after resizing the machine-readable image file.

19. The one or more non-transitory machine-readable media of claim 17, wherein to select the seam of superpixels in the machine-readable image file comprises to:

assign an energy value to each of the plurality of superpixels; and

determine a seam of superpixels having a minimal seam energy using the assigned energy values.

20. The one or more non-transitory machine-readable media of claim 19, wherein to assign an energy value to each of the plurality of superpixels comprises to:

assign a low energy value to superpixels that are similar to neighboring superpixels; and

assign a high energy value to superpixels that are dissimilar to neighboring superpixels.

21. The one or more non-transitory machine-readable media of claim 19, wherein the plurality of instructions further cause the processor to assign a new energy value to each superpixel belonging to the seam of superpixels and to each superpixel adjacent to the seam of superpixels, after resizing the machine-readable image file.

22. The one or more non-transitory machine-readable media of claim 17, wherein to select the seam of pixels within the seam of superpixels comprises to:

assign an energy value to each of the pixels within the seam of superpixels; and

determine a seam of pixels within the seam of superpixels having a minimal seam energy using the assigned energy values.

23. The one or more non-transitory machine-readable media of claim 22, wherein to assign an energy value to each of the pixels within the seam of superpixels comprises to:

assign a low energy value to pixels that are similar to neighboring pixels; and

assign a high energy value to pixels that are dissimilar to neighboring pixels.

24. The one or more non-transitory machine-readable media of claim 17, wherein to segment the machine-readable image file into the plurality of superpixels comprises to:

generate a point cloud comprising a plurality of points that each correspond to a plurality of pixels of the machine-readable image file; and

apply a watershed algorithm to the point cloud to generate the plurality of superpixels.

25. The one or non-transitory more machine-readable media of claim 24, wherein to generate the point cloud comprises to generate a plurality of points that each correspond to ten or more pixels of the machine-readable image file.