Rendering information into images

• US 10,032,263 B2
• Filed: 06/12/2017
• Issued: 07/24/2018
• Est. Priority Date: 06/12/2016
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Hendry, Ian C., Gnaegy, John C.
• Primary Examiner(s): Sajous, Wesner
• Application Number:
  US15/620,514
• Publication Number:
  US 20170358064A1
• Time in Patent Office: 407 Days
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 5/92 based on global image properties
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/90 Determination of colour characteristics

Abstract

Systems, methods, and computer readable media to improve the operation of a display system are disclosed. Techniques disclosed herein selectively darken a region of an image so that when text or other information is rendered into that region, the contrast between the text or other information and the underlying image in that area is sufficient to ensure the text or other information is visible and readable. In one embodiment, a region into which information is to be rendered may be combined or blended with tone mapped values of those same pixels in accordance with a given function, where the function gives more weight to the tone mapped pixel values the closer those pixels are to the midline of the region and more weight to untone-mapped image pixel values the further those pixels are from the midline of the region.

Claims

1. A computer implemented method to display information in an image, comprising:

accessing an image in a memory, the image comprising pixels, each pixel having an image value;

identifying a region in the image, the region having a centerline and outer edges and encompassing a plurality of pixels of the image;

identifying a function that varies from an initial value at the region'"'"'s centerline to a final value at an outer edge of the region;

determining, for at least some of the pixels in the region—

the pixel'"'"'s image value,

a location of the pixel,

a first value for the pixel based on the function and the pixel'"'"'s location,

a tone map value for the pixel based on a tone map function and the pixel'"'"'s image value, and

a region value for the pixel based on the pixel'"'"'s image value, the pixel'"'"'s tone map value and the pixel'"'"'s first value;

updating, in the memory corresponding to the region, the image value of each pixel with the pixel'"'"'s corresponding region value;

rendering, after updating, information into the memory corresponding to the region to generate a revised image; and

displaying the revised image on a display device.

2. The method of claim 1, wherein the region value of each image pixel comprises:

the image pixel'"'"'s corresponding image pixel value along the region'"'"'s outer edges; and

the image pixel'"'"'s corresponding tone map value along the region'"'"'s centerline.

3. The method of claim 1, wherein the tone map function asymptotically approaches a value less than 1.0.

4. The method of claim 1, wherein determining the region value for the pixel comprises:

determining an intermediate pixel value based on the pixel'"'"'s tone map value and the pixel'"'"'s first value; and

blending the intermediate pixel value and the pixel'"'"'s image value.

5. The method of claim 4, wherein blending comprises:

determining an average value of the intermediate pixel value and the pixel'"'"'s image value; and

replacing the pixel'"'"'s value with the average value.

6. The method of claim 1, wherein rendering comprises rendering the information in a color over the region'"'"'s updated pixel values.

7. The method of claim 6, wherein the information comprises one or more of textual information and graphical information.

18. The method of claim 1, further comprising, before displaying:

determining a second tone map value for a second plurality of pixels of the image, wherein the second plurality of pixels is not within the region; and

updating, in the memory, the image value of each of the second plurality of pixels corresponding to each pixel'"'"'s corresponding second tone map value.

8. A computer system comprising:

a display element;

a memory operatively coupled to the display element; and

one or more processing elements coupled to the display element and the memory, the one or more processing elements configured to execute program instructions stored in the memory to cause the computer system to—

access an image in the memory, the image having pixels, each pixel having an image value,

identify a region in the image, the region having a centerline and outer edges and encompassing a plurality of pixels of the image,

identify a function that varies from an initial value at the region'"'"'s centerline to a final value at an outer edge of the region,

determine, for at least some of the pixels in the region—

the pixel'"'"'s image value,

a location of the pixel,

a first value for the pixel based on the function and the pixel'"'"'s location,

a tone map value for the pixel based on a tone map function and the pixel'"'"'s image value, and

a region value for the pixel based on the pixel'"'"'s image value, the pixel'"'"'s tone map value and the pixel'"'"'s first value;

update, in the memory corresponding to the region, the image value of each pixel with the pixel'"'"'s region value;

render, after the update, information into the memory corresponding to the region to generate a revised image; and

display the revised image on the display element.

9. The computer system of claim 8, wherein the region value of each image pixel comprises:

the image pixel'"'"'s corresponding image pixel value along the region'"'"'s outer edges; and

the image pixel'"'"'s corresponding tone map value along the region'"'"'s centerline.

10. The computer system of claim 8, wherein program instructions stored in the memory further cause the computer system to—

determine an intermediate pixel value based on the pixel'"'"'s tone map value and the pixel'"'"'s first value; and

blend the intermediate pixel value and the pixel'"'"'s image value.

11. The computer system of claim 10, wherein the program instructions stored in the memory to cause the computer system to blend comprise program instructions to cause the computer system to:

determine an average value of the intermediate pixel value and the pixel'"'"'s image value; and

replace the pixel'"'"'s value with the average value.

12. The computer system of claim 8, wherein the program instructions stored in the memory to cause the computer system to render comprise program instructions to cause the computer system to render the information in a color over the region'"'"'s updated pixel values.

13. The computer system of claim 12, wherein the information comprises one or more of textual information and graphical information.

19. The computer system of claim 8, further comprising program instructions to cause the computer system, before the one or more processors display, to:

determine a second tone map value for a second plurality of pixels of the image, wherein the second plurality of pixels is not within the region; and

update, in the memory, the image value of each of the second plurality of pixels corresponding to each pixel'"'"'s corresponding second tone map value.

14. A non-transitory program storage device comprising instructions stored thereon to cause one or more processors to:

access an image in a memory, the image having pixels, each pixel having an image value;

identify a region in the image, the region having a centerline and outer edges and encompassing a plurality of pixels of the image;

identify a function that varies from an initial value at the region'"'"'s centerline to a final value at an outer edge of the region;

determine, for at least some of the pixels in the region—

the pixel'"'"'s image value,

a location of the pixel,

a first value for the pixel based on the function and the pixel'"'"'s location,

a tone map value for the pixel based on a tone map function and the pixel'"'"'s image value, and

a region value for the pixel based on the pixel'"'"'s image value, the pixel'"'"'s tone map value and the pixel'"'"'s first value;

update, in the memory corresponding to the region, the image value of each pixel with the pixel'"'"'s region value;

render, after the update, information into the memory corresponding to the region to generate a revised image; and

display the revised image on a display device.

15. The non-transitory program storage device of claim 14, further comprising instructions to cause the one or more processors to:

determine an intermediate pixel value based on the pixel'"'"'s tone map value and the pixel'"'"'s first value; and

blend the intermediate pixel value and the pixel'"'"'s image value.

16. The non-transitory program storage device of claim 15, wherein the instructions to cause the one or more processors to blend comprise instructions to cause the one or more processors to:

determine an average value of the intermediate pixel value and the pixel'"'"'s image value; and

replace the pixel'"'"'s value with the average value.

17. The non-transitory program storage device of claim 14, wherein the information comprises one or more of textual information and graphical information.

20. The non-transitory program storage device of claim 14, further comprising instructions to cause the one or more processors, before the one or more processors display, to:

determine a second tone map value for a second plurality of pixels of the image, wherein the second plurality of pixels is not within the region; and

update, in the memory, the image value of each of the second plurality of pixels corresponding to each pixel'"'"'s corresponding second tone map value.