System and method of adjusting the color of image objects based on chained reference points, gradient characterization, and pre-stored indicators of environmental lighting conditions

• US 9,204,018 B2
• Filed: 01/08/2015
• Issued: 12/01/2015
• Est. Priority Date: 01/21/2014
• Status: Expired due to Fees

• Current Assignee: Carbon Objects, Inc.
• Original Assignee: Carbon Objects, Inc.
• Inventors: Tomlinson, Allen Reed, O'Connell, Frances Allie
• Primary Examiner(s): BEKELE, MEKONEN T
• Application Number:
  US14/592,618
• Publication Number:
  US 20150207960A1
• Time in Patent Office: 327 Days
• Field of Search: 382/162, 382/167, 382/164, 358/1.9, 358/2.1, 345/581
• US Class Current: 1/1
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: H04N 1/6077 Colour balance, e.g. colour cast correction
• CPC Class Codes: H04N 1/6091 by environmental factors, e.g. temperature or humidity H04N1/6086, H04N1/6088 take precedence

Abstract

The invention relates to a system and method for adjusting the color of image objects based on chained reference points and/or light gradients to account for the effects of lighting conditions that can vary across different environments, according to an implementation of the invention. To chain reference points and/or characterize light gradients, the system may determine a color shift value, which represents the effects of lighting conditions of a target environment (in which the lighting conditions may be unknown) relative to a reference environment (in which the lighting conditions may be known). The chained reference points, gradient characterization, color shift values, and/or other information may be used to store a delta map that characterizes the lighting conditions of an environment so that it may be reused for subsequent color shifts without re-characterizing the environment.

Claims

1. A computer implemented method of adjusting color characteristics of image objects in association with an image of an environment, the environment having at least a first location having a first set of lighting conditions and a second location having a second set of lighting conditions, the method being implemented by a computer system having one or more physical processors programmed by computer program instructions stored on a non-transitory computer readable medium that, when executed by the one or more physical processors, program the computer system to perform the method, the method comprising:

identifying, by the computer system, a first marker in the first location, the first marker having a set of known color characteristics under known lighting conditions;

obtaining, by the computer system, a set of measured color characteristics of the first marker in the first location, the set of measured color characteristics being representative of effects of the first set of lighting conditions on the first marker;

comparing, by the computer system, the set of known color characteristics with the set of measured color characteristics;

determining, by the computer system, a first color shift value based on the comparison, wherein the first color shift value represents a difference between the known lighting conditions and the first set of lighting conditions;

identifying, by the computer system, a reference, different from the first marker, that exists in both the first location and the second location;

obtaining, by the computer system, a second set of measured color characteristics of the reference in the first location, the set of measured color characteristics being representative of effects of the first set of lighting conditions on the reference;

adjusting, by the computer system, the second set of measured color characteristics based on the first color shift value, wherein the adjusted second set of measured color is utilized to remove the effects of the first set of lighting conditions to generate an illuminant invariant version of the reference; and

obtaining, by the computer system, a third set of measured color characteristics of the reference in the second location, the third set of measured color characteristics being representative of effects of the second set of lighting conditions on the reference;

comparing, by the computer system, the adjusted second set of measured color characteristics with the third set of measured color characteristics;

determining, by the computer system, a second color shift value based on the comparison of the adjusted second set of measured color characteristics with the third set of measured color characteristics, wherein the second color shift value represents a difference between the first set of lighting conditions and the second set of lighting conditions.

2. The method of claim 1, the method further comprising:

obtaining, by the computer system, an image of an object not in the first location, the object having a set of color characteristics;

adjusting, by the computer system, the set of color characteristics of the object based on the first color shift value; and

generating, by the computer system, a simulated image of the object in the first location based on the adjusted set of color characteristics of the object to simulate the effects of the first set of lighting conditions on the object.

3. The method of claim 1, the method further comprising:

identifying, by the computer system, an object in the first location;

determining, by the computer system, a set of color characteristics of the object in the first location;

adjusting, by the computer system, the set of color characteristics based on the first color shift value; and

generating, by the computer system, a simulated image of the object that removes the effects of the first set of lighting conditions from the object.

4. The method of claim 1, the method further comprising:

identifying, by the computer system, an object in the second location;

determining, by the computer system, a set of color characteristics of the object in the second location;

adjusting, by the computer system, the set of color characteristics based on the second color shift value; and

generating, by the computer system, a simulated image of the object that removes the effects of the second set of lighting conditions from the object.

5. The method of claim 1, the method further comprising:

identifying, by the computer system, an object in the second location;

determining, by the computer system, a set of color characteristics of the object in the second location;

adjusting, by the computer system, the set of color characteristics based on the second color shift value; and

generating, by the computer system, a simulated image of the object that removes the effects of the second set of lighting conditions from the object.

6. The method of claim 1, the method further comprising:

determining, by the computer system, a first light intensity at the first location based on the first color shift value;

determining, by the computer system, a second light intensity at the second location based on the second color shift value;

determining, by the computer system, a light gradient of the environment based on the first light intensity and the second light intensity.

7. The method of claim 6, the method further comprising:

obtaining, by the computer system, an image of an object not in the environment, the object having a set of color characteristics;

adjusting, by the computer system, the set of color characteristics of the object based on the light gradient and a location at which the object is to be placed in the environment; and

generating, by the computer system, a first simulated image of the object based on the light gradient and a location at which the object is to be located and a second simulated image of the object based on the light gradient and another location at which the object is to be located to simulate the light gradient on the object as the object is moved from one location of the environment to another location of the environment.

8. The method of claim 6, the method further comprising:

extrapolating, by the computer system, a third light intensity at a third location of the environment based on the light gradient.

9. The method of claim 6, the method further comprising:

obtaining, by the computer system, identification information that identifies the environment;

storing, by the computer system in a memory, the light gradient in association with the identification information;

receiving, by the computer system, a request to simulate an appearance of an object in the environment, the request comprising the identification information;

obtaining, by the computer system from the memory, the light gradient; and

applying, by the computer system, the light gradient to the object.

10. A system of adjusting color characteristics of image objects in association with an image of an environment, the environment having at least a first location having a first set of lighting conditions and a second location having a second set of lighting conditions, the system comprising:

a computer system comprising one or more physical processors programmed by computer program instructions stored on a non-transitory computer readable medium that, when executed by the one or more physical processors, program the computer system to;

identify a first marker in the first location, the first marker having a set of known color characteristics under known lighting conditions;

obtain a set of measured color characteristics of the first marker in the first location, the set of measured color characteristics being representative of effects of the first set of lighting conditions on the first marker;

compare the set of known color characteristics with the set of measured color characteristics;

determine a first color shift value based on the comparison, wherein the first color shift value represents a difference between the known lighting conditions and the first set of lighting conditions;

identify a reference, different from the first marker, that exists in both the first location and the second location;

obtain a second set of measured color characteristics of the reference in the first location, the set of measured color characteristics being representative of effects of the first set of lighting conditions on the reference;

adjust the second set of measured color characteristics based on the first color shift value, wherein the adjusted second set of measured color is utilized to remove the effects of the first set of lighting conditions to generate an illuminant invariant version of the reference; and

obtain a third set of measured color characteristics of the reference in the second location, the third set of measured color characteristics being representative of effects of the second set of lighting conditions on the reference;

compare the adjusted second set of measured color characteristics with the third set of measured color characteristics,

determine a second color shift value based on the comparison of the adjusted second set of measured color characteristics with the third set of measured color characteristics, wherein the second color shift value represents a difference between the first set of lighting conditions and the second set of lighting conditions.

11. The system of claim 10, wherein the computer system is further programmed to:

obtain an image of an object not in the first location, the object having a set of color characteristics;

adjust the set of color characteristics of the object based on the first color shift value; and

generate a simulated image of the object in the first location based on the adjusted set of color characteristics of the object to simulate the effects of the first set of lighting conditions on the object.

12. The system of claim 10, wherein the computer system is further programmed to:

identify an object in the first location;

determine a set of color characteristics of the object in the first location;

adjust the set of color characteristics based on the first color shift value; and

generate a simulated image of the object that removes the effects of the first set of lighting conditions from the object.

13. The system of claim 10, wherein the computer system is further programmed to:

identify an object in the second location;

determine a set of color characteristics of the object in the second location;

adjust the set of color characteristics based on the second color shift value; and

generate a simulated image of the object that removes the effects of the second set of lighting conditions from the object.

14. The system of claim 10, wherein the computer system is further programmed to:

identify an object in the second location;

determine a set of color characteristics of the object in the second location;

adjust the set of color characteristics based on the second color shift value; and

generate a simulated image of the object that removes the effects of the second set of lighting conditions from the object.

15. The system of claim 10, wherein the computer system is further programmed to:

determine a first light intensity at the first location based on the first color shift value;

determine a second light intensity at the second location based on the second color shift value;

determine a light gradient of the environment based on the first light intensity and the second light intensity.

16. The system of claim 15, wherein the computer system is further programmed to:

obtain an image of an object not in the environment, the object having a set of color characteristics;

adjust the set of color characteristics of the object based on the light gradient and a location at which the object is to be placed in the environment; and

generate a first simulated image of the object based on the light gradient and a location at which the object is to be located and a second simulated image of the object based on the light gradient and another location at which the object is to be located to simulate the light gradient on the object as the object is moved from one location of the environment to another location of the environment.

17. The system of claim 15, wherein the computer system is further programmed to:

extrapolate a third light intensity at a third location of the environment based on the light gradient.

18. The system of claim 15, wherein the computer system is further programmed to:

obtain identification information that identifies the environment;

store, in a memory, the light gradient in association with the identification information;

receive a request to simulate an appearance of an object in the environment, the request comprising the identification information;

obtain, from the memory, the light gradient; and

apply the light gradient to the object.

19. A computer implemented method of adjusting color characteristics of image objects in association with an image of an environment, the environment having at least a first location having a first set of lighting conditions and a second location having a second set of lighting conditions, the method being implemented by a computer system having one or more physical processors programmed by computer program instructions stored on a non-transitory computer readable medium that, when executed by the one or more physical processors, program the computer system to perform the method, the method comprising:

identifying, by the computer system, a first marker in the first location, the first marker having a set of known color characteristics under known lighting conditions;

obtaining, by the computer system, a set of measured color characteristics of the first marker in the first location, the set of measured color characteristics being representative of effects of the first set of lighting conditions on the first marker;

comparing, by the computer system, the set of known color characteristics with the set of measured color characteristics;

determining, by the computer system, a first color shift value based on the comparison, wherein the first color shift value represents a difference between the known lighting conditions and the first set of lighting conditions;

identifying, by the computer system, a reference, different from the first marker, that exists in both the first location and the second location;

obtaining, by the computer system, a second set of measured color characteristics of the reference in the first location, the set of measured color characteristics being representative of effects of the first set of lighting conditions on the reference;

adjusting, by the computer system, the second set of measured color characteristics based on the first color shift value, wherein the adjusted second set of measured color is utilized to remove the effects of the first set of lighting conditions to generate an illuminant invariant version of the reference;

obtaining, by the computer system, an image of an object not in the first location, the object having a set of color characteristics;

adjusting, by the computer system, the set of color characteristics of the object based on the first color shift value; and

generating, by the computer system, a simulated image of the object in the first location based on the adjusted set of color characteristics of the object to simulate the effects of the first set of lighting conditions on the object.