SELECTIVE COLORIZATION USING MONOCHROMIC IMAGERY

• US 20170094258A1
• Filed: 09/25/2015
• Published: 03/30/2017
• Est. Priority Date: 09/25/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Cooper, Ted
• Granted Patent: US 10,110,880 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 2209/048 having several pick-up sensors
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/125 Colour sequential image capture, e.g. using a colour wheel
• CPC Class Codes: H04N 23/13 with multiple sensors
• CPC Class Codes: H04N 23/667 Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes

Abstract

An unmanned aerial vehicle (UAV) may include stereo cameras that include a left camera and a right camera. The cameras may capture a majority of images in black and white, which may be captured in a single color channel. At certain times during a flight, the UAV may be configured to generate colorized images. To generate colorized images, the camera may capture a first image using a default color channel (e.g., green), which may be used for black and white images, and then selectively apply a color filter to capture a second image associated with a different color channel than the first image. A multi-channel color image may be created using at least the first image and the second image. By adding a third image in the remaining color channel, the created image may be generated as a red-green-blue image.

Claims

1. An unmanned aerial vehicle (UAV) equipped with a stereo camera system, the UAV comprising:

a first image sensor;

a first electronic color filter that selectively transitions between a first transparent state and a blue filter state;

a second image sensor;

a second electronic color filter that selectively transitions between a second transparent state and a red filter state;

one or more processors; and

memory to store computer-executable instructions that, when executed, cause the one or more processors to perform acts comprising;

capturing, at a first time and via the first image sensor, a first image while the first electronic color filter is in the first transparent state, the first image being a black and white image associated with a green color channel;

determining a triggering event to initiate generation of a red-green-blue (RGB) image;

capturing, at a second time and via the first image sensor, a second image while the first electronic color filter is in a blue filter state;

capturing, between the first time and the second time and via the second image sensor, a third image while the second electronic color filter is in a red filter state; and

generating the RGB image using at least the first image, the second image, and the third image.

2. The UAV as recited in claim 1, wherein the acts further comprise:

capturing, at a third time and via the first image sensor, a fourth image while the first electronic color filter is in the first transparent state; and

averaging the first image and the fourth image to create an averaged image, and

wherein the generating the RGB image includes using the averaged image, the second image, and the third image.

3. The UAV as recited in claim 1, wherein the triggering event is associated with at least one of a change in navigation that is greater than a threshold change or a proximity to a way point.

4. The UAV as recited in claim 1, wherein the triggering event is associated with a change of attributes between consecutive images that is greater than a threshold change.

5. The UAV as recited in claim 1, wherein the triggering event is associated with at least one of a proximity to a destination or deposit of a package at the destination.

6. A method comprising:

capturing, using an image sensor, a first image as a black and white image associated with a first color channel;

detecting a trigger event; and

in response to detecting the triggering event;

causing light to pass through a color filter associated with a second color channel that is different than the first color channel;

capturing a second image associated with the second color channel; and

creating a multi-channel color image using the first image and the second image.

7. The method as recited in claim 6, further comprising, in response to the trigger event:

causing light to pass through a different color filter associated with a third color channel that is different than the first color channel and different than the second color channel; and

capturing a third image associated with the third color channel; and

wherein the creating the multi-channel color image further includes using the third image, the first image, and the second image to create a red-green-blue (RGB) image.

8. The method as recited in claim 6, further comprising transmitting a signal to an electronic color filter to cause the electronic color filter to transition from a transparent state to a color filter state.

9. The method as recited in claim 6, further comprising transmitting a signal to a motor to rotate a filter wheel that includes at least a transparent portion and the color filter.

10. The method as recited in claim 6, further comprising transmitting a signal to an actuator to actuate a filter slider to move between at least a first position and a second position, wherein the first position includes the color filter.

11. The method as recited in claim 6, wherein the second image is captured before the first image.

12. The method as recited in claim 6, wherein the trigger is determined based in part on at least one of an obtained waypoint, a duration of a predetermined about of time, a detection of an unknown object, or proximity to a destination.

13. The method as recited in claim 6, wherein the trigger is determined based at least in part on execution of a navigation control, the navigational control being at least one of a change in heading that is greater than a threshold heading change or a change in elevation that is greater than a threshold elevation change.

14. The method as recited in claim 6, wherein the trigger is determined based in part on a comparison of an amount of change of at least one attribute between the first image and a prior image.

15. The method as recited in claim 14, wherein the attribute is at least one of an amount of light, a histogram, or a change in pixel values.

16. A monochromatic camera comprising:

an image sensor;

a lens; and

a color filter device that selectively filters light before the light reaches the image sensor,

wherein the camera is configured to;

capture, via the image sensor, a first image in a first color channel while the color filter does not filter the light before the light reaches the image sensor;

receive a signal to cause the color filter to filter the light;

capture, via the image sensor, a second image in a second color channel that is different than the first color channel, while the color filter filters the light before the light reaches the image sensor.

17. The monochromatic camera as recited in claim 16, further comprising a motor, and wherein the color filter includes a filter wheel that rotates under power of the motor to cause the color filter to align with the lens and the image sensor.

18. The monochromatic camera as recited in claim 16, further comprising an actuator, and wherein the color filter includes a filter slider that moves under power of the actuator to cause the color filter to align with the lens and the image sensor.

19. The monochromatic camera as recited in claim 16, wherein the color filter is an electronic color filter that transitions, in response to the signal, from a transparent state to a color filter state.

20. The monochromatic camera as recited in claim 16, further comprising an infrared component that is configured to selectively create infrared imagery using at least the lens of the camera, and wherein the camera is configured to cause the infrared component to capture the infrared imagery.