ENVIRONMENTAL SCENE CONDITION DETECTION

• US 20160232423A1
• Filed: 02/11/2015
• Published: 08/11/2016
• Est. Priority Date: 02/11/2015
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Xu, Shili, Biswas, Mainak, Zhong, Xin, Guo, Feng, Ren, Jianfeng, Yang, Yang, Gao, Dashan, Yang, Ruiduo
• Granted Patent: US 9,734,425 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B60H 1/00735 Control systems or circuits characterised by their input, i.e. by the detection, measurement or calculation of particular conditions, e.g. signal treatment, dynamic models
• CPC Class Codes: B60Q 1/20 Fog lights
• CPC Class Codes: B60S 1/0844 including a camera
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

A method of processing data includes receiving, at a computing device, data representative of an image captured by an image sensor. The method also includes determining a first scene clarity score. The method further includes determining whether the first scene clarity score satisfies a threshold, and if the first scene clarity score satisfies the threshold, determining a second scene clarity score based on second data extracted from the data.

Claims

1. A method for processing data, comprising:

at a computing device;

receiving data representative of an image captured by an image sensor, the image depicting a scene;

determining a first scene clarity score based on first data extracted from the data;

determining whether the first scene clarity score satisfies a threshold; and

if the first scene clarity score satisfies the threshold, determining a second scene clarity score based on second data extracted from the data.

2. The method of claim 1, further comprising determining whether the scene includes an environmental condition based on the second scene clarity score.

3. The method of claim 2, further comprising determining whether to initiate an action based on the second scene clarity score.

4. The method of claim 3, further comprising determining whether the environmental condition is present on a window of a vehicle.

5. The method of claim 4, further comprising activating a heating, ventilation, and air conditioning (HVAC) system or windshield wiper of the vehicle based on the determination of whether the environmental condition is present on the window of the vehicle.

6. The method of claim 3, wherein the first scene clarity score and the second scene clarity score are indicative of contrast in the image.

7. The method of claim 3, wherein the action includes activating a fog light of a vehicle, providing an audio warning, providing a visual warning, or a combination thereof.

8. The method of claim 3, wherein the action includes processing the data to produce clarified image data.

9. The method of claim 8, wherein producing the clarified image data includes using local tone mapping (LTM) to boost local contrast in high intensity regions of the image.

10. The method of claim 8, wherein intensity values of one or more pixels in the image are modified by amounts selected based on one or both of the first scene clarity score and the second scene clarity score.

11. The method of claim 8, further comprising performing object detection based on the clarified image data and providing an output to an output device of a vehicle based on the object detection.

12. The method of claim 1, wherein the first data includes dark feature data that corresponds to pixels in the image that have a high intensity value in at least one of a red channel, a green channel, and a blue channel of the image, and wherein the second data includes the dark feature data and gradient feature data.

13. An apparatus comprising:

a computing device, the computing device configured to;

receive data representative of an image captured by an image sensor, the image depicting a scene;

determine a first scene clarity score of the image based on first data extracted from the data;

determine whether the first scene clarity score satisfies a threshold; and

if the first scene clarity score satisfies the threshold determine a second scene clarity score based on second data extracted from the data.

14. The apparatus of claim 13, wherein the computing device is further configured to determine whether to initiate an action based on the second scene clarity score.

15. The apparatus of claim 14, wherein the computing device includes a local tone mapping (LTM) module, and wherein the action includes using the LTM module to generate clarified image data.

16. The apparatus of claim 15, wherein the clarified image data is generated based on additional data representative of a second image, the second image captured by the image sensor after the image sensor captured the image.

17. The apparatus of claim 15, wherein generating the clarified image includes modifying intensity values of one or more pixels to increase local contrast based on an average intensity value of neighboring pixels.

18. The apparatus of claim 17, wherein generating the clarified image includes reducing the intensity value of the pixel.

19. The apparatus of claim 15, wherein intensity values of one or more pixels in the image are modified by amounts selected based on one or both of the first scene clarity score and the second scene clarity score.

20. The apparatus of claim 19, wherein the amounts are positively correlated with one or both of the first scene clarity score and the second scene clarity score.

21. The apparatus of claim 15, further comprising a display device onboard a vehicle, the display device configured to display a clarified image represented by the clarified image data.

22. The apparatus of claim 15, wherein the computing device is configured to detect objects in the clarified image data.

23. The apparatus of claim 14, further comprising activating a heating, ventilation, and air conditioning (HVAC) system, a windshield wiper, or a fog light of a vehicle, providing an audio warning, providing a visual warning, or a combination thereof.

24. A computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform operations including:

receiving data representative of an image captured by an image sensor, the image depicting a scene;

determining a first scene clarity score based on first data extracted from the data;

determining whether the first scene clarity score satisfies a threshold and if the first scene clarity score satisfies the threshold;

determining a second scene clarity score based on second data extracted from the data; and

determining whether to initiate an action based on the second scene clarity score.

25. The computer-readable storage medium of claim 24, wherein the first scene clarity score and the second scene clarity score are calculated using supervised machine learning models.

26. The computer-readable storage medium of claim 25, wherein the first scene clarity score is calculated by comparing a first linear support vector model to the first data, and wherein the second scene clarity score is calculated by comparing a second linear support vector model to the second data.

27. The computer-readable storage medium of claim 26, wherein each of the first linear support vector model and the second linear support vector model is generated using sample videos.

28. An apparatus comprising:

means for receiving data representative of an image; and

means for image processing configured to;

determine a first scene clarity score of the image based on first data extracted from the data, the first scene clarity score indicative of contrast in the image;

determine whether the first scene clarity score satisfies a threshold and if the first scene clarity score satisfies the threshold;

determine a second scene clarity score based on second data extracted from the data; and

determine whether to automatically initiate an action based on the second scene clarity score.

29. The apparatus of claim 28, wherein the action includes processing the data to produce clarified image data.

30. The apparatus of claim 28, wherein the action includes activating a fog light of a vehicle, providing an audio warning, providing a visual warning, or a combination thereof.