SENSOR NOISE PROFILE

• US 20160180541A1
• Filed: 12/18/2015
• Published: 06/23/2016
• Est. Priority Date: 12/19/2014
• Status: Active Grant

• Current Assignee: Arm Limited (SoftBank Group Corp.)
• Original Assignee: Apical Limited (SoftBank Group Corp.)
• Inventors: ROMANENKO, Ilya
• Granted Patent: US 9,892,517 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 2218/04 Denoising
• CPC Class Codes: G06T 2200/28 involving image processing hardware
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20024 Filtering details
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06V 10/449 Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/467 Encoded features or binary features, e.g. local binary patterns [LBP]
• CPC Class Codes: G06V 10/82 using neural networks

Abstract

The invention relates to feature extraction technique based on edge extraction. It can be used in computer vision systems, including image/facial/object recognition systems, scene interpretation, classification and captioning systems. A model or profile of the noise in the sensor is used to improve feature extraction or object detection on an image from a sensor.

Claims

1. A method of extracting a feature from an image, comprising the processor or circuitry implemented steps of:

(a) providing a digital image from a sensor;

(b) using a model or profile of the noise in the sensor to improve feature extraction or object detection on the image.

2. Method of claim 1 where the feature is an edge.

3. Method of claim 1 where the feature is a local binary pattern.

4. Method of claim 1 where the model or profile of the sensor noise is used to normalize feature extraction response.

5. Method of claim 1 where the variance of the sensor noise in defined areas of the image are used to normalize feature extraction response.

6. Method of claim 1 where the feature extraction or object detection is based on edge detection.

7. Method of claim 1 and that includes the steps of, for each pixel of the input image, calculating an edge response for an orientation, and normalising the edge response for an orientation by taken into account the noise variance.

8. Method of claim 1, implemented in a system that is not part of, does not use, or is not downstream of, an image processing pipeline.

9. Method of claim 1, operating in the RAW domain with linear data.

10. Method of claim 1, in which the edge response is calculated by convoluting a filter kernel with the intensity of the image.

11. Method of claim 1, in which the filter is a Gabor filter or a CNN filter.

12. Method of claim 1, in which the normalization of the edge response for an orientation a is calculated for each pixel (x,y) in the image from:

13. Method of claim 1, in which the input image is the RAW image sensor data.

14. Method of claim 1, in which the image edge response is fed into a linear classifier such as an SVM or into a classification layer of a CNN.

15. Method of claim 1, implemented in real-time.

16. Method of claim 1, operating as a computer vision system, applied to posture detection, people detection, object detection.

17. Method of claim 1 when used in one of the following:

Smartphone;

Computer vision systems;

Objects recognition systems;

Human detection systems;

Facial recognition systems;

Scene interpretation systems;

Image classification systems;

Image captioning systems;

Autonomous vehicle computer vision systems;

Robotics computer vision systems.

18. Method of claim 1 when implemented in embedded hardware, such as a hardware block.

19. Image processing hardware configured to receive a digital image from a sensor and use a model or profile of the noise in the sensor to improve feature extraction or object detection on the image.

20. A device including image processing hardware configured to receive a digital image from a sensor and use a model or profile of the noise in the sensor to improve feature extraction or object detection on the image.

21. The device of claim 20, being or including one of the following:

Smartphone;

Computer vision system;

Objects recognition system;

Human detection system;

Facial recognition system;

Scene interpretation system;

Image classification system;

Image captioning system;

Autonomous vehicle computer vision system;

Robotics computer vision system.