DISTRIBUTING PROCESSING FOR IMAGING PROCESSING

• US 20150172539A1
• Filed: 12/17/2013
• Published: 06/18/2015
• Est. Priority Date: 12/17/2013
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Neglur, Smita
• Granted Patent: US 9,402,018 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/667 Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

Approaches are described for managing the processing of images or video on a computing device. A portable computing device can include one or more dedicated components, such as an application-specific integrated circuit (ASIC) or other dedicated processor component, to be integrated into the computing device to perform at least a portion of the imaging processing of captured images or video. For example, the dedicated processor component can enable the offloading of basic image signal processing, as well as higher level or “machine vision” processing from the device processor of the device. In this way, the dedicated processor component can perform signal processing for which the input is an image (or video), and where image or video data can be analyzed, interpreted and/or manipulated to generate an output, the output of image processing being either an image or a set of characteristics or parameters related to the image. The output can be provided to a device processor for further processing.

Claims

1. A computing device, comprising:

a computing device processor;

a camera configured to capture an image; and

an application-specific integrated circuit (ASIC) in communication with the camera, wherein at least a portion of image computational processing is performed by the ASIC, the ASIC configured to at least;

analyze the image to determine an area of interest,

define a portion of the image that includes the area of interest, and

in response to determining that image computational processing satisfies a predetermined threshold, cause a transfer of the portion of the image to the computing device processor for further processing.

2. The computing device of claim 1, wherein the ASIC further includes:

an image signal processor (ISP) configured to perform image correction on the image, the ISP connected to a dedicated image bus that is capable of providing the image to the computing device processor; and

a digital signal processor (DSP) configured to analyze the image to determine machine vision information, the machine vision information associated with at least one of gesture tracking, head tracking, face tracking, object tracking, object recognition, one or more image fusion algorithms, one or more sensor fusion algorithms, or stereo processing of the image.

3. The computing device of claim 1, wherein upon receiving the portion of the image from the ASIC, the computing device processor:

analyzes the image to identify a plurality of feature points;

generates, based at least in part upon the plurality of feature points, a plurality of feature vectors representative of an object in the image; and

identifies the object based at least in part on the plurality of feature vectors.

4. A computing device, comprising:

a dedicated processor operable to be connected to a first camera, the dedicated processor configured to identify, from an image acquired using the first camera, an area of interest in the image, the dedicated processor being further configured to generate machine vision information based on the area of interest; and

a general purpose processor configured to receive, from the dedicated processor and in response to processing by the dedicated processor satisfying a predetermined threshold, the machine vision information and a portion of the image corresponding to the area of interest, the general purpose processor being further configured to perform at least one process on the portion of the image corresponding to the area of interest.

5. The computing device of claim 4, further comprising:

a combiner configured to receive images from a plurality of cameras of the computing device, wherein the dedicated processor is capable of processing the images from the plurality of cameras while retaining source information associated with each image, the source information identifying a camera of the plurality of cameras that was used to acquire a respective image.

6. The computing device of claim 4, wherein the general purpose processor is further configured to:

analyze the portion of the image corresponding to the area of interest to identify a plurality of feature points;

generate, based at least in part upon the plurality of feature points, a plurality of feature vectors representative of an object in the image; and

identify the object based at least in part on the plurality of feature vectors.

7. The computing device of claim 4, wherein the dedicated processor is a first application-specific integrated circuit (ASIC), and wherein the computing device further includes a second camera, the first camera being connected to the first ASIC and the second camera being connected to a second ASIC, wherein the general purpose processor is configured to receive image data from the first ASIC and the second ASIC.

8. The computing device of claim 4, wherein the dedicated processor is an application-specific integrated circuit (ASIC), and wherein the computing device further includes a second camera, the first camera and the second camera being connected to the ASIC, wherein the ASIC is configured to receive image data from the first camera and the second camera and process a first image from the first camera and a second image from second camera.

10. The computing device of claim 4, wherein the dedicated processor further includes:

memory storing a configuration file, the dedicated processor configured to use the configuration file to perform one or more image signal processing algorithms.

11. The computing device of claim 4, wherein the dedicated processor includes an image signal processor (ISP) that performs at least one of:

dead pixel correction, lens correction, geometric correction, data compression, generating a histogram of pixel information, automatic exposure control, automatic gain control, automatic white balance, de-mosaicing, edge enhancement, image thresholding, image smoothing or image binarization.

12. The computing device of claim 4, wherein the general purpose processor performs at least one of:

gesture detection, object tracking, object recognition, stereo processing of two or more images, three dimensional (3D) image capture, image metadata tagging, time-stamping, geographical tagging, symbol reading, depth mapping, image rectification, edge detection, image scaling, object character recognition, audio recognition, or video recognition.

13. The computing device of claim 4, wherein the dedicated processor is operable to run one or more image computational processes in parallel, and wherein the one or more image computational processes include one of gesture detection, object detection, object recognition, or image enhancement.

14. The computing device of claim 4, wherein the dedicated processor is further configured to process at least one of video information and audio information, and wherein the area of interest includes at least one of a head of a user of the computing device, a facial feature of the user, a hand of the user, a foot of the user, an object, audio information, character information, or video information.

15. The computing device of claim 4, wherein at least a portion of machine vision processing is offloaded from the general purpose processor to the dedicated processor, thereby at least partly reducing power consumption of the general purpose processor.

9. The computing device of 8, wherein the ASIC is configured to:

receive a selection of one of the first camera and the second camera;

retrieve processing information from a configuration file based upon the selection, the processing information indicating one or more image signal processing algorithms or settings;

process an image captured from the selected one of the first camera and the second camera using the one or more image signal processing algorithms or settings; and

initiate a transfer of the image from the selected one of the first camera and the second camera to the general purpose processor.

16. A method, comprising:

receiving an image at a dedicated processor from a first camera, the dedicated processor operable to be connected to the first camera;

identifying, by the dedicated processor, an area of interest in the image,

generating, by the dedicated processor, machine vision information based on the area of interest;

determining that processing by the dedicated processor satisfies a predetermined threshold; and

receiving the machine vision information and a portion of the image corresponding to the area of interest from the dedicated processor at a general purpose processor of a computing device, the general purpose processor being configured to perform at least one process on the portion of the image corresponding to the area of interest.

17. The method of claim 16, wherein the dedicated processor is an application-specific integrated circuit (ASIC), wherein the computing device further includes a second camera, and the method further comprising:

receiving a selection of one of the first camera or a second camera;

retrieving processing information from a configuration file based upon the selection, the processing information indicating one or more image signal processing algorithms or settings;

processing, by the dedicated processor, an image captured from the selected one of the first camera or the second camera using the one or more image signal processing algorithms or settings; and

imitating a transfer of the processed image to the general purpose processor.

18. The method of claim 16, further comprising:

accessing a combiner by the dedicated processor; and

selecting, by the combiner, between one of the first camera or a second camera, wherein upon selection of one of the first camera or the second camera, the dedicated processor is configured to process captured data from the selected one of first camera or the second camera using one or more image signal processing algorithms or settings.

19. The method of claim 16, wherein the at least one process includes:

analyzing the portion of the image corresponding to the area of interest to identify a plurality of feature points;

generating, based at least in part upon the plurality of feature points, a plurality of feature vectors representative of an object in the image; and

identifying the object based at least in part on the plurality of feature vectors.

20. The method of claim 16, wherein the dedicated processor is operable to run one or more image computational processes in parallel, and wherein the one or more image computational processes include one of gesture detection, object detection, object recognition, or image enhancement.