Image processor with evaluation layer implementing software and hardware algorithms of different precision

• US 9,323,995 B2
• Filed: 09/13/2013
• Issued: 04/26/2016
• Est. Priority Date: 03/11/2013
• Status: Active Grant

• Current Assignee: Avago Technologies International Sales Pte Limited (Broadcom, Inc.)
• Original Assignee: Avago Technologies General IP PTE Limited (Broadcom, Inc.)
• Inventors: Aliseychik, Pavel A., Mazurenko, Ivan L., Letunovskiy, Aleksey A., Petyushko, Alexander A., Kholodenko, Alexander B.
• Primary Examiner(s): Kassa, Yosef
• Application Number:
  US14/238,522
• Publication Number:
  US 20160004919A1
• Time in Patent Office: 956 Days
• Field of Search: 382/103, 382/107, 382/206, 382/209, 382/218, 382/284, 382/294, 358/537, 358/540, 358/450
• US Class Current: 1/1
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2200/21 involving computational photography
• CPC Class Codes: G06T 2207/20048 Transform domain processing
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation
• CPC Class Codes: G06T 2219/2004 Aligning objects, relative positioning of parts
• CPC Class Codes: G06T 3/02 Affine transformations for image registration G06T3/147; for image mosaicing G06T3/4038
• CPC Class Codes: G06T 3/403 Edge-driven scaling; Edge-based scaling
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/181 involving edge growing; involving edge linking
• CPC Class Codes: G06V 10/955 using specific electronic processors
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/113 Recognition of static hand signs

Abstract

An image processor comprises image processing circuitry implementing a plurality of processing layers including at least an evaluation layer and a recognition layer. The evaluation layer comprises a software-implemented portion and a hardware-implemented portion, with the software-implemented portion of the evaluation layer being configured to generate first object data of a first precision level using a software algorithm, and the hardware-implemented portion of the evaluation layer being configured to generate second object data of a second precision level lower than the first precision level using a hardware algorithm. The evaluation layer further comprises a signal combiner configured to combine the first and second object data to generate output object data for delivery to the recognition layer. By way of example only, the evaluation layer may be implemented in the form of an evaluation subsystem of a gesture recognition system of the image processor.

Claims

1. An image processor comprising:

image processing circuitry implementing a plurality of processing layers including at least an evaluation layer and a recognition layer;

the evaluation layer comprising a software-implemented portion and a hardware-implemented portion;

the software-implemented portion of the evaluation layer being configured to generate first object data of a first precision level using a software algorithm;

the hardware-implemented portion of the evaluation layer being configured to generate second object data of a second precision level lower than the first precision level using a-hardware algorithm;

wherein the evaluation layer further comprises a signal combiner configured to combine the first and second object data to generate output object data for delivery to the recognition layer.

2. The image processor of claim 1 wherein the evaluation layer comprises an evaluation subsystem of a gesture recognition system.

3. The image processor of claim 1 wherein the plurality of processing layers further comprises a preprocessing layer that receives raw image data and provides preprocessed image data to the evaluation layer.

4. The image processor of claim 1 wherein the first object data is updated at a first rate and the second object data is updated at a second rate that is higher than the first rate.

5. The image processor of claim 1 wherein the first rate is less than once per input frame of the received image data and the-second rate is once per input frame.

6. The image processor of claim 1 wherein the first object data is generated based on a comparison between a current input frame and at least one object model.

7. The image processor of claim 1 wherein the second object data is generated based on a comparison between a current input frame and at least one previous input frame.

8. The image processor of claim 1 wherein the evaluation layer implements search-render-compare operations in generating the first and second object data.

9. The image processor of claim 8 wherein separate search modules associated with the search-render-compare operations are instantiated in each of the software-implemented portion and the hardware-implemented portion.

10. The image processor of claim 8 wherein a render module associated with the search-render-compare operations is instantiated only in the software-implemented portion and a compare module associated with the search-render-compare operations is instantiated only in the hardware-implemented portion.

11. The image processor of claim 1 wherein the software-implemented portion of the evaluation layer implements a plurality of high-level processing operations including one or more of singular point detection and tracking, frontal surface detection and tracking, edge detection and tracking, and fast heuristics such as mass center detection.

12. The image processor of claim 1 wherein the hardware-implemented portion of the evaluation layer implements a plurality of low-level image processing primitives including one or more of a 3D primitive such as an affine or isometric transformation, a primitive such as curve tracking, a comparison primitive such as least squares comparison, a search primitive such as motion prediction or gradient search, and a statistics primitive.

13. The image processor of claim 1 wherein the image processing circuitry comprises at least one graphics processor integrated circuit.

14. An integrated circuit comprising the image processor of claim 1.

15. A method comprising:

configuring a plurality of processing layers of an image processor including at least an evaluation layer and a recognition layer;

generating first object data of a first precision level using a software algorithm in a software-implemented portion of the evaluation layer;

generating second object data of a second precision level lower than the first precision level using a hardware algorithm in a hardware-implemented portion of the evaluation layer; and

combining the first and second object data to generate output object data for delivery to the recognition layer.

16. The method of claim 15 wherein generating first and second object data comprises:

generating the first object data based on a comparison between a current input frame and at least one object model; and

generating the second object data based on a comparison between a current input frame and at least one previous input frame;

wherein the first object data is updated at a first rate and the second object data is updated at a second rate that is higher than the first rate.

17. A non-transitory computer-readable storage medium having computer program code embodied therein, wherein the computer program code when executed in a processing device causes the processing device to perform the method of claim 15.

18. An image processing system comprising:

one or more image sources providing image data;

one or more image destinations; and

an image processor coupled to said one or more image sources and said one or more image destinations;

wherein the image processor comprises;

image processing circuitry implementing a plurality of processing layers including at least an evaluation layer and a recognition layer;

the evaluation layer comprising a software-implemented portion and a hardware-implemented portion;

the software-implemented portion of the evaluation layer being configured to generate first object data of a first precision level using a software algorithm;

the hardware-implemented portion of the evaluation layer being configured to generate second object data of a second precision level lower than the first precision level using a hardware algorithm;

wherein the evaluation layer further comprises a signal combiner configured to combine the first and second object data to generate output object data for delivery to the recognition layer.

19. The system of claim 18 wherein at least one of the one or more image sources comprises a depth sensor.

20. The system of claim 19 herein the depth sensor is part of a depth imager that incorporates the image processor.