Data processing device for processing multiple sensor data and system including the same

• US 9,846,919 B2
• Filed: 12/22/2015
• Issued: 12/19/2017
• Est. Priority Date: 02/16/2015
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kim, Young Jun, Lee, Jin Aeon
• Primary Examiner(s): Wu, Jingge
• Application Number:
  US14/977,975
• Publication Number:
  US 20160239937A1
• Time in Patent Office: 728 Days
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement

Abstract

Provided is a data processing device and a method of operating the same. The data processing device includes: a plurality of preprocessors configured to perform correction processing on a plurality of sensor data; a first switching circuit configured to selectively map and input the plurality of sensor data from at least two sensors to at least two preprocessors among the plurality of preprocessors; and a hybrid data processing engine configured to perform at least one of image enhancement and depth information determination on the plurality of sensor data received, via an on-the-fly method, from the at least two preprocessors.

Claims

1. A data processing device comprising:

a plurality of preprocessors configured to perform correction processing on a plurality of sensor data;

a first switching circuit configured to selectively map and input the plurality of sensor data from at least two sensors to at least two preprocessors among the plurality of preprocessors; and

a hybrid data processing engine configured to perform at least one of image enhancement and depth information determination on the plurality of sensor data received, via an on-the-fly method, from the at least two preprocessors,

wherein a first preprocessor, among the plurality of preprocessors, comprises a first rectification engine configured to rectify first sensor data input to the first preprocessor based on second sensor data input to a second preprocessor, among the plurality of preprocessors.

2. The data processing device according to claim 1, wherein the first switching circuit is configured to selectively map and input the plurality of sensor data from at least two different types of sensors.

3. The data processing device according to claim 1, wherein the first switching circuit is configured to selectively map and input the plurality of sensor data from the at least two sensors to the at least two preprocessors according to types of the plurality of sensor data.

4. The data processing device according to claim 3, wherein when the first sensor data from a first sensor and the second sensor data from a second sensor are input to the first switching circuit, the first switching circuit is configured to map and input the first sensor data and the second sensor data to the first preprocessor and the second preprocessor, respectively, from among the plurality of preprocessors.

5. The data processing device according to claim 4, wherein when the second sensor data from the second sensor and third sensor data from a third sensor are input to the first switching circuit, the first switching circuit is configured to map and input the second sensor data and the third sensor data to the first preprocessor and the second preprocessor, respectively.

6. The data processing device according to claim 1, wherein each of the plurality of preprocessors comprises at least one of:

a lens shading corrector configured to correct a difference between light and shade caused by lens shading;

a bad pixel corrector configured to correct a bad pixel; and

a chromatic aberration corrector configured to correct a chromatic aberration.

7. The data processing device according to claim 1, wherein the hybrid data processing engine comprises a plurality of hardware elements configured to perform the at least one of the image enhancement and the depth information determination.

8. The data processing device according to claim 1, further comprising a plurality of converters configured to selectively convert at least one of resolutions and pixel formats of the plurality of sensor data received from the at least two preprocessors.

9. A method of processing a plurality of sensor data by a data processing device, the method comprising:

selectively mapping and inputting the plurality of sensor data from at least two sensors to at least two preprocessors among a plurality of preprocessors of the data processing device;

performing, by the at least two preprocessors, correction processing on the plurality of sensor data; and

performing at least one of image enhancement and depth information determination on the plurality of sensor data received, via an on-the-fly method, from the at least two preprocessors,

wherein the method further comprises rectifying, by a first preprocessor among the plurality of preprocessors, first sensor data input to the first preprocessor based on second sensor data input to a second preprocessor, among the plurality of preprocessors.

10. The method according to claim 9, wherein the selectively mapping and inputting the plurality of sensor data comprises selectively mapping and inputting the plurality of sensor data from at least two different types of sensors.

11. The method according to claim 9, wherein the selectively mapping and inputting the plurality of sensor data comprises selectively mapping and inputting the plurality of sensor data from the at least two sensors to the at least two preprocessors according to types of the plurality of sensor data.

12. The method according to claim 11, wherein the selectively mapping and inputting the plurality of sensor data according to types of the plurality of sensor data comprises, when the first sensor data from a first sensor and the second sensor data from a second sensor are received, mapping and inputting the first sensor data and the second sensor data to the first preprocessor and the second preprocessor, respectively, from among the plurality of preprocessors.

13. The method according to claim 9, wherein the performing the correction processing comprises correcting, by each of the at least two preprocessors, at least one of a bad pixel, a chromatic aberration, and a difference between light and shade caused by lens shading.

14. The method according to claim 9, wherein the performing the at least one of the image enhancement and the depth information determination comprises performing, by a plurality of hardware elements, the at least one of the image enhancement and the depth information determination.

15. A data processing device comprising:

a plurality of preprocessors configured to perform correction processing on a plurality of sensor data to generate a plurality of correction-processed sensor data;

a first switching circuit configured to selectively map and input the plurality of sensor data from at least two sensors to at least two preprocessors among the plurality of preprocessors; and

a hybrid data processing engine configured to obtain the plurality of correction-processed sensor data from an external memory, and to perform at least one of image enhancement and depth information determination on the plurality of correction-processed sensor data,

wherein a first preprocessor, among the plurality of preprocessors, comprises a first rectification engine configured to rectify first sensor data input to the first preprocessor based on second sensor data input to a second preprocessor, among the plurality of preprocessors.

16. The data processing device according to claim 15, wherein the first switching circuit is configured to selectively map and input the plurality of sensor data from at least two different types of sensors.

17. The data processing device according to claim 15, wherein the first switching circuit is configured to selectively map and input the plurality of sensor data from the at least two sensors to the at least two preprocessors according to types of the plurality of sensor data.

18. The data processing device according to claim 15, wherein each of the plurality of preprocessors comprises at least one of:

a lens shading corrector configured to correct a difference between light and shade caused by lens shading;

a bad pixel corrector configured to correct a bad pixel; and

a chromatic aberration corrector configured to correct a chromatic aberration.

19. The data processing device according to claim 15, wherein the hybrid data processing engine comprises a plurality of hardware elements configured to perform the at least one of the image enhancement and the depth information determination.