Post-manufacture camera calibration

• US 9,892,514 B2
• Filed: 10/10/2014
• Issued: 02/13/2018
• Est. Priority Date: 10/10/2014
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Maguire, Yael, Shen, Shizhe
• Primary Examiner(s): Yenke, Brian
• Assistant Examiner(s): Khalid, Omer
• Application Number:
  US14/511,579
• Publication Number:
  US 20160104284A1
• Time in Patent Office: 1,222 Days
• Field of Search: 348180, 348187, 348188, 348189, 455 6711, 455 677, 382254, 382255, 382276, 382309
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: H04N 1/00 Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
• CPC Class Codes: H04N 1/00172 directly from a still digital camera or from a storage medium mounted in a still digital camera H04N1/0018 takes precedence
• CPC Class Codes: H04N 17/002 for television cameras
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet

Abstract

Some embodiments include a method of operating a calibration server for a camera module. The method can include: receiving, by the computing server, a first training image taken by the camera module in a mobile device and a corresponding image-context attribute from the mobile device; aggregating, by the computing device, the first training image into a set of contextually similar images based on the image-context attribute; computing a calibration parameter model based on the set of contextually similar images utilizing dimension reduction statistical analysis; and scheduling to update the calibration parameter model to configure an image processor to adjust a raw photograph of the camera module according to the calibration parameter model.

Claims

1. A computer-implemented method comprising:

receiving, at a computer server system from a mobile device, a first training image taken by a camera module of the mobile device and an image-context attribute describing a context associated with the first training image, wherein the image-context attribute is separate from the first training image;

adding, by the computer server system, the first training image into a sub-group of training images in accordance with a similarity based, at least in part, on the image-context attribute;

in response to determining that a training set of images including the sub-group provides a threshold amount of data samples to compute a calibration parameter model, computing, by the computer server system, the calibration parameter model based, at least in part, on the sub-group of training images utilizing dimension reduction statistical analysis, wherein the calibration parameter model configures at least an image adjustment process corresponding to the image-context attribute and wherein the image adjustment process is configured to define how to post-process a raw photograph after being captured by the camera module; and

causing, by the computer server system, an image processor to calibrate the raw photograph of the camera module by providing the calibration parameter model to the image processor.

2. The computer-implemented method of claim 1, further comprising providing a current version of the calibration parameter model to the image processor for deployment;

and wherein the image processor is part of the mobile device; and

wherein the image processor is configured to process raw photographs taken by the camera module prior to saving or displaying the raw photographs.

3. The computer-implemented method of claim 1, wherein the image processor is part of the computer server system that performs said computing of the calibration parameter model;

wherein the computing server system is configured as an online storage of raw photographs taken by the camera module of the mobile device; and

wherein the image processor is configured to adjust automatically the raw photographs in the online storage and subsequently uploaded photographs according to the calibration parameter model.

4. The computer-implemented method of claim 1, wherein the image processor is part of an external server separate from the computer server system and the mobile device;

and further comprising;

receiving, at the external server, the raw photograph taken by the camera module from the mobile device; and

processing, by the external server, the raw photograph based on the calibration parameter model to present in a network accessible interface.

5. The computer-implemented method of claim 1, wherein computing the calibration parameter model is in response to receiving the first training image.

6. The computer-implemented method of claim 1, wherein computing the calibration parameter model is according to a preset schedule.

7. The computer-implemented method of claim 1, further comprising:

adjusting a test image based on a current version of the calibration parameter model to determine whether the calibration parameter model has improved in accuracy over a previous version of the calibration parameter; and

wherein providing the calibration parameter model for the image processor is in response to determining that the current version has improved in accuracy.

8. The computer-implemented method of claim 1, wherein the first training image is a low spatial frequency image.

9. A non-transitory computer-readable data storage medium storing computer-executable instructions that, when executed, cause a computer system to perform a computer-implemented method, the computer-executable instructions comprising:

instructions for receiving, at a computer server system from a mobile device, a first training image taken by a camera module of the mobile device and an image-context attribute describing a context associated with the first training image, wherein the image-context attribute is separate from the first training image;

instructions for adding, by a computer server system, the first training image into a sub-group of training images in accordance with a similarity based, at least in part, on the image-context attribute;

instructions for in response to determining that a training set of images including the sub-group provides a threshold amount of data samples to compute a calibration parameter model, computing, by the computer server system, the calibration parameter model based, at least in part, on the subgroup of training images utilizing dimension reduction statistical analysis, wherein the calibration parameter model configures at least an image adjustment process corresponding to the image-context attribute and wherein the image adjustment process is configured to define how to post-process a raw photograph after being captured by the camera module; and

instructions for causing, by a computer server system, an image processor to calibrate the raw photograph of the camera module by providing the calibration parameter model to the image processor.

10. The non-transitory computer-readable data storage medium of claim 9, wherein the computer-executable instructions further comprises instructions for providing a current version of the calibration parameter model to the image processor for deployment;

and wherein the image processor is part of the mobile device; and

wherein the image processor is configured to process raw photographs taken by the camera module prior to saving or displaying the raw photographs.

11. The non-transitory computer-readable data storage medium of claim 9, wherein the image processor is part of the computer server system that performs said computing of the calibration parameter model;

wherein the computing server system is configured as an online storage of raw photographs taken by the camera module of the mobile device; and

wherein the image processor is configured to adjust automatically the raw photographs in the online storage and subsequently uploaded photographs according to the calibration parameter model.

12. The non-transitory computer-readable data storage medium of claim 9, wherein the image processor is part of an external server separate from the computer server system and the mobile device;

and wherein the computer-executable instructions further comprises;

instructions for receiving, at the external server, the raw photograph taken by the camera module from the mobile device; and

instructions for processing, by the external server, the raw photograph based on the calibration parameter model to present in a network accessible interface.

13. The non-transitory computer-readable data storage medium of claim 9, wherein computing the calibration parameter model is in response to receiving the first training image.

14. The non-transitory computer-readable data storage medium of claim 9, wherein computing the calibration parameter model is according to a preset schedule.

15. The non-transitory computer-readable data storage medium of claim 9, wherein the computer-executable instructions further comprises:

instructions for adjusting a test image based on a current version of the calibration parameter model to determine whether the calibration parameter model has improved in accuracy over a previous version of the calibration parameter; and

wherein providing the calibration parameter model for the image processor is in response to determining that the current version has improved in accuracy.

16. A computer system, comprising:

a non-transitory memory device configured to store executable instructions;

a processor configured by the executable instructions to;

receive, from a mobile device, a first training image taken by a camera module of the mobile device and an image-context attribute describing a context associated with the first training image, wherein the image-context attribute is separate from the first training image;

add the first training image into a sub-group of training images in accordance with a similarity based, at least in part, on the image-context attribute;

in response to determining that a training set of images including the sub-group provides a threshold amount of data samples to compute a calibration parameter model, compute the calibration parameter model based, at least in part, on the sub-group of training images utilizing dimension reduction statistical analysis, wherein the calibration parameter model configures at least an image adjustment process corresponding to the image-context attribute and wherein the image adjustment process is configured to define how to post-process a raw photograph after being captured by the camera module; and

cause an image processor to calibrate the raw photograph of the camera module by providing the calibration parameter model to the image processor.