Automated matrix photo framing using range camera input

• US 10,122,997 B1
• Filed: 05/03/2017
• Issued: 11/06/2018
• Est. Priority Date: 05/03/2017
• Status: Active Grant

• Current Assignee: Lowe's Companies, Inc.
• Original Assignee: Lowe's Companies, Inc.
• Inventors: Sheffield, Mason E., Alexander, Oleg
• Primary Examiner(s): Czekaj, Dave
• Assistant Examiner(s): Joisil, Berteau
• Application Number:
  US15/586,207
• Publication Number:
  US 20180324401A1
• Time in Patent Office: 552 Days
• CPC Class Codes: G06T 17/10 Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/282 for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

Described herein are a system and methods for generating 3D models using imaging data obtained from an array of camera devices. In embodiments, the system may determine a depth or range between an object and the array of camera device. Based on this depth information, the system may automatically identify appropriate framing information for that array of camera devices based on operational constraints. One or more properties of the camera devices in the array of camera devices may then be adjusted in order to obtain image information in accordance with the identified framing information. The object may then be rotated in accordance with the operational constraints and the process may be repeated until a full set of images has been obtained.

Claims

1. A method of generating a 3D model by multiple camera devices, comprising:

determining at least a level of image overlap for an object to be represented by a 3D model;

identifying boundary information indicative of an outline of the object;

determining, based on the boundary information, a number of individual image boundaries for each image in a matrix of images to be captured by the multiple camera devices such that the matrix of images encompasses the outline and the images in the matrix of images overlap to an extent corresponding to the level of image overlap;

causing an imaging property of individual camera devices in the multiple camera devices to be adjusted to enable the multiple camera devices to capture respective images having the determined individual image boundaries; and

causing the multiple camera devices to capture the matrix of images.

2. The method of claim 1, wherein the boundary information is identified using machine vision.

3. The method of claim 1, wherein the imaging property of the individual camera devices comprises at least one of a pan, tilt, or level of magnification.

4. The method of claim 1, wherein the level of image overlap comprises a distance between at least two adjacent image boundaries for the number of individual image boundaries for the matrix of images.

5. The method of claim 1, wherein the level of image overlap comprises at least one of a number of pixels shared by adjacent images in the matrix of images or a percentage of image area shared by adjacent images in the matrix of images.

6. The method of claim 1, further comprising repositioning the object, wherein the method is repeated to obtain a complete set of matrices of images as determined based on a determined number of object rotations, wherein the complete set of matrices of images is captured within 90 second.

20. The method of claim 1, wherein the level of image overlap comprises at least some area shared by adjacent images in the matrix of images.

7. A 3D imaging system comprising:

an input sensor;

one or more camera devices;

a processor; and

a memory including instructions that, when executed with the processor, cause the system to, at least;

determine at least a level of image overlap for an item to be represented by a 3D model;

obtain input from the input sensor indicating depth information associated with the item;

determine at least one boundary indicative of an outline of the item;

adjust the one or more camera devices such that a matrix of images captured by the one or more camera devices encompasses the outline and the images in the matrix of images overlap to an extent corresponding to the level of image overlap; and

capture, using the one or more camera devices, image information using the adjustment.

8. The 3D imaging system of claim 7, wherein the input sensor comprises at least a depth sensor.

9. The 3D imaging system of claim 8, wherein the depth sensor comprises an imaging device that uses structured light to determine depth information.

10. The 3D imaging system of claim 9, wherein the boundary for the item is determined based on a variance in the depth information.

11. The 3D imaging system of claim 10, wherein the variance in depth information is determined to comprise a boundary when the variance in depth information exceeds a threshold variance level.

12. The 3D imaging system of claim 7, wherein adjustment of the one or more camera devices comprises adjusting a level of zoom.

13. A 3D imaging apparatus comprising:

an array of cameras configured to capture a matrix of images of an object; and

a control unit communicatively coupled to the array of cameras, the control unit configured to;

determine, via an interface, at least a level of image overlap for an object to be represented by a 3D model;

identify a boundary indicative of an outline of the object;

determine a number of image boundaries for images in the matrix of images from the identified boundary of the object such that the images in the matrix of images overlap to an extent corresponding to the level of image overlap; and

provide instructions to the array of cameras to cause it to capture the matrix of images with the image boundaries determined by the control unit.

14. The 3D imaging apparatus of claim 13, further comprising a staging device, wherein the control unit is further configured to cause the staging device to rotate the object.

15. The 3D imaging apparatus of claim 14, wherein the control unit is further configured to:

subsequent to causing the staging device to rotate the object, identify a second boundary of the object;

determine a number of second image boundaries from the identified second boundary of the object such that the images in the matrix of images overlap to an extent corresponding to the level of image overlap; and

provide instructions to the array of cameras to cause it to capture a second matrix of images with the second image boundaries determined by the control unit.

16. The 3D imaging apparatus of claim 13, wherein the 3D imaging apparatus is configured to cause the staging device to rotate the object by a determined number of degrees.

17. The 3D imaging apparatus of claim 13, wherein the number of degrees is determined based on a number of rotations indicated via the interface.

18. The 3D imaging apparatus of claim 13, wherein the number of image boundaries are identified from the boundary of the object by segmenting an area within the boundary.

19. The 3D imaging apparatus of claim 13, wherein the area within the boundary is segmented according to a number of cameras in the array of cameras.