Mobile Stereo Device: Stereo Imaging, Measurement and 3D Scene Reconstruction with Mobile Devices such as Tablet Computers and Smart Phones

• US 20130271579A1
• Filed: 04/14/2012
• Published: 10/17/2013
• Est. Priority Date: 04/14/2012
• Status: Abandoned Application

• Current Assignee: Younian Wang
• Original Assignee: Younian Wang
• Inventors: Wang, Younian
• Application Number:
  US13/447,175
• Publication Number:
  US 20130271579A1
• Time in Patent Office: Days
• US Class Current: 348/47
• CPC Class Codes: G01B 11/002 for measuring two or more coordinates
• CPC Class Codes: G01B 11/24 for measuring contours or curvatures
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

A mobile stereo imaging and processing device includes an ordinary mobile device such as tablet computers and smartphones; two or more digital cameras with stereo mounting requirements; a photography timing controller to operate the cameras, preferably with synchronized image capture capacity; a processor instruction package (software processing units) to control and operate the device, visualize and measure the images, compute and derive 3-dimentional object information from the captured 2-dimentional images; and some optional components such as remote control, GPS and IMU and optional processor instruction units for the control, operation and processing of the corresponding components.

Claims

1. An mobile stereo imaging system comprising of:

a. An ordinary mobile device such as any tablet computer or smartphone, which has the typical components and features such as (but not limited to) a processor, RAM, operating system, display unit (screen), digital storage media, Wi-Fi or 3G/4G wireless connection;

b. Two or more digital cameras with or without video capture capacity;

c. A controller unit to control and operate the cameras, preferably a photography timing controller for synchronized image capture from two or more cameras. Synchronized image capture is the preferable choice since it can collect 3D information for moving objects. However a simplified controller without synchronization feature can also be used to operate the cameras taking images for still object for 3D information collection;

d. An optional remote control for operating the cameras and data processing and transfer remotely;

e. An optional position sensor, such as a GPS (global positioning system) device to record the position of the cameras;

f. An optional attitude sensor, such as an IMU (inertial measurement unit) to record the attitude of the cameras;

g. An optional level bubble to keep the device horizontal;

h. An optional wired connection, such as a USB connection, to transfer images and data between the mobile device and other devices such as desktop computers.

i. The other necessary connection cables, control units and other pieces for the above described components to make the device working correctly.

2. Camera mounting methods in the mobile stereo imaging system:

a. The cameras should be mounted or installed in the mobile device with certain distance to each other. Example locations can be the middle of each site of the device or the corners of the device. At least two cameras will be used. In case of just two cameras are used, these two cameras should not be mounted immediately next to each other. That means certain distance between cameras has to be maintained. The longer the distance is, the better the accuracy it can achieve. The pointing direction of the cameras can be parallel, or with some inclined angles. The mounting locations and number of cameras to use can be very flexible. The only requirement is that the image from one camera should have some overlap with the image from its adjacent camera, but two cameras cannot be installed at the same location.

b. A typical configuration is to use two cameras with the pointing direction parallel to each other, and perpendicular to the line going through the centers of two cameras. This typical case will make the processing of the acquired images much simpler and faster. It is also the ideal configuration for stereo and 3d video capture. It has to be pointed out that mounting two cameras at almost same location, but pointing to opposite directions (i.e. pointing direction almost 180 degrees, such as with Apple iPad2) does not meet the mounting requirements of this invention, thus is not included in this invention, because it is impossible for these two cameras to capture overlapped images simultaneously.

3. A processor instruction (software) package in the mobile stereo imaging system which controls and operate the mobile devices and cameras for taking images and video;

viewing and processing the images and video in 2-dimentional or stereo mode;

measuring the images manually or automatically, computing and extracting 3-dimentional object information from the photographed 2-dimentional images. The software package can be specified in, but not limited to the following modules (can have more or less modules depending on its usage and applications);

a. A processor instruction module (software) for camera intrinsic and extrinsic parameters calibration programmed from publically available algorithms and methods or specifically designed or developed algorithms and methods. The camera intrinsic parameters include focal length, location of principal point, camera lens distortion parameters etc. The extrinsic parameters include the position of the camera center and the attitude of the camera image plane.

b. A processor instruction module (software) for controlling the camera operations.

c. A processor instruction module (software) for image viewing and measurement. The viewing and measurement can be made in 2-dimentional viewer (images are displayed separately) or in a stereo viewer where two overlapped images are displayed stereoscopically.

d. A processor instruction module (software) for automatically matching or recognizing the same points or other features on all images which are corresponding to the same object point or feature. For example a house corner is photographed and shown up on two images, the purpose of this module is to automatically recognize the location of the house corner on both images and report their image coordinates. This procedure is commonly called image matching. There are several algorithms publically available for image matching such as Normalized Cross Correlation.

e. A processor instruction module (software) which computes the 3-dimentional coordinates of the object space points from the camera parameters and measured image point coordinates. It also includes the computation of derived variables such as length, height, size, area and volume of an object and the detailed 3-dimentioanl digital surface model and representation of the object surface with point clouds.

f. Other optional processor instruction modules to control, operate and process the optional components such as GPS and IMU.

g. Other optional processor instruction modules for visualization, analysis, transfer or sharing of the stereoscopic images and derived data and results.