Scene imaging method using a portable two-camera omni-imaging device for human-reachable environments

• US 9,071,752 B2
• Filed: 09/25/2012
• Issued: 06/30/2015
• Est. Priority Date: 09/25/2012
• Status: Active Grant

• Current Assignee: National Chiao Tung University (Government of The Republic of China)
• Original Assignee: National Chiao Tung University (Government of The Republic of China)
• Inventors: Tsai, Wen-Hsiang, Kuo, Yu-Tung
• Primary Examiner(s): Lee, Y
• Assistant Examiner(s): Haghani, Shadan E
• Application Number:
  US13/626,607
• Publication Number:
  US 20140085408A1
• Time in Patent Office: 1,008 Days
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 3/12 Panospheric to cylindrical image transformations
• CPC Class Codes: G08B 13/19628 of wide angled cameras and camera groups, e.g. omni-directional cameras, fish eye, single units having multiple cameras achieving a wide angle view
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

A scene imaging method using a portable two-camera omni-imaging device for human-reachable environments is disclosed. Firstly, an upper omni-image and a lower omni-image of at least one scene are captured. Next, image inpainting and image dehazing perform on the upper omni-image and the lower omni-image. Next, image unwrapping performs on the upper omni-image and the lower omni-image by a pano-mapping table, so as to form an upper panoramic image and a lower panoramic image respectively, and the upper panoramic image and the lower panoramic image respectively have an upper part and a lower part overlapping each other. Finally, a data item on the upper part and the lower part is obtained by view points of the scene and the pano-mapping table, thereby stitching the upper and lower panoramic images.

Claims

1. A scene imaging method using a portable two-camera omni-imaging device for human-reachable environments, which is processed by a processor, comprising steps of:

a two-camera omni-imaging device consisting of a first omni-camera and a second omni-camera respectively capturing an upper omni-image and a lower omni-image of at least one scene;

performing image inpainting and image dehazing on said upper omni-image and said lower omni-image;

performing image unwrapping on said upper omni-image and said lower omni-image by a pano-mapping table, so as to form an upper panoramic image and a lower panoramic image respectively, and said upper panoramic image and said lower panoramic image respectively have an upper part and a lower part overlapping each other;

obtaining a data item on said upper part and said lower part by a plurality of view points of said scene and said pano-mapping table; and

stitching said upper panoramic image and said lower panoramic image to form a total panoramic image according to said data, wherein

said first omni-camera and said second omni-camera are aligned coaxially in a back-to-back fashion, and

said step of obtaining said data by said view points and said pano-mapping table further comprises steps of;

establishing a plurality of first matching lines on said upper omni-image and said lower omni-image by said view points;

using said pano-mapping table to transform said first matching lines, so as to obtain a plurality of corresponding second matching lines on said upper panoramic image and said lower panoramic image, and said second matching lines comprise a plurality of second upper matching lines on said upper part and a plurality of second lower matching lines on said lower part, and said second upper matching lines respectively correspond to said second lower matching lines, and each said second upper matching line corresponds to one said view point;

performing edge detection on said second upper matching lines and said second lower matching lines, so as to obtain a plurality of upper feature points on said second upper matching lines and a plurality of lower feature points on said second lower matching lines;

defining that said upper feature points and said lower feature points matching each other respectively have substitution costs, and that said upper feature points that said lower feature points don'"'"'t match respectively have insertion costs, and that said lower feature points that said upper feature points don'"'"'t match respectively have deletion costs;

performing dynamic programming technique on said upper feature points and said lower feature points according said substitution costs, said insertion costs, and said deletion costs, so as to obtain a minimum cost sequence as an optimal matching point sequence, and said optimal matching point sequence comprises parts of said upper feature points and parts of said lower feature points matching each other, and said parts of said upper feature points are used as upper matching points, said parts of said lower feature points are used as lower matching points;

determining whether light rays of each said upper matching point and each said lower matching point matching each other intersect on a region that a first view of said first omni-camera overlaps a second view of said second omni-camera;

if no, deleting determined said upper matching point and determined said lower matching point, and then executing a step of choosing two said upper matching points on same said second upper matching line and two said lower matching points on same said second lower matching line, which match each other; and

if yes, executing the step of choosing two said upper matching points on same said second upper matching line and two said lower matching points on same said second lower matching line, which match each other;

determining whether a difference of a first interval and a second interval is less than a first predetermined value, and said first interval is a largest interval between said two said upper matching points, and said second interval is a largest interval between said two said lower matching points;

if no, deleting said two said upper matching points and said two said lower matching points, and then executing a step of choosing at least two adjacent said upper matching points on different said second upper matching lines and at least two adjacent said lower matching points on different said second lower matching lines, which match each other; and

if yes, executing the step of choosing at least two adjacent said upper matching points on different said second upper matching lines and at least two adjacent said lower matching points on different said second lower matching lines, which match each other;

determining whether a difference of a first distance and a second distance is less than a second predetermined value, and said first distance is a vertical distance between said two adjacent said upper matching points, and said second distance is a vertical distance between said two adjacent said lower matching points;

if no, deleting determined said upper matching points corresponding to said first distance and said lower matching points corresponding to said second distance, and then executing a step of determining whether at least one said upper matching point and at least one said lower matching point are deleted; and

if yes, executing the step of determining whether at least one said upper matching point and at least one said lower matching point are deleted; and

as a result of the step of determining whether at least one said upper matching point and at least one said lower matching point are deleted, if no, regarding said upper matching points and said lower matching points as said data; and

if yes, with deleted said upper matching point being used as a first replaceable matching point and deleted said lower matching point being used as a second replaceable matching point, choosing two neighboring said upper matching points at two sides of said first replaceable matching point and two neighboring said lower matching points at two sides of said second replaceable matching point, which match each other, and building a first connection line connecting with said two neighboring said upper matching points and intersecting one said second upper matching line to generate a first intersection, and building a second connection line connecting with said two neighboring said lower matching points and intersecting one said second lower matching line to generate a second intersection, and regarding said upper matching points and said lower matching points on said upper part and said lower part, said first intersection, and said second intersection as said data.

2. The scene imaging method using the portable two-camera omni-imaging device for human-reachable environments according to claim 1, wherein said first omni-camera comprises a first projective camera and a first hyperboloidal-shaped mirror, and said second omni-camera comprises a second projective camera and a second hyperboloidal-shaped mirror, and wherein said first hyperboloidal-shaped mirror and said second hyperboloidal-shaped mirror are aligned coaxially in said back-to-back fashion, and said first projective camera is disposed above said first hyperboloidal-shaped mirror, and said second projective camera is disposed below said second hyperboloidal-shaped mirror.

3. The scene imaging method using the portable two-camera omni-imaging device for human-reachable environments according to claim 2, wherein said pano-mapping table is obtained by rotation-invariant property of said first omni-camera and said second omni-camera, and a relationship between an elevation angle of a projecting point in real-world space and radiuses of mapping points on said first hyperboloidal-shaped mirror and said second hyperboloidal-shaped mirror.

4. The scene imaging method using the portable two-camera omni-imaging device for human-reachable environments according to claim 1, wherein said substitution cost is expressed by |H(u)−

H(l)|/H_max,and wherein H(u) is a hue value of said upper feature point, H(l) is a hue value of said lower feature point, and H_max is a maximum hue value, and wherein said insertion cost is 0.5, and said deletion cost is 0.5.

5. The scene imaging method using the portable two-camera omni-imaging device for human-reachable environments according to claim 1, wherein in said step of establishing said first matching lines by said view points, said first matching lines are established by said view points and rotation-invariant property of said first omni-camera and said second omni-camera.

6. The scene imaging method using the portable two-camera omni-imaging device for human-reachable environments according to claim 1, wherein said at least one scene is a plurality of scene spot, said method further comprising steps of:

recording said upper matching points, said lower matching points, said first intersections, said second intersections, said upper omni-image, said lower omni-image, said upper panoramic image, said lower panoramic image, said total panoramic image, and geographical coordinates of each said scene spot;

transforming each said total panoramic image into a perspective-view image by a perspective-mapping table; and

establishing an environment three-view diagram by said geographical coordinates of each said scene.

7. The scene imaging method using the portable two-camera omni-imaging device for human-reachable environments according to claim 6, further comprising a step of changing a viewpoint through said total panoramic image, said perspective-view image, or said environment three-view diagram.