Stereo-panoramic data

• US 10,061,189 B2
• Filed: 09/05/2012
• Issued: 08/28/2018
• Est. Priority Date: 09/06/2011
• Status: Active Grant

• Current Assignee: King Abdullah University of Science and Technology
• Original Assignee: King Abdullah University of Science and Technology
• Inventors: Cutchin, Steve
• Primary Examiner(s): Harold, Jefferey F
• Assistant Examiner(s): Karwan, Sihar
• Application Number:
  US13/604,077
• Publication Number:
  US 20130057643A1
• Time in Patent Office: 2,183 Days
• Field of Search: 348 36, 702 2
• CPC Class Codes: G03B 35/08 by simultaneous recording
• CPC Class Codes: G03B 37/02 with scanning movement of lens or cameras
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects

Abstract

Systems and methods for automatically generating three-dimensional panoramic images for use in various virtual reality settings are disclosed. One embodiment of the system includes a stereo camera capture device (SCD), a programmable camera controller (PCC) that rotates, orients, and controls the SCD, a robotic maneuvering platform (RMP), and a path and adaptation controller (PAC). In that embodiment, the PAC determines the movement of the system based on an original desired path and input gathered from the SCD during an image capture process.

Claims

1. A method, comprising the steps of:

(a) accessing a program having a desired path through a physical area;

(b) instructing a robotic maneuvering platform (RMP) to move to a start location on the desired path;

(c) instructing a programmable camera controller (PCC) to execute capture of stereoscopic panoramic data at the start location;

(d) receiving stereoscopic panoramic data from the start location;

(e) deciding whether additional data-capture locations along the desired path exist; and

(f) responsive to deciding that additional data-capture locations exist along the desired path;

(f1) determining a stereocomplexity (Sc) from the received stereoscopic panoramic data, the Sc being based at least in part upon a norm of a stereo-differential between stereo pairs of the stereoscopic panoramic data;

(f2) determining a step size to a subsequent location on the desired path, the step size being a function of the determined Sc;

(f3) determining the subsequent location on the desired path based on the determined step size;

(f4) instructing the RMP to move to the determined subsequent location on the desired path;

(f5) instructing the PCC to execute capture of stereoscopic panoramic data at the subsequent location;

(f6) receiving stereoscopic panoramic data from the subsequent location; and

(f7) repeating the process from step (e).

2. A method, comprising:

determining a stereocomplexity (Sc) of stereoscopic panoramic data, wherein the Sc is based at least in part upon a norm of a stereo-differential between stereo pairs of the stereoscopic panoramic data;

determining a step size based on the Sc;

determining a data-capture location on a desired path based on the determined Sc, wherein in the data-capture location is determined based on the step size; and

instructing a robotic maneuvering platform (RMP) to move to the data-capture location.

3. The method of claim 2, the step of instructing the RMP to move to the data-capture location comprising:

instructing the RMP to move to a subsequent data capture location along the desired path in an increment that is equal to the step size.

4. The method of claim 2, the step of determining the Sc comprising:

comparing left-camera data of the stereoscopic panoramic data with right-camera data of the stereoscopic panoramic data;

determining non-overlapping data components between the left-camera data of the stereoscopic panoramic data and the right-camera data of the stereoscopic panoramic data; and

filtering the non-overlapping data components.

5. The method of claim 4, the step of determining the Sc further comprising:

comparing a left-eye matrix norm and a right-eye matrix norm as a function of the filtered non-overlapping data components.

6. The method of claim 2, the step of determining the Sc comprising:

computing j_i=∥

α

_i∥

₁=max_yΣ

_y-1ⁿ|α

_xy|;

computing j_r=∥

α

_r∥

₁=max_yΣ

_y-1ⁿ|α

_xy|;

computing j_l=∥

α

_l∥

_∞

=max_xΣ

_x-1ⁿ|α

_xy|; and

computing j_r=∥

α

_r∥

_∞

=max_xΣ

_x-1ⁿ|α

_xy|,

wherein;

α

xy denotes pixels in a matrix;

j_l denotes a norm of a left eye matrix;

j_r denotes a norm of a right eye matrix

i denotes a location;

j denotes a stereo pair; and

k denotes a number of pairs of stereo images at location i.

7. The method of claim 6, the step of determining the Sc further comprising:

computing Sc=max(Sct,Sc),

wherein Sct=abs(j_r−

j_l).

8. The method of claim 7, the step of determining the step size further comprising:

computing;

Step Size=(1/Sc)(d),

wherein d denotes a normalized distance factor.

9. The method of claim 8, the normalized distance factor (c) being based on an interocular distance, the d further being based on a distance of an entire desired path, the d further being based on a focal distance.

10. The method of claim 2, further comprising:

receiving the stereoscopic panoramic data.

11. The method of claim 10, the step of determining the Sc comprising:

determining the Sc from the received stereoscopic panoramic data.

12. The method of claim 2, further comprising producing a plurality of stereo-panoramic images at the data-capture location.

13. The method of claim 6, further comprising producing a plurality of stereo-panoramic images.

14. The method of claim 7, further comprising producing a plurality of stereo-panoramic.

15. The method of claim 8, further comprising producing a plurality of stereo-panoramic.

16. The method of claim 3, comprising:

instructing a programmable camera controller (PCC) to execute capture of stereoscopic panoramic data at the subsequent location.

17. The method of claim 16, comprising:

receiving stereoscopic panoramic data from the subsequent location.

18. The method of claim 17, comprising:

accessing a program having the desired path through a physical area;

instructing the robotic maneuvering platform (RMP) to move to a start location on the desired path;

instructing the programmable camera controller (PCC) to execute capture of stereoscopic panoramic data at the start location;

receiving the stereoscopic panoramic data from the start location;

deciding whether additional data-capture locations along the desired path exist; and

responsive to deciding that the additional data-capture locations exist along the desired path;

determining the stereocomplexity (Sc) from the received stereoscopic panoramic data.

19. The method of claim 4, wherein the left-camera data of the stereoscopic panoramic data and the right-camera data of the stereoscopic panoramic data is captured via dual programmable digital cameras physically arranged on a common mount.

20. The method of claim 19, wherein the common mount facilitates a rotation about a vertical axis that extends midway between the dual programmable digital cameras.