Three-dimensional camera adjunct

• US 8,243,123 B1
• Filed: 02/02/2006
• Issued: 08/14/2012
• Est. Priority Date: 02/02/2005
• Status: Active Grant

• Current Assignee: Anne C. Avellone, David M. Geshwind
• Original Assignee: Anne C. Avellone, David M. Geshwind
• Inventors: Geshwind, David M., Avellone, Anne C.
• Primary Examiner(s): Ye, Lin
• Assistant Examiner(s): Cowan, Euel
• Application Number:
  US11/347,121
• Time in Patent Office: 2,385 Days
• Field of Search: 348/42, 348E13002-E13005, 348/E13.022, 348/E13.044, 348/E13.068, 348/47, 348/48, 382/154
• US Class Current: 348/42
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/25 using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
• CPC Class Codes: H04N 13/261 with monoscopic-to-stereoscopic image conversion

Abstract

Additional cameras, optionally in conjunction with markers or projectors, capture three-dimensional information about the environment and characters of a filmed scene. This data is later used to convert, generally as a post-production process under highly automated computer control, or as a post broadcast process, a relatively high quality two-dimensional image stream to three-dimensional or stereoscopic, generally binocular, format.

Claims

1. A process for the production of three-dimensional stereoscopic motion picture visual image information comprising:

a. capturing at least one primary motion picture information stream;

b. capturing at least one auxiliary motion picture information stream;

c. computing at least in part from at least one of said at least one auxiliary motion picture information streams three-dimensional depth information;

d. creating at least one modified motion picture information stream at least in part by variously distorting the two-dimensional geometry of diverse portions of at least one of said at least one primary motion picture information stream in accordance with said three-dimensional depth information resulting in diverse effects on the parallax of portions of the three-dimensional stereoscopic motion picture visual image information stream created by the association of step e.; and

,

e. associating at least two of said motion picture information streams, further comprising at least one of said at least one modified motion picture information stream, to create a three-dimensional stereoscopic motion picture visual image information stream.

2. A process as in claim 1, wherein:

f. the operation of steps a. and b. are coordinated with regard to a variable point-of-view during said image stream capture.

3. A process as in claim 1, wherein:

f. the operation of steps a. and b. are coordinated with regard to variable focal characteristics during said image stream capture.

4. A process as in claim 1, wherein:

f. system components a. and b. operate in a temporally coordinated fashion during said image stream capture.

5. A process as in claim 1, wherein:

f. at least one of said at least one primary motion picture information stream is captured at a higher quality in at least some aspect than that of at least one of said at least one auxiliary motion picture information stream.

6. A process as in claim 1, wherein:

f. the processing of step d. comprises, at least in part, other than optical flow processing.

7. A process as in claim 1, wherein:

f. motion capture information is applied in step d. in addition to said three-dimensional depth information of step c.

8. A process as in claim 1, wherein:

f. information relating to the configuration of at least one image capture system used to capture at least one of said motion picture information streams is applied in step d. in addition to said three-dimensional depth information of step c.

9. A process as in claim 1, wherein:

f. depth information derived from a computer imaging process is applied in step d. in addition to said three-dimensional depth information of step c.

10. A process as in claim 1, wherein:

f. image information derived from a computer imaging process is used in combination with said three-dimensional stereoscopic motion picture visual image information stream.

11. A process as in claim 1, comprising, in addition:

f. projecting at least one geometric pattern onto at least part of the field within the view captured in at least one motion picture information stream; and

,

g. the geometry as captured of at least one of said at least one pattern comprises, at least in part, at least part of the basis of the computing of said three-dimensional depth information of step c.

12. A process as in claim 11, wherein the optical axis of said projection is oriented substantially co-linear with the optical axis of the capture of the associated motion picture information stream.

13. A process as in claim 11, wherein the optical axis of said projection is oriented substantially off-axis in relation to the optical axis of the capture of the associated motion picture information stream.

14. A process as in claim 1, wherein:

f. the computing of step c. comprises, at least in part, the comparison of information from at least two of said motion picture information streams.

15. A process as in claim 1, wherein:

f. the computing of step c. comprises, at least in part, the comparison of information from a single one of said motion picture information streams, captured at least two distinct times.

16. A process as in claim 1, wherein:

f. at least one of said at least one primary motion picture information streams comprises visual image information; and

,

g. at least one of said at least one auxiliary motion picture information streams comprises data image information.

17. A process as in claim 1, wherein:

f. as least some visual information derived from at least one of said at least one auxiliary motion picture information streams is used at least in part to fill at least some part of at least one dis-occlusion gap created during the creation of said at least one modified motion picture information stream in step d.

18. A process as in claim 1, wherein:

f. the three-dimensional depth information is exportable for use in conjunction with a second computer imaging system.

19. A process as in claim 1, wherein:

f. at least one of said at least one primary motion picture information stream and at least some of said three-dimensional depth information are communicated from a first system to a second system; and

,

g. steps d. and e. are carried out within said second system.

20. A product comprising a three-dimensional stereoscopic motion picture visual image information stream produced at least in part by the process of claim 1, and conveyed upon an information bearing medium.

21. A product comprising the exhibition of a three-dimensional stereoscopic motion picture visual image information stream produced at least in part by the process of claim 1.

22. A system for the production of three-dimensional stereoscopic motion picture visual image information comprising:

a. at least one system component to capture a primary motion picture information stream;

b. at least one system component to capture an auxiliary motion picture information stream;

c. a system component to compute at least in part from at least one of said at least one auxiliary motion picture information streams three-dimensional depth information;

d. a system component to create at least one modified motion picture information stream at least in part by variously distorting the two-dimensional geometry of diverse portions of at least one of said at least one primary motion picture information stream in accordance with said three-dimensional depth information resulting in diverse effects on the parallax of portions of the three-dimensional stereoscopic motion picture visual image information stream created by the association of system component e.; and

,

e. a system component to associate at least two of said motion picture information streams, further comprising at least one of said at least one modified motion picture information stream, to create a three-dimensional stereoscopic motion picture visual image information stream.

23. A system as in claim 22, wherein:

f. the operation of system components a. and b. are coordinated with regard to a variable point-of-view during said image stream capture.

24. A system as in claim 22, wherein:

f. the operation of system components a. and b. are coordinated with regard to variable focal characteristics during said image stream capture.

25. A system as in claim 22, wherein:

f. system components a. and b. operate in a temporally coordinated fashion during said image stream capture.

26. A system as in claim 22, wherein:

f. at least one of said at least one system component to capture a primary motion picture information stream operates at a higher quality in at least some aspect than that of at least one of said at least one system component to capture an auxiliary motion picture information stream.

27. A system as in claim 22, wherein:

f. the creation performed by system component d. comprises, at least in part, other than optical flow processing.

28. A system as in claim 22, wherein:

f. said at least one system component to capture a primary motion picture information stream comprises a left and right stereoscopic pair.

29. A system as in claim 22, wherein:

f. said at least one system component to capture an auxiliary motion picture information stream comprises a left and right stereoscopic pair.

30. A system as in claim 22, wherein:

f. said at least one system component to capture an auxiliary motion picture information stream comprises at least one system component mounted upon and operated in coordination with said at least one system component to capture a primary motion picture information stream.

31. A system as in claim 30, wherein:

g. said mounted system component is detachably mounted.

32. A system as in claim 22, wherein:

f. said at least one system component to capture an auxiliary motion picture information stream comprises at least one system component mounted remotely from said at least one system component to capture a primary motion picture information stream.

33. A system as in claim 32, wherein:

g. said at least one system component mounted remotely is operated in coordination with in slave mode to said at least one system component to capture a primary motion picture information stream.

34. A system as in claim 22, further comprising:

f. a system component to capture information relating to the configuration of at least one of said system components used to capture a motion picture information stream; and

,

g. that said configuration information is made available to be used by system component c. during the computation of said three-dimensional depth information.

35. A system as in claim 22, comprising in addition:

f. at least one system component to project at least one geometric pattern upon at least part of the field within the view of at least one system component to capture a motion picture information stream.

36. A system as in claim 35, wherein:

g. at least one system component to project at least one geometric pattern comprises a multiplicity of system components separated spatially which project distinct patterns such that the pattern projected from a first such system component can be distinguished from the pattern projected from at least a second such system component.

37. A system as in claim 35, wherein:

g. at least one system component to project at least one geometric pattern comprises a multiplicity of system components separated spatially which project patterns at distinct times.

38. A system as in claim 35, wherein:

g. visual image information and projected geometric patterns are captured, at least in part, in a time division multiplexed fashion.

39. A system as in claim 35, wherein:

g. at least some geometric patterns are both projected and captured, at least in part, along the same optical path.

40. A system as in claim 35, wherein:

g. at least some geometric patterns are projected and at least some visual image information is captured, at least in part, along the same optical path.

41. A system as in claim 22, wherein:

f. said at least one system component to capture a primary motion picture information stream comprises a system component to capture visual image information;

g. said at least one system component to capture an auxiliary motion picture information stream comprises a pair, mounted to the left and right of said system component to capture a primary motion picture information stream;

h. said system component f. operates at a higher quality in at least some aspect than the system components comprising said pair of system components of g.

i. said system components of f. and g. are operated in a coordinated fashion with regard to orientation; and

,

j. said system components of f. and g. are operated in a coordinated fashion with regard to optical configuration.

42. A system as in claim 41, wherein:

k. at least one of said system components of g. captures data image information at least some of the time.

43. A motion picture camera system for the production of at least one three-dimensional stereoscopic motion picture visual image information stream comprising:

a. a relatively high-quality motion picture camera to capture a primary motion picture visual image information stream;

b. a pair of relatively lesser-quality in some aspect auxiliary motion picture cameras mounted to the left and right of the central optical input axis of said camera element a. to capture auxiliary streams of motion picture information comprising at least in part binocular stereoscopic information;

said binocular stereoscopic information capable of being used to compute at least one stereoscopic depth information stream;

said at least one stereoscopic depth information stream capable of being used to process at least in part said primary motion picture visual image information stream to create at least one three-dimensional stereoscopic motion picture visual image information stream;

c. that said cameras operate in a coordinated fashion with respect to a variable point of view during the capture of said motion picture information streams; and

d. that said cameras operate in a temporally coordinated fashion during the capture of said motion picture information streams.

44. The system of claim 43, further comprising:

that system components a. and b. are separable sub-systems integrated for operation.

45. The system of claim 44, further comprising:

that said cameras operate in a coordinated fashion with respect to variable focal characteristics during the capture of said multiple-frame sequences of motion picture visual information streams.

46. The system of claim 44, further comprising:

e. at least one projector of at least one geometric pattern, projecting at least in part onto at least some part of the field within the view of said auxiliary cameras; and

, that the information captured by said auxiliary cameras is, at least in part, said at least one geometric pattern.

47. The system of claim 45, further comprising:

e. at least one projector of at least one geometric pattern, projecting at least in part onto at least some part of the field within the view of said auxiliary cameras; and

, that the information captured by said auxiliary cameras is, at least in part, said at least one geometric pattern.

48. The system of claim 45, further comprising:

f. a system component to compute at least one stereoscopic depth information stream from said binocular stereoscopic information; and

,

g. a system component to create at least one three-dimensional stereoscopic motion picture visual image information stream from said primary motion picture visual image information stream processed using at least in part said at least one stereoscopic depth information stream.

49. The system of claim 43, further comprising:

that said cameras operate in a coordinated fashion with respect to variable focal characteristics during the capture of said multiple-frame sequences of motion picture visual information streams.

50. The system of claim 49, further comprising:

e. at least one projector of at least one geometric pattern, projecting at least in part onto at least some part of the field within the view of said auxiliary cameras; and

, that the information captured by said auxiliary cameras is, at least in part, said at least one geometric pattern.

51. The system of claim 43, further comprising:

e. at least one projector of at least one geometric pattern, projecting at least in part onto at least some part of the field within the view of said auxiliary cameras; and

, that the information captured by said auxiliary cameras is, at least in part, said at least one geometric pattern.

52. The system of claim 43, further comprising:

e. a system component to compute at least one stereoscopic depth information stream from said binocular stereoscopic information; and

,

f. a system component to create at least one three-dimensional stereoscopic motion picture visual image information stream from said primary motion picture visual image information stream processed using at least in part said at least one stereoscopic depth information stream.