Spatio-Temporal Light Field Cameras

• US 20130321581A1
• Filed: 10/24/2012
• Published: 12/05/2013
• Est. Priority Date: 06/01/2012
• Status: Active Grant

• Current Assignee: Ostendo Technologies Inc.
• Original Assignee: Ostendo Technologies Inc.
• Inventors: El-Ghoroury, Hussein S., Alpaslan, Zahir Y., Cai, Jingbo, Maiers, Marty, Warner, Philip, McNeill, Dale A.
• Granted Patent: US 9,179,126 B2
• Time in Patent Office: Days
• US Class Current: 348/46
• CPC Class Codes: G02B 13/0085 employing wafer level optics
• CPC Class Codes: G02B 27/1066 for enhancing image performance, like resolution, pixel numbers, dual magnifications or dynamic range, by tiling, slicing or overlapping fields of view
• CPC Class Codes: G02B 27/1073 characterized by manufacturing or alignment methods
• CPC Class Codes: G02B 27/123 The splitting element being a lens or a system of lenses, including arrays and surfaces with refractive power
• CPC Class Codes: G02B 3/0012 characterised by the manufacturing method
• CPC Class Codes: G02B 3/0056 arranged along two different directions in a plane, e.g. honeycomb arrangement of lenses G02B3/0043 takes precedence; miniaturised objectives for electronic devices employing wafer level optics G02B13/0085
• CPC Class Codes: G02B 30/27 involving lenticular arrays
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10052 Images from lightfield camera
• CPC Class Codes: G06T 7/557 from light fields, e.g. from plenoptic cameras
• CPC Class Codes: H04N 13/207 using a single 2D image sensor
• CPC Class Codes: H04N 13/282 for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems
• CPC Class Codes: H04N 13/286 having separate monoscopic and stereoscopic modes
• CPC Class Codes: H04N 13/289 Switching between monoscopic and stereoscopic modes
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof
• CPC Class Codes: H04N 23/16 Optical arrangements associated therewith, e.g. for beam-splitting or for colour correction
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/54 Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 23/661 : Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/667 : Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
• CPC Class Codes: H04N 23/67 : Focus control based on electronic image sensor signals
• CPC Class Codes: H04N 23/68 : for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/80 : Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 23/815 : for controlling the resolution by using a single image
• CPC Class Codes: H04N 23/90 : Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 23/951 : by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 23/957 : Light-field or plenoptic cameras or camera modules
• CPC Class Codes: H04N 25/48 : Increasing resolution by shifting the sensor relative to the scene
• CPC Class Codes: H04N 5/04 : Synchronising for television systems using pulse code modulation H04N7/56

Abstract

Spatio-temporal light field cameras that can be used to capture the light field within its spatio temporally extended angular extent. Such cameras can be used to record 3D images, 2D images that can be computationally focused, or wide angle panoramic 2D images with relatively high spatial and directional resolutions. The light field cameras can be also be used as 2D/3D switchable cameras with extended angular extent. The spatio-temporal aspects of the novel light field cameras allow them to capture and digitally record the intensity and color from multiple directional views within a wide angle. The inherent volumetric compactness of the light field cameras make it possible to embed in small mobile devices to capture either 3D images or computationally focusable 2D images. The inherent versatility of these light field cameras makes them suitable for multiple perspective light field capture for 3D movies and video recording applications.

Claims

1. A light field camera comprising:

a micro photo-detector array device having a light detection surface defining an array of pixels, each pixel being a light detector that is individually addressable to output an electrical signal responsive to the intensity of light coupled into an aperture of the respective pixel; and

a micro lens array of micro lens elements;

the micro photo-detector array device and the micro lens array being assembled together and temporally angularly articulated as a single assembly about at least one axis parallel to a plane of the light detection surface of the micro photo-detector array device and at least through a maximum articulation angle;

each micro lens element of the micro lens array being associated and aligned relative to a plurality of pixels within a respective two dimensional pixel group array of the micro photo-detector array device, with each lens optically mapping light that impinges an aperture of the respective lens from a discrete set of directions within a light field defined by an angular extent of the respective lens onto the corresponding plurality of pixels within the two dimensional array of pixel groups of the micro photo-detector array device, the discrete set of directions defining an angular resolution between adjacent directions and an angular extent of the set of light directions;

the temporal angular articulation about the at least one axis expanding the angular extent and/or the angular resolution in a direction about the axis of the angular articulation.

2. The light field camera of claim 1 wherein each pixel is individually addressable to output the electrical signal also responsive to the color of light coupled into an aperture of the respective pixel.

3. The light field camera of claim 2 wherein the micro photo-detector array device and the micro lens array are temporally angularly articulated as a single assembly.

4. The light field camera of claim 1 wherein the temporal angular articulation is about two orthogonal axes.

5. The light field camera of claim 1 wherein each micro lens element is a Fourier lens system.

13. The light field camera of claim 1 wherein each lens element of the micro lens array is associated and aligned with a plurality of pixels within a respective two dimensional pixel group array of the micro photo-detector array device, with each lens optically mapping light that impinges an aperture of the respective lens from a discrete set of directions within a light field defined by an angular extent of the respective lens onto the corresponding plurality of pixels within the two dimensional array pixel group of the micro photo-detector array device.

14. The light field camera of claim 13 wherein the micro lens elements are truncated to each have respective dimensional aspects match that of the two dimensional array of the pixels comprising the pixel and their optical center aligned with the center of their respective two dimensional array of the pixels.

15. The light field camera of claim 13 wherein the angular articulation temporally expands the angular extent of the respective lens onto the corresponding plurality of pixels about each axis by a maximum angular articulation, thereby temporally expanding a detectable light field size and increasing the number of the light detection directions about each axis by the ratio of said angular extent expansion to the angular extent along each of the two articulation axes.

16. The light field camera of claim 15 wherein each pixel within each pixel group is individually addressable which, together with the association of the pixels within each pixel group with the temporally expanded set of directions allowing the addressability of the pixels over the temporally expanded set of directions, thereby enables the light field camera to be able to detect the light that impinges its aperture from the expanded light field defined by the discrete set of directions comprising the set of the temporally expanded set of light directions.

17. The light field camera of claim 16 wherein each micro photo-detector detects individual pixel color and intensity, thereby enabling the light field camera to detect the color, intensity and direction of the light that impinges its aperture from the expanded light field size defined by the discrete set of directions comprising the set of the temporally expanded set of light directions.

18. The light field camera of claim 15 wherein the temporal articulation causes any light directions within the expanded set of light directions to be detected during the time interval during which the respective light direction falls with the temporally articulated angular extent.

19. The light field camera of claim 13 wherein each pixel of each of the pixel groups in the pixel group array is individually addressable to detect the color, intensity and direction of the light that impinges the aperture of the light field camera, thereby enabling the light field camera to spatially detect the intensity, color and direction of the light that impinges its aperture.

20. The light field camera of claim 19 wherein the angular articulation temporally expands the angular extent of the respective lens onto the corresponding plurality of pixels about each axis by a maximum angular articulation, thereby temporally expanding a detectable light field size and increasing the number of the light detection directions about each axis by the ratio of said angular extent expansion to the angular extent along each of the two articulation axes, the light field camera having a spatial resolution that is determined by the number of pixel groups along each axis of the two dimensional array of pixel groups and having a directional resolution that is determined by the number of individual light directions within the temporally expanded set of light directions.

21. The light field camera of claim 20:

wherein each lens element of the micro lens array is associated and aligned with a plurality of pixels within a respective two dimensional pixel group array of the micro photo-detector array device, with each lens optically mapping light that impinges an aperture of the respective lens from a discrete set of directions within a light field defined by an angular extent of the respective lens onto the corresponding plurality of pixels within the two dimensional array pixel group of the micro photo-detector array device; and

wherein the angular articulation is either temporally continuous or discrete, and has a repetition rate that is proportional to and synchronized with an image capture frame rate, whereby the maximum angular articulation around each of the two axes determines the full angular extent of the light field camera, the angular coverage shape it subtends and its aspect ratio; and

wherein the angular articulation around each of the two axes is at least equal to the image capture frame rate multiplied by a factor that equals a ratio of the size in degrees of the full angular extent along each respective axis to the size in degrees of the angular extent;

whereby the light field camera records a light field defined by the spatial coordinates, within the two dimensional array of the pixel groups, a set of angular directional coordinates defined by the two dimensional array pixel group and the temporal values of the articulation angle.

22. The light field camera of claim 20 wherein the spatial resolution and directional resolution of the light field camera is determined by the size of a pixel group, the resolution and pixel pitch of the micro photo-detector array device, the lens pitch of the micro lens array, the angular extent and the maximum articulation angle.

23. The light field camera of claim 13:

wherein the micro lens elements are truncated to have their respective size match the dimensions of the two dimensional array of the pixels in the respective pixel group; and

wherein the optical center of the lens element at the center of the micro lens array is aligned with the center of its respective pixel group, and the lens elements away from the center of the micro lens array have their optical centers offset from the center of their respective pixel group, with the offset gradually increasing for lens elements further away from the center of the micro lens array.

24. The light field camera of claim 23:

wherein the angular articulation is either temporally continuous or discrete, and has a repetition rate that is proportional to and synchronized with an image capture frame rate, whereby the maximum angular articulation around each of the two axes determines the full angular extent of the light field camera, the angular coverage shape it subtends and its aspect ratio;

wherein the angular articulation temporally expands the angular extent of the respective lens onto the corresponding plurality of pixels about each axis by a maximum angular articulation, thereby temporally expanding a detectable light field size and increasing the number of the light detection directions about each axis by the ratio of said angular extent expansion to the angular extent along each of the two articulation axes, the light field camera having a spatial resolution that is determined by the number of pixel groups along each axis of the two dimensional array of pixel groups and having a directional resolution that is determined by the number of individual light directions within the temporally expanded set of light directions; and

wherein the offset of the optical centers of the micro lens elements enable the increase of the directional resolution without sacrificing either the spatial resolution or the full angular extent of the light field camera.

25. The light field camera of claim 1 wherein the angular articulation is either temporally continuous or discrete, and has a repetition rate that is proportional to and synchronized with an image capture frame rate, whereby the maximum angular articulation around each of the two axes determines a full angular extent of the light field camera, the angular coverage shape it subtends and its aspect ratio.

26. The light field camera of claim 25:

wherein each lens element of the micro lens array is associated and aligned with a plurality of pixels within a respective two dimensional pixel group array of the micro photo-detector array device, with each lens optically mapping light that impinges an aperture of the respective lens from a discrete set of directions within a light field defined by an angular extent of the respective lens onto the corresponding plurality of pixels within the two dimensional array pixel group of the micro photo-detector array device, and

wherein the angular articulation around each of the two axes is at least equal to the image capture frame rate multiplied by a factor that equals a ratio of the size in degrees of the full angular extent along each respective axis to the size in degrees of the angular extent.

27. The light field camera of claim 25 having an angular resolution defined as the angular separation between the individual light directions within the temporally expanded set of light directions and determined by the fractional value of the angular extent being addressed by each pixel within a pixel group.

28. The light field camera of claim 27:

wherein the angular articulation is either temporally continuous or discrete, and has a repetition rate that is proportional to and synchronized with an image capture frame rate, whereby the maximum angular articulation around each of the two axes determines the full angular extent of the light field camera, the angular coverage shape it subtends and its aspect ratio, and

wherein the maximum angular articulation is selected to affect an increase in the value of the full angular extent.

29. The light field camera of claim 27:

wherein the angular articulation temporally expands the angular extent of the respective lens onto the corresponding plurality of pixels about each axis by a maximum angular articulation, thereby temporally expanding a detectable light field size and increasing the number of the light detection directions about each axis by the ratio of said angular extent expansion to the angular extent along each of the two articulation axes, the light field camera having a spatial resolution that is determined by the number of pixel groups along each axis of the two dimensional array of pixel groups and having a directional resolution that is determined by the number of individual light directions within the temporally expanded set of light directions, and

wherein the maximum angular articulation is selected to affect an increase in the spatial resolution for a given value of the angular resolution.

30. The light field camera of claim 29 further comprising:

wherein each lens of the micro lens array is associated and aligned with a plurality of pixels within a respective two dimensional pixel group array of the micro photo-detector array device, with each lens optically mapping light that impinges an aperture of the respective lens from a discrete set of directions within a light field defined by an angular extent of the respective lens onto the corresponding plurality of pixels within the two dimensional array pixel group of the micro photo-detector array device;

wherein the angular articulation is either temporally continuous or discrete, and has a repetition rate that is proportional to and synchronized with an image capture frame rate, whereby the maximum angular articulation around each of the two axes determines the full angular extent of the light field camera, the angular coverage shape it subtends and its aspect ratio; and

wherein each of the micro lens elements of the micro lens array is an optical system comprising at least one refracting surface whereby the number of said refracting surfaces and their optical characteristics determine the angular extent and affect the value of the full angular extent and the angular resolution.

31. The light field camera of claim 30 wherein the optical system of the micro lens elements is a Fourier optical system that is focused at infinity and maps directional aspects of the light field that impinges its aperture within the angular extent onto the two dimensional array of the pixels comprising the pixel groups.

32. The light field camera of claim 1 wherein the light field camera is embedded in a mobile device.

33. The light field camera of claim 1 having a temporal angular articulation wherein the light field camera records a curved wavefront of a light field it captures, thus enabling the light field camera to capture curved parallaxes of the light field.

34. The light field camera of claim 33 wherein the curved wavefront recording capability enables the light field camera to record information not only about the spatial and directional aspects of the light field, but also about the phase of the wavefront of the light field it captures, thus enabling the light field camera to record a four dimensional spatio-directional space of the light field plus the phase of its wavefront.

35. The light field camera of claim 1 wherein the light field camera captures two dimensional light field images that are then computationally focused at a desired plane or curved surface or computationally focused within a desired depth volume having flat or curved surfaces.

36. The light field camera of claim 1 configured as a three dimensional light field camera with arbitrary spatial and directional resolutions.

37. The light field camera of claim 1 having operational modes for switching between operation as a three dimensional light field camera and operation as a two dimensional light field camera by adapting the desired operational mode.

38. The light field camera of claim 1 wherein the light field camera is embedded in a mobile device and networked to other mobile devices that also have a light field camera embedded therein, the mobile devices being interconnected either through wireless or wireline networks or through bulk data transfer using flash memory modules, thus making it possible to exchange light field camera output data with a multiplicity of such mobile devices that captured a different perspective of a viewing scene, and use such networked light field camera output data to create a collective light field of the viewed scene.

39. The light field camera of claim 38 wherein the networked mobile devices are either cell phones or computers, or alternatively the light field camera is integrated with its own network connectivity interface capabilities.

40. The light field camera of claim 1 configured to record either still or motion video images.

6. A light field camera comprising:

a micro photo-detector array device having a light detection surface and

a micro lens array of micro lens elements;

whereby the micro photo-detector array device and the micro lens array are assembled together and temporally angularly articulated as a single assembly around two orthogonal axes parallel to a plane of the light detection surface of the micro photo-detector array device;

wherein the temporal angular articulation is within a range of plus to minus a maximum angular articulation for each axis.

7. The light field camera of claim 6 wherein the micro photo-detector array device comprises an array of pixels whereby each of the pixels is a light detector that is individually addressable to output an electrical signal responsive to the color and intensity of light coupled into an aperture of the respective pixel.

8. The light field camera of claim 7 wherein the micro lens array is comprised of an array of lens elements that are precisely aligned relative to one another and relative to the micro photo-detector pixel array.

9. The light field camera of claim 8 wherein the light detection surface of the micro photo-detector array device is placed either at or a distance away from the focal plane of the micro lens elements, the placement either being fixed or variable using a z-axis actuator.

10. The light field camera of claim 9:

wherein each lens element of the micro lens array is associated and aligned with a plurality of pixels within a respective two dimensional pixel group array of the micro photo-detector array device, with each lens optically mapping light that impinges an aperture of the respective lens from a discrete set of directions within a light field defined by an angular extent of the respective lens onto the corresponding plurality of pixels within the two dimensional array pixel group of the micro photo-detector array device; and

wherein the light detection surface of the micro photo-detector array device is placed at a distance away from the focal plane of the micro lens elements causing the plurality of pixels comprising each of the pixel groups of the micro photo-detector array device to be shared collectively in recording the light field, thus enabling the light field camera to achieve angular and directional resolutions that are higher than if the micro photo-detector array device is placed at the focal plane of the micro lens elements.

11. The light field camera of claim 10:

wherein each lens element of the micro lens array is associated and aligned with a plurality of pixels within a respective two dimensional pixel group array of the micro photo-detector array device, with each lens optically mapping light that impinges an aperture of the respective lens from a discrete set of directions within a light field defined by an angular extent of the respective lens onto the corresponding plurality of pixels within the two dimensional array pixel group of the micro photo-detector array device; and

wherein each pixel within each of the pixel groups records a known weighted sum of the light field information from a multiplicity of directions, whereby the light field information can be resolved computationally to increase the angular and directional resolutions of the light field camera.

12. The light field camera of claim 10 wherein the recording of the pixels of the micro photo-detector array device together with the temporal angular articulation enables a desired angular and/or directional resolution using pixel groups comprising a fewer number of pixels than if there were no articulation, and providing more pixels for the formation of more pixel groups which consequently enables an increase in the achievable spatial resolution of the light field camera.

41. A light field camera comprising:

an micro photo-detector array device comprising a multiplicity of pixels;

a micro lens array aligned and physically bonded to a surface of the micro photo-detector array device, and

a two-axis gimbal supported on a gimbal base and having a bonding pad upon which the micro photo-detector array device with the micro lens array aligned and physically bonded to its surface is physically and electrically bonded;

two sets of electromechanical actuators aligned with the two axes of the gimbal to affect temporal angular articulation of said bonding pad around the two axes of the gimbal; and

a set of electrical contacts connecting said bonding pad to a set of micro photo-detector array device interface contacts.

42. The light field camera of claim 41 wherein the 2-axis gimbal is implemented using multiple silicon substrate layers for a 2-axis pivot of the gimbal, and a mechanical resistance spring that defines the neutral position of the gimbal relative to the gimbal base.

43. The light field camera of claim 42 wherein the mechanical resistance spring comprises the two sets of silicon bridges.

44. The light field camera of claim 42 wherein the 2-axis pivot is formed by the etching, using semiconductor lithography, of an inner ring and an outer ring around the periphery of the bonding pad, the rings being joined with a set of silicon bridges along each respective axis.

45. The light field camera of claim 44 wherein electrical connections between the bonding pad and the gimbal base are routed through the two sets of silicon bridges.

46. The light field camera of claim 41 wherein the two-axis gimbal comprises a spherical pivot on the backside of the bonding pad and a matched spherical socket on the topside of the gimbal base.

47. The light field camera of claim 46 wherein the electromechanical actuators are bonded to a backside of the bonding pad and to a topside of the gimbal base.

48. The light field camera of claim 46 wherein the electromechanical actuators are embedded in the spherical pivot and spherical socket.

49. The light field camera of claim 46 further comprising a mechanical resistance spring associated with the 2-axis gimbal in the form of a set of four spiral shaped springs defined by an etched spring layer and bonded at its inner end at the backside of the four corners of the bonding pad and at its outer end at the topside of the gimbal base.

50. The light field camera of claim 49 wherein electrical connections between the bonding pad and the set of the gimbal base are routed through the four spiral shaped springs.

51. The light field camera of claim 41 wherein the electromechanical actuators are driven by electrical drive signals to affect the temporal angular articulation, the electrical signals being either temporally continuous or discrete and having a repetition rate that is proportional to and synchronized with an image capture frame rate provided through the device interface contacts whereby said electrical drive signals include a correction value provided by a set of sensors bonded to a backside of the contact pad and a topside of the gimbal base.

52. The light field camera of claim 41 wherein the temporal angular articulation is affected by the electromechanical actuators having a maximum angular articulation value around each of the two axes of the gimbal.

53. The light field camera of claim 41 wherein the micro photo-detector array device comprises an array of pixels whereby each of the pixels is a few microns in size and is individually addressable through the device interface contacts to output an electrical signal that is responsive to color and intensity of light coupled into its aperture.

54. The light field camera of claim 53 wherein the micro lens array is comprised of an array of lens elements that is fabricated using ultraviolet imprint lithography at the wafer level to be precisely aligned relative to one another and relative to the micro photo-detector pixel arrays also fabricated at a wafer level using semiconductor wafer level alignment techniques.

55. The light field camera of claim 54 wherein the light detection surface of the micro photo-detector array device is placed either at or a distance away from the focal plane of the micro lens elements of the micro lens array, and the placement is either fixed by a design selection, or adjustable over a specified range using a z-axis actuator.

56. The light field camera of claim 55:

wherein each of the lens elements of the micro lens array is associated and precisely aligned with a plurality of pixels within a two dimensional pixel group array of the micro photo-detector array device, with each lens optically mapping the light that impinges its aperture from a discrete set of directions within a light field defined by an angular extent of said lens onto the corresponding plurality of pixels within the two dimensional pixel group array of the micro photo-detector array device, and

wherein the micro lens array is comprised of an array of lens elements that is fabricated using ultraviolet imprint lithography at the wafer level to be precisely aligned relative to one another and relative to the micro photo-detector pixel arrays also fabricated at a wafer level using semiconductor wafer level alignment techniques, and

wherein the placement of the light detection surface of the micro photo-detector array device at a distance away from the focal plane of the micro lens elements causes the plurality of pixels comprising each of the pixel groups of the micro photo-detector array device to be shared collectively in recording the light field, thus enabling the light field camera to achieve angular and directional resolutions that are higher than when the micro photo-detector array device is placed at the focal plane of the micro lens elements.

57. The light field camera of claim 56:

wherein each of the lens elements of the micro lens array is associated and precisely aligned with a plurality of pixels within a two dimensional pixel group array of the micro photo-detector array device, with each lens optically mapping the light that impinges its aperture from a discrete set of directions within a light field defined by an angular extent of said lens onto the corresponding plurality of pixels within the two dimensional pixel group array of the micro photo-detector array device, and

wherein the light field recording collective sharing of the pixels of the micro photo-detector array device causes each pixel within each of the pixel groups to record a known weighted sum of the light field information from a multiplicity of directions, whereby light field information can be resolved computationally to increase the angular and directional resolutions of the light field camera.

58. The light field camera of claim 56:

wherein each of the lens elements of the micro lens array is associated and precisely aligned with a plurality of pixels within a two dimensional pixel group array of the micro photo-detector array device, with each lens optically mapping the light that impinges its aperture from a discrete set of directions within a light field defined by an angular extent of said lens onto the corresponding plurality of pixels within the two dimensional pixel group array of the micro photo-detector array device, and

wherein the light field recording collective sharing of the pixels of the micro photo-detector array device together with the temporal angular articulation provides a desired angular and/or directional resolution using pixel groups of a fewer number of pixels, thus making more pixels for the formation of more pixel groups available, thereby enabling an increase in the achievable spatial resolution of the light field camera.

59. The light field camera of claim 41 wherein each of the lens elements of the micro lens array is associated and precisely aligned with a plurality of pixels within a two dimensional pixel group array of the micro photo-detector array device, with each lens optically mapping the light that impinges its aperture from a discrete set of directions within a light field defined by an angular extent of said lens onto the corresponding plurality of pixels within the two dimensional pixel group array of the micro photo-detector array device.

60. The light field camera of claim 59 wherein the spatial resolution and directional resolutions of the light field camera are determined by the size of the pixel group, the resolution and pixel pitch of the micro photo-detector array device, the lens pitch of the micro lens array, the angular extent and the maximum articulation angle.

61. The light field camera of claim 59:

wherein the micro lens array is comprised of an array of lens elements that is fabricated using ultraviolet imprint lithography at the wafer level to be precisely aligned relative to one another and relative to the micro photo-detector pixel arrays also fabricated at a wafer level using semiconductor wafer level alignment techniques, and

wherein the micro lens elements of the micro lens array are truncated to have their respective dimensional aspects match that of the two dimensional array of the pixels comprising the pixel groups and their optical center aligned with the center of their respective two dimensional array of the pixels comprising the pixel groups.

62. The light field camera of claim 59:

wherein the micro lens array is comprised of an array of lens elements that is fabricated using ultraviolet imprint lithography at the wafer level to be precisely aligned relative to one another and relative to the micro photo-detector pixel arrays also fabricated at a wafer level using semiconductor wafer level alignment techniques, and

wherein the micro lens elements of the micro lens array are truncated to have their respective size match dimensions of the two dimensional array of the pixels of the pixel groups and the respective optical center of the lens element at the center of the micro lens array aligned with the center of its respective pixel group and the lens elements away from the center of the micro lens array having their optical centers being offset from the center of their respective pixel group, with said offset gradually increasing for lens elements further away from the center of the micro lens array.

63. The light field camera of claim 59 wherein the temporal articulation causes any light directions within the expanded set of light directions to be detected during the time interval during which said light directions fall within the temporally articulated angular extent.

64. The light field camera of claim 59 wherein in the two dimensional array of the pixel groups, each of the individual pixel groups are individually addressable through the contact pad to detect the color, intensity and direction of the light that impinges the aperture of the light field camera, thereby enabling the light field camera to spatially detect the light that impinges its aperture.

65. The light field camera of claim 64:

wherein the micro photo-detector array device comprises an array of pixels whereby each of the pixels is a few microns in size and is individually addressable through the device interface contacts to output an electrical signal that is responsive to the color and intensity of light coupled into its aperture,

wherein the angular articulation temporally expands the angular extent by twice the maximum angular articulation and thereby temporally expands a detectable light field size and increases the number of light detection directions by the ratio of said angular extent expansion to the angular extent along each of the two articulation axes,

wherein the individual pixel color and intensity detection addressability and the temporal association of the pixels within a pixel group with the temporally expanded set of light directions enables the detection of the color and intensity of the light impinging from each of the temporally expanded set of directions, thereby enabling the light field camera to detect the color, intensity and direction of the light that impinges its aperture from the expanded light field defined by the discrete set of directions comprising the set of the temporally expanded set of light directions, and

wherein the directional, spatial, color and intensity detection addressability of the light field camera uses a multiple field data output from the light field camera whereby for each designated pixel group address within the spatial array of the pixel groups, at least one output data field is used to specify the direction of the light impinging the aperture of the light field camera and at least one output data field is used to specify the color and intensity of the light detected from that designated direction.

66. The light field camera of claim 64 wherein the angular articulation temporally expands the angular extent by twice the maximum angular articulation and thereby temporally expands a detectable light field size and increases the number of the light detection directions by the ratio of said angular extent expansion to the angular extent along each of the two articulation axes, the light field camera having spatial detection with a spatial resolution that is determined by the number of pixel groups along each axis of the two dimensional array of pixel groups and having a directional resolution that is determined by the number of individual light directions within the temporally expanded set of light directions.

67. The light field camera of claim 66 having an angular resolution which is defined as the angular separation between the individual light directions within the temporally expanded set of light directions and determined by the fractional value of the angular extent being addressed by each one pixel within the pixel group, and

wherein the angular articulation is either temporally continuous or discrete and has a repetition rate that is proportional to and synchronized with an image capture frame rate whereby the maximum angular articulation around each of the two axes determines the full angular extent of the light field camera, the angular coverage shape it subtends, and its aspect ratio, and

wherein the value of the maximum angular articulation is selected to affect an increase in the value of either the full angular extent or the spatial resolution for a given value of the angular resolution.

68. The light field camera of claim 66:

wherein the micro photo-detector array device comprises an array of pixels whereby each of the pixels is a few microns in size and is individually addressable through the device interface contacts to output an electrical signal that is responsive to the color and intensity of light coupled into its aperture,

wherein the micro lens array is comprised of an array of lens elements that is fabricated using ultraviolet imprint lithography at the wafer level to be precisely aligned relative to one another and relative to the micro photo-detector pixel arrays also fabricated at a wafer level using semiconductor wafer level alignment techniques,

wherein the angular articulation is either temporally continuous or discrete and has a repetition rate that is proportional to and synchronized with an image capture frame rate whereby the maximum angular articulation around each of the two axes determines the full angular extent of the light field camera, the angular coverage shape it subtends, and its aspect ratio, and

wherein the offset of the optical centers of the micro lens elements increases the directional resolution without sacrificing either the spatial resolution or the full angular extent of the light field camera.

69. The light field camera of claim 66:

wherein each of the lens elements of the micro lens array is associated and precisely aligned with a plurality of pixels within a two dimensional pixel group array of the micro photo-detector array device, with each lens optically mapping the light that impinges its aperture from a discrete set of directions within a light field defined by an angular extent of said lens onto the corresponding plurality of pixels within the two dimensional pixel group array of the micro photo-detector array device, and

wherein the angular articulation around each of the two axes is at least equal to the image capture frame rate multiplied by a factor that equals to ratio of the size in degrees of the full angular extent along each respective axis to the size in degrees of the angular extent, and

wherein in the two dimensional array of the pixel groups, each of the individual pixel groups are individually addressable through the contact pad to detect the color, intensity and direction of the light that impinges the aperture of the light field camera, thereby enabling the light field camera to spatially detect the light that impinges its aperture;

the light field camera being capable of recording a light field defined by spatial coordinates within the two dimensional array of the pixel groups, a set of angular directional coordinates defined by the two dimensional array pixel group and the temporal values of the articulation angle.

70. The light field camera of claim 59 wherein the angular articulation temporally expands the angular extent by twice the maximum angular articulation and thereby temporally expands a detectable light field size and increases the number of the light detection directions by the ratio of said angular extent expansion to the angular extent along each of the two articulation axes.

71. The light field camera of claim 70 wherein the association of the pixels within pixel groups with the temporally expanded set of directions and the individual pixel addressability enables the addressability of the temporally expanded set of directions, thereby enabling the light field camera to detect light that impinges its aperture from the expanded light field defined by the discrete set of directions comprising the set of the temporally expanded set of light directions.

72. The light field camera of claim 70 wherein the individual pixel color and intensity detection addressability and the temporal association of the pixels within a pixel group with the temporally expanded set of light directions enabling the detection of the color and the intensity of the light impinging from each of the temporally expanded set of directions, thereby enabling the light field camera to detect the color, intensity and direction of the light that impinges its aperture from the expanded light field defined by the discrete set of directions comprising the set of the temporally expanded set of light directions.

73. The light field camera of claim 70 having an angular resolution which is defined as the angular separation between the individual light directions within the temporally expanded set of light directions and determined by the fractional value of the angular extent being addressed by each one pixel within the pixel group.

74. The light field camera of claim 73:

wherein the micro photo-detector array device comprises an array of pixels whereby each of the pixels is a few microns in size and is individually addressable through the device interface contacts to output an electrical signal that is responsive to color and intensity of light coupled into its aperture,

wherein the micro lens array is comprised of an array of lens elements that is fabricated using ultraviolet imprint lithography at the wafer level to be precisely aligned relative to one another and relative to the micro photo-detector pixel arrays also fabricated at a wafer level using semiconductor wafer level alignment techniques, and

wherein the angular articulation is either temporally continuous or discrete and has a repetition rate that is proportional to and synchronized with an image capture frame rate whereby the maximum angular articulation around each of the two axes determines the full angular extent of the light field camera, the angular coverage shape it subtends, and its aspect ratio, and

wherein each of the micro lens elements is an optical system with at least one refracting surface, whereby the number of said refracting surfaces and their optical characteristics determine the angular extent and affect the value of the full angular extent and the angular resolution.

75. The light field camera of claim 74:

wherein the micro lens array is comprised of an array of lens elements that is fabricated using ultraviolet imprint lithography at the wafer level to be precisely aligned relative to one another and relative to the micro photo-detector pixel arrays also fabricated at a wafer level using semiconductor wafer level alignment techniques,

wherein each of the lens elements of the micro lens array is associated and precisely aligned with a plurality of pixels within a two dimensional pixel group array of the micro photo-detector array device, with each lens optically mapping the light that impinges its aperture from a discrete set of directions within a light field defined by an angular extent of said lens onto the corresponding plurality of pixels within the two dimensional pixel group array of the micro photo-detector array device, and

wherein the optical system of the micro lens elements is a Fourier optical system that is focused at infinity and maps the directional aspects of the light field that impinges its aperture within the angular extent onto the two dimensional arrays of the pixels.

76. The light field camera of claim 41 wherein the angular articulation is either temporally continuous or discrete and has a repetition rate that is proportional to and synchronized with an image capture frame rate whereby the maximum angular articulation around each of the two axes determines the full angular extent of the light field camera, the angular coverage shape it subtends, and its aspect ratio.

77. The light field camera of claim 76:

wherein each of the lens elements of the micro lens array is associated and precisely aligned with a plurality of pixels within a two dimensional pixel group array of the micro photo-detector array device, with each lens optically mapping the light that impinges its aperture from a discrete set of directions within a light field defined by an angular extent of said lens onto the corresponding plurality of pixels within the two dimensional pixel group array of the micro photo-detector array device, and

wherein the angular articulation around each of the two axes is at least equal to the image capture frame rate multiplied by a factor that equals the ratio of the size in degrees of the full angular extent along each respective axis to the size in degrees of the angular extent.

78. The light field camera of claim 41 wherein the light field camera is embedded in a mobile device.

79. The light field camera of claim 41 wherein the temporal angular articulation enables the light field camera to record a curved wavefront of the light field it captures, thus enabling the light field camera to be able to capture curved parallaxes of the light field.

80. The light field camera of claim 79 wherein the curved wavefront recording capability enables the light field camera to record information not only about the spatial and directional aspects of the light field but also about the phase of the wavefront of the light field it captures, thus enabling the light field camera to record a four dimensional spatio-directional space of the light field plus the phase of its wavefront.

81. The light field camera of claim 41, in plurality and in a tiled array.

82. The light field camera of claim 81 wherein each of the lens elements of the micro lens array is associated and aligned with a plurality of pixels within a two dimensional pixel group array of the micro photo-detector array device, with each lens optically mapping the light that impinges its aperture from a discrete set of directions within a light field defined by an angular extent of said lens onto the corresponding plurality of pixels within the two dimensional pixel group array of the micro photo-detector array device,

wherein the angular articulation temporally expands the angular extent by twice the maximum angular articulation and thereby temporally expands a detectable light field size and increases the number of light detection directions by the ratio of said angular extent expansion to the angular extent along each of the two articulation axes,

wherein the individual pixel color and intensity detection addressability and the temporal association of the pixels within a pixel group with the temporally expanded set of light directions enables the detection of the color and intensity of the light impinging from each of the temporally expanded set of directions, thereby enabling the light field camera to detect the color, intensity and direction of the light that impinges its aperture from the expanded light field defined by the discrete set of directions comprising the set of the temporally expanded set of light directions;

the light field camera with the collective set of the spatial arrays of the pixel groups, each with individual pixel color, intensity and direction detection addressability, providing an expanded spatial aperture to detect the light field that impinges across the expanded spatial aperture spanning the collective set of the spatial arrays of the tiled arrays.

83. The light field camera of claim 81 wherein the light field camera captures two dimensional light field images that are then computationally focused at a desired plane or curved surface or computationally focused within a desired depth volume having flat or curved surfaces.

84. The light field camera of claim 81 configured as a three dimensional light field camera with arbitrary spatial and directional resolutions.

85. The light field camera of claim 81 having operational modes for switching between operation as a three dimensional light field camera and operation as a two dimensional light field camera by adapting the desired operational mode.

86. The light field camera of claim 81 wherein the light field camera is embedded in a mobile device and networked to other mobile devices that also have a light field camera embedded therein, the mobile devices being interconnected either through wireless or wireline networks or through bulk data transfer using flash memory modules, thus making it possible to exchange light field camera output data with a multiplicity of such mobile devices that captured a different perspective of a viewing scene, and use such networked light field camera output data to create a collective light field of the viewed scene.

87. The light field camera of claim 86 wherein the networked mobile devices are either cell phones or computers, or alternatively the light field camera is integrated with its own network connectivity interface capabilities.

88. The light field camera of claim 81 configured to record either still or motion video images.

89. The light field camera of claim 41 wherein the light field camera captures two dimensional light field images that are then computationally focused at a desired plane or curved surface or computationally focused within a desired depth volume having flat or curved surfaces.

90. The light field camera of claim 41 configured as a three dimensional light field camera with arbitrary spatial and directional resolutions.

91. The light field camera of claim 41 having operational modes for switching between operation as a three dimensional light field camera and operation as a two dimensional light field camera by adapting the desired operational mode.

92. The light field camera of claim 41 wherein the light field camera is embedded in a mobile device and networked to other mobile devices that also have a light field camera embedded therein, the mobile devices being interconnected either through wireless or wireline networks or through bulk data transfer using flash memory modules, thus making it possible to exchange light field camera output data with a multiplicity of such mobile devices that captured a different perspective of a viewing scene, and use such networked light field camera output data to create a collective light field of the viewed scene.

93. The light field camera of claim 92 wherein the networked mobile devices are either cell phones or computers, or alternatively the light field camera is integrated with its own network connectivity interface capabilities.

94. The light field camera of claim 41 configured to record either still or motion video images.

95. A method comprising:

providing a plurality of light field cameras each having;

a micro photo-detector array device having a light detection surface and

a micro lens array of micro lens elements;

whereby the micro photo-detector array device and the micro lens array are assembled together and temporally angularly articulated as a single assembly around two orthogonal axes parallel to a plane of the light detection surface of the micro photo-detector array device;

the temporal angular articulation being within a range of plus to minus a maximum angular articulation for each axis;

embedding each light field camera in a respective mobile device;

interconnecting each mobile device to at least one host device through wireless or wireline networks or through bulk data transfer using flash memory modules;

exchanging light field camera output data between mobile devices that captured a different perspective of a viewing scene and the host device to form collective light field camera output data; and

using the collective light field camera output data to create a collective light field of the viewed scene.

96. The method of claim 95 wherein each light field camera is provided with a tiled array of the micro photo-detector array devices and micro lens arrays.

97. The method of claim 95 wherein the mobile devices are cell phones, tablet PCs or light field cameras with their own connectivity capabilities.

98. The method of claim 95 wherein forming the collective light field camera output data is performed by either of the following ways:

(1) processing the collective light field camera output data by each mobile device to generate the collective light field wherein each mobile device acts as a host device;

(2) wherein the host device is a network server, and processing the collective light field camera output data by the network server, then downloading the collective light field of the viewed scene to the mobile devices.

99. The method of claim 95 wherein for each light field camera in each mobile device:

the micro photo-detector array device comprises an array of pixels whereby each of the pixels is a few microns in size and is individually addressable through the device interface contacts to output an electrical signal that is responsive to color and intensity of light coupled into its aperture,

the angular articulation temporally expands the angular extent by twice the maximum angular articulation and thereby temporally expands a detectable light field size and increases the number of light detection directions by the ratio of said angular extent expansion to the angular extent along each of the two articulation axes,

the individual pixel color and intensity detection addressability and the temporal association of the pixels within a pixel group with the temporally expanded set of light directions enabling the detection of the color and intensity of the light impinging from each of the temporally expanded set of directions, thereby enabling the light field camera to detect the color, intensity and direction of the light that impinges its aperture from the expanded light field defined by the discrete set of directions comprising the set of the temporally expanded set of light directions, and

the directional, spatial, color and intensity detection addressability of the light field camera using a multiple field data output from the light field camera for light field camera output data whereby for each designated pixel group address within the spatial array of the pixel groups, at least one output data field is used to specify the direction of the light impinging the aperture of the light field camera and at least one output data field is used to specify the color and intensity of the light detected from that designated direction;

and wherein the exchange of the light field output data of the plurality of light field cameras and the host device to create the collective light field camera output data is accomplished by;

first, augmenting the light field output data of each light field camera with data representing the camera location and orientation and capture time of the light field to form augmented light field output data,

second, exchanging, through the interconnectivity of the host device, the augmented light field camera output data to form a superset of augmented light field camera output data,

third, computationally fusing the superset of augmented light field camera output data to create a collective light field of the viewed scene, and

forth, exchanging, through the interconnectivity of the host mobile device the collective light field of the viewed scene with the plurality of mobile devices.

100. The method of claim 99 as applied to create two dimensional wide angle views or panoramas of a viewing scene captured collaboratively by multiple participants using their interconnected mobile devices, each with an embedded light field camera.

101. The method of claim 99 wherein computationally fusing the superset of augmented light field camera output data to create a collective light field of the viewed scene comprises converting the light field data of each of the mobile devices from the coordinates of the respective light field cameras to a set of viewing scene coordinates.

102. The method of claim 99 wherein computationally fusing the superset of augmented light field camera output data to create a collective light field of the viewed scene includes computationally fusing two dimensional digital camera output data to create a collective light field of the viewed scene.

103. The method of claim 102 wherein the computational light field data fusing is performed by a networked server using an internet application that allows subscribers to upload to the server the augmented light field data captured by the light field camera or corresponding data captured by a two dimensional digital camera embedded in the respective mobile device, then to download the collective light field computationally fused by the server from the data captured by all of the subscribers attending the same event.

104. The method of claim 99 used to capture three dimensional light field images of social events, for three dimensional surveillance and three dimensional cinema capture.

105. The method of claim 99 wherein the computational light field data fusing is performed by each individual participant'"'"'s mobile device or other personal computational assets of the individual participants to generate the collective light field of the viewing scene which in turn is shared among the participants using either the media through which their captured light field data was originally exchanged or a different media.

106. The method of claim 105 wherein the captured and augmented light field data and the fused collective light field data are exchanged among the participants of the networked light field photography using one or more of a mobile wireless telephone network, wireless local network, a personal area network, and bulk data transfer using flash memory modules or an internet accessible site.

107. A method comprising:

providing a plurality of two dimensional digital cameras, each in a respective mobile device, to capture the two dimensional light field data of a viewing scene;

augmenting the two dimensional light field data of a viewed scene with the location, orientation and time of capture the two dimensional light field data;

interconnecting each mobile device to at least one host device through wireless or wireline networks or through bulk data transfer using flash memory modules;

exchanging the two dimensional light field data from the two dimensional digital cameras between mobile devices that captured a different perspective of a viewing scene and a host device to form collective light field data; and

using the collective light field data to create a collective light field of the viewed scene.

108. The method of claim 107 further comprising providing data back to the mobile devices representing the collective light field.