Light Field Camera Image, File and Configuration Data, and Methods of Using, Storing and Communicating Same

• US 20100265385A1
• Filed: 02/10/2010
• Published: 10/21/2010
• Est. Priority Date: 04/18/2009
• Status: Abandoned Application

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Fishman, Alex, Ng, Yi-Ren, Knight, Timothy J., Pitts, Colvin
• Application Number:
  US12/703,367
• Publication Number:
  US 20100265385A1
• Time in Patent Office: Days
• US Class Current: 348/340
• CPC Class Codes: G02B 27/0075 with means for altering, e.g. increasing, the depth of field or depth of focus
• CPC Class Codes: H04N 5/772 the recording apparatus and the television camera being placed in the same enclosure
• CPC Class Codes: H04N 9/8205 involving the multiplexing of an additional signal and the colour video signal

Abstract

Certain devices and methods are directed to acquiring, generating and/or outputting image data corresponding to a scene. In one aspect, the method comprises (i) acquiring light field data which is representative of a light field from the scene, (ii) acquiring configuration data which is representative of how light rays optically propagate through the data acquisition device (used to acquire the light field data), (iii) generating first image data using the light field data and the configuration data, wherein the first image data includes a focus or focus depth that is different from a focus or focus depth of the light field data, (iv) generating a first electronic data file including (a) the first image data, (b) the light field data, and (c) the configuration data, and (v) outputting the first electronic data file. In one aspect, the light field acquisition device comprises optics, a light field sensor (located in the optical path of the optics) to acquire light field image data, processing circuitry to: (i) determine configuration data which is representative of how light rays optically propagate through the optics and light field sensor, and (ii) generate and output the electronic data file, wherein the electronic data file includes (a) image data, (b) light field data which is representative of a light field from the scene, and (c) configuration data. The device also includes memory (internal and/or external) to store the electronic data file.

Claims

1. A method of generating and outputting image data corresponding to a scene, the method comprising:

acquiring light field data which is representative of a light field from the scene, wherein the light field data is acquired using a data acquisition device;

acquiring configuration data which is representative of how light rays optically propagate through the data acquisition device;

generating first image data using the light field data and the configuration data, wherein the first image data includes a focus or focus depth that is different from a focus or focus depth of the light field data;

generating a first electronic data file including (i) the first image data, (ii) the light field data, and (iii) the configuration data; and

outputting the first electronic data file.

2. The method of claim 1 wherein generating the first electronic data file further includes arranging the first image data of the first electronic data file in a standard image format.

3. The method of claim 1 wherein generating the first electronic data file further includes arranging the first image data of the first electronic data file in a JPEG format.

4. The method of claim 2 wherein generating a first electronic data file further includes interleaving, threading, watermarking, encoding, multiplexing and/or meshing the first image data and the light field data.

5. The method of claim 2 wherein generating a first electronic data file further includes generating a header of the first electronic data file, wherein the header includes the configuration data.

6. The method of claim 2 further including:

reading the first electronic data file;

displaying the first image data;

receiving a user input;

generating second image data, in response to the user input, using (i) the light field data of the electronic data file and (ii) the configuration data, wherein the second image data is different from the first image data;

generating a second electronic data file including (i) the second image data, (ii) the light field data, and (iii) the configuration data; and

outputting the second electronic data file.

7. The method of claim 6 wherein the second image data includes a focus or focus depth that is different from the focus or focus depth of the first image data.

8. The method of claim 7 wherein generating the second electronic data file further includes arranging the second image data of the second electronic data file in a standard image format.

9. The method of claim 6 wherein generating a second electronic data file further includes interleaving, threading, watermarking, encoding, multiplexing and/or meshing the second image data and the light field data.

10. The method of claim 2 further including compressing the light field data to generate compressed light field data, and wherein the light field data of the first electronic data file is the compressed light field data.

11. The method of claim 2 wherein:

acquiring configuration data includes acquiring an N-bit key; and

the method further includes determining optical model data by correlating the N-bit key to predetermined optical model data and wherein generating first image data includes generating first image data using the light field data and the optical model data.

12. The method of claim 2 further including:

reading the first electronic data file;

displaying the first image data;

receiving a user input;

generating second image data, in response to the user input, using (i) the light field data of the electronic data file and (ii) the configuration data, wherein the second image data is different from the first image data;

generating a second electronic data file including the second image data; and

outputting the second electronic data file.

13. The method of claim 12 wherein the second image data includes a focus or focus depth that is different from the focus or focus depth of the first image data.

14. The method of claim 13 wherein generating the second electronic data file further includes arranging the second image data of the second electronic file in a standard image format.

15. The method of claim 1 wherein the configuration data includes data which is representative of an aperture function or an exit pupil which is associated with the acquisition of the light field data.

16. The method of claim 1 wherein the configuration data includes data which is representative of a mapping from a two-dimensional position on a captured 2D array of pixel values of the data acquisition device to a four-dimensional parameterization of the light field from the scene.

17. A system comprising:

read circuitry to read a first electronic data file which is stored in a memory, wherein the first electronic data file includes (i) first image data, (ii) light field data which is representative of a light field from a scene, and (iii) configuration data which is representative of how light rays optically propagate through a light field data acquisition device;

a display to visually output an image of the scene using the first image data;

a user interface to receive a user input;

processing circuitry, coupled to the read circuitry, display and user interface, to;

determine an optical model data using the configuration data, wherein the optical model data is representative of an optical model of the light field data acquisition device,

generate second image data, in response to the user input, using the light field data and the optical model data, wherein the second image data includes a focus or focus depth that is different from a focus or focus depth of the first image data, and

generate a second electronic data file including the second image data; and

write circuitry, coupled to the processing circuitry, to write the second electronic data file to the memory.

18. The system of claim 17 wherein the second electronic data file further includes (i) the light field data which is representative of a light field from the scene, and (ii) the configuration data and/or the optical model data.

19. The system of claim 18 wherein the configuration data includes data which is representative of an aperture function or an exit pupil which is associated with the light field data acquisition device that acquired the light field data.

20. The system of claim 18 wherein the processing circuitry generates the second electronic data file by interleaving, threading, watermarking, encoding, multiplexing and/or meshing the second image data and the light field data.

21. The system of claim 18 wherein the second electronic data file includes a header, wherein the header includes the configuration data and/or the optical model data.

22. The system of claim 18 wherein the processing circuitry generates the first electronic data file by compressing the light field data to generate compressed light field data, and wherein the light field data of the second electronic data file is the compressed light field data.

23. The system of claim 17 wherein the processing circuitry arranges the second image data of the second electronic data file in a standard image format.

24. The system of claim 17 wherein the processing circuitry arranges the second image data of the second electronic data file in a JPEG format.

25. The system of claim 17 wherein the configuration data of the first electronic data file includes an N-bit key, and wherein the processing circuitry determines the optical model data by correlating the N-bit key to a plurality of different, predetermined optical model data.

26. A light field acquisition device for acquiring light field image data of a scene, the device comprising:

optics, wherein the optics includes an optical path;

a light field sensor, located in the optical path of the optics, to acquire light field image data;

a user interface to receive a user input, wherein, in response to the user input, the light field sensor acquires the light field image data of the scene; and

processing circuitry, coupled the light field sensor and the user interface, to generate and output an electronic data file, the processing circuitry to;

determine configuration data which is representative of how light rays optically propagate through the optics and light field sensor; and

generate and output the electronic data file, wherein the electronic data file includes (i) image data, (ii) light field data which is representative of a light field from the scene, and (iii) configuration data; and

memory, coupled to the processing circuitry, to store the electronic data file therein.

27. The device of claim 26 wherein the configuration data includes data which is representative of an aperture function or exit pupil of the light field acquisition device.

28. The device of claim 26 wherein the configuration data includes data which is representative of a mapping from a two-dimensional position on a captured 2D array of pixel values to a four-dimensional parameterization of a light field from the scene.

29. The device of claim 26 wherein the processing circuitry generates the electronic data file by interleaving, threading, watermarking, encoding, multiplexing and/or meshing the image data and the light field data.

30. The device of claim 26 wherein the processing circuitry generates a header which includes the configuration data, wherein the electronic data file includes the header.

31. The device of claim 26 wherein the processing circuitry generates the electronic data file by compressing the light field data to generate compressed light field data, and wherein the light field data of the electronic data file is the compressed light field data.

32. The device of claim 26 wherein the processing circuitry arranges the image data of the electronic data file in a standard image format.

33. The device of claim 26 wherein the processing circuitry arranges the image data of the electronic data file in a JPEG format.

34. The device of claim 26 wherein the configuration data of the electronic data file includes an N-bit key which is representative of predetermined optical model data.