Preprocessor for Full Parallax Light Field Compression

• US 20160021355A1
• Filed: 07/14/2015
• Published: 01/21/2016
• Est. Priority Date: 07/15/2014
• Status: Abandoned Application

• Current Assignee: Ostendo Technologies Inc.
• Original Assignee: Ostendo Technologies Inc.
• Inventors: Alpaslan, Zahir Y., Graziosi, Danillo B., El-Ghoroury, Hussein S.
• Application Number:
  US14/799,269
• Publication Number:
  US 20160021355A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: H04N 13/106 Processing image signals for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/30 Image reproducers optical systems for producing stereoscopic or other three-dimensional effects G02B30/00
• CPC Class Codes: H04N 19/162 User input
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression

Abstract

Preprocessing of the light field input data for full parallax compressed light field 3D display systems is described. The described light field input data preprocessing can be utilized to format or extract information from input data, which can then be used by the light field compression system to further enhance the compression performance, reduce processing requirements, achieve real-time performance and reduce power consumption. This light field input data preprocessing performs a high-level 3D scene analysis and extracts data properties to be used by the light field compression system at different stages. As a result, rendering of redundant data is avoided while at the same rendering quality is improved.

Claims

1. A preprocessor for a light field display system that provides full parallax, compressed, three dimensional processing of light field input data, the preprocessor comprising:

a data receiver that receives light field input data in a data space;

a display configuration receiver that receives configuration information for the light field display system; and

a display space converter that converts the data space of the light field input data responsive to the configuration information for the light field display system.

2. The preprocessor of claim 1 wherein the configuration information for the light field display system includes position information for a light field modulation surface of the light field display system.

3. The preprocessor of claim 2 wherein the display space converter converts the data space of the light field input data responsive to a distance between an object in the light field input data and the light field modulation surface of the light field display system.

4. The preprocessor of claim 2 further comprising a list generator that creates an ordered list of 3D planes representing objects in the light field input data, ordered by their distances to the light field modulation surface of the light field display system.

5. A method of preprocessing light field input data for a light field display system that provides full parallax, compressed, three dimensional processing of light field input data, the method comprising:

receiving the light field input data in a data space;

receiving configuration information for the light field display system; and

converting the data space of the light field input data responsive to the configuration information for the light field display system.

6. The method of claim 5 wherein the configuration information for the light field display system includes position information for a light field modulation surface of the light field display system.

7. The method of claim 6 further comprising converting the data space of the light field input data responsive to a distance between an object in the light field input data and the light field modulation surface of the light field display system.

8. The method of claim 6 further comprising creating an ordered list of 3D planes representing objects in the light field input data, ordered by their distances to the light field modulation surface of the light field display system.

9. A preprocessor for a light field display system that provides full parallax, compressed, three dimensional processing of light field input data, the preprocessor comprising:

an interaction receiver that receives selections generated interactively by a user; and

a data receiver that accesses light field input data to be provided to the light field display system to respond to the selections generated interactively by the user.

10. The preprocessor of claim 9 wherein the selections generated interactively by the user include motion vector data, and the data receiver preemptively accesses the light field input data responsive to the motion vector data in anticipation of the light field input data that will be provided to the light field display system.

11. The preprocessor of claim 9 wherein the selections generated interactively by the user include zoom information, and the data receiver preemptively accesses the light field input data responsive to the zoom information in anticipation of the light field input data that will be provided to the light field display system.

12. The preprocessor of claim 9 wherein the selections generated interactively by the user includes a display mode change, and the data receiver accesses the light field input data to provide the light field input data to the light field display system responsive to the display mode change.

13. The preprocessor of claim 9 further comprising:

a first storage device that stores the light field input data, the first storage device having a first transfer speed; and

a second storage device having a second transfer speed that is faster than the first transfer speed;

wherein the preprocessor is coupled to the first storage device and the second storage device, the preprocessor receiving the light field input data from the first storage device and storing selected portions of the light field input data on the second storage device to respond to the selections generated interactively by the user.

14. A method of preprocessing light field input data for a light field display system that provides full parallax, compressed, three dimensional processing of light field input data, the method comprising:

receiving selections generated interactively by a user; and

accessing light field input data to be provided to the light field display system to respond to the selections generated interactively by the user.

15. The method of claim 14 wherein the selections generated interactively by the user include motion vector data, and accessing the light field input data further comprises preemptively accessing the light field input data responsive to the motion vector data in anticipation of the light field input data that will be provided to the light field display system.

16. The method of claim 14 wherein the selections generated interactively by the user include zoom information, and accessing the light field input data further comprises preemptively accessing the light field input data responsive to the zoom information in anticipation of the light field input data that will be provided to the light field display system.

17. The method of claim 14 wherein the selections generated interactively by the user include a display mode change, and accessing the light field input data further comprises accessing the light field input data to provide the light field input data to the light field display system responsive to the display mode change.

18. The method of claim 14 further comprising:

storing the light field input data on a first storage device having a first transfer speed; and

receiving the light field input data from the first storage device and storing selected portions of the light field input data on a second storage device to respond to the selections generated interactively by the user, the second storage device having a second transfer speed that is faster than the first transfer speed.

19. A preprocessor for a light field display system that provides full parallax, compressed, three dimensional processing of light field input data, the preprocessor comprising:

a data receiver that receives light field input data;

an object identifier that identifies a plurality of three dimensional objects in the light field input data to be displayed on a light field display;

a boundary identifier that generates a list of object boundary representations for the plurality of three dimensional objects.

20. The preprocessor of claim 19 wherein the boundary identifier:

finds minimum coordinate values and maximum coordinate values of vertices for each of the plurality of three dimensional objects to define a bounding box aligned with a light field modulation surface of the light field display;

selects a face of the bounding box that is parallel to and closest to the light field modulation surface of the light field display; and

includes the selected face in the list of object boundary representations.

21. The preprocessor of claim 20 wherein the boundary identifier first defines an unaligned bounding box and then defines the bounding box aligned with the light field modulation surface of the light field display for the unaligned bounding box for each of the plurality of three dimensional objects.

22. The preprocessor of claim 20 wherein the list of object boundary representations is ordered according to a distance of the selected face of the bounding box from the light field modulation surface of the light field display.

23. The preprocessor of claim 22 wherein the ordering according to the distance of the selected face of the bounding box from the light field modulation surface is without regard to whether the selected face of the bounding box is in front of or behind the light field modulation surface.

24. The preprocessor of claim 19 further comprising a display configuration receiver that receives position information for a light field modulation surface from the light field display.

25. The preprocessor of claim 19 further comprising:

a first storage device that stores the light field input data, the first storage device having a first transfer speed; and

a second storage device having a second transfer speed that is faster than the first transfer speed;

wherein the preprocessor is coupled to the first storage device and the second storage device, the preprocessor receiving the light field input data from the first storage device and storing selected portions of the light field input data on the second storage device.

26. A method of preprocessing light field input data for a light field display system that provides full parallax, compressed, three dimensional processing of light field input data, the method comprising:

identifying a plurality of three dimensional objects in light field input data to be displayed on a light field display;

generating a list of object boundary representations for the plurality of three dimensional objects.

27. The method of claim 26 further comprising:

finding minimum coordinate values and maximum coordinate values of vertices for each of the plurality of three dimensional objects to define a bounding box aligned with a light field modulation surface of the light field display;

selecting a face of the bounding box that is parallel to and closest to the light field modulation surface of the light field display; and

including the selected face in the list of object boundary representations.

28. The method of claim 27 further comprising first defining an unaligned bounding box and then defining the bounding box aligned with the light field modulation surface of the light field display for the unaligned bounding box for each of the plurality of three dimensional objects.

29. The method of claim 27 wherein the list of object boundary representations is ordered according to a distance of the selected face of the bounding box from the light field modulation surface of the light field display.

30. The method of claim 29 wherein the ordering according to the distance of the selected face of the bounding box from the light field modulation surface is without regard to whether the selected face of the bounding box is in front of or behind the light field modulation surface.

31. The method of claim 26 further comprising receiving position information for a light field modulation surface from the light field display.

32. The method of claim 30 further comprising:

storing the light field input data on a first storage device having a first transfer speed; and

receiving the light field input data from the first storage device and storing selected portions of the light field input data on a second storage device having a second transfer speed that is faster than the first transfer speed.

33. A preprocessor for a light field display system that provides full parallax, compressed, three dimensional processing of light field input data, the preprocessor comprising:

a data receiver that receives light field input data from a first storage device that stores the light field input data, the first storage device having a first transfer speed;

a light field data classifier that identifies portions of the light field input data to be processed by the light field display system; and

a data transmitter that stores the identified portions of the light field input data on a second storage device having a second transfer speed that is faster than the first transfer speed, the second storage device being coupled to the light field display system.

34. The preprocessor of claim 33 wherein the light field data classifier identifies portions of the light field input data that represent scenes adjacent to a scene being displayed as data to be processed by the light field display system.

35. The preprocessor of claim 33 wherein the light field data classifier identifies portions of the light field input data that represent a view of a portion of a scene being displayed as data to be processed by the light field display system.

36. The preprocessor of claim 33 wherein the data transmitter stores the identified portions of the light field input data on the second storage device such that the light field input data for objects that are closer to a light field modulation surface of a light field display can be transferred with priority access.

37. The preprocessor of claim 33 further comprising a list generator that creates an ordered list of 3D planes representing objects in the identified portions of the light field input data, ordered by their distances to a light field modulation surface of the light field display system.

38. A method of preprocessing light field input data for a light field display system that provides full parallax, compressed, three dimensional processing of light field input data, the method comprising:

receiving light field input data from a first storage device that stores the light field input data, the first storage device having a first transfer speed;

identifying portions of the light field input data to be processed by the light field display system; and

storing the identified portions of the light field input data on a second storage device having a second transfer speed that is faster than the first transfer speed, the second storage device being coupled to the light field display system.

39. The method of claim 38 wherein portions of the light field input data that represent scenes adjacent to a scene being displayed are identified as data to be processed by the light field display system.

40. The method of claim 38 wherein portions of the light field input data that represent a view of a portion of a scene being displayed are identified as data to be processed by the light field display system.

41. The method of claim 38 wherein the identified portions of the light field input data are stored on the second storage device such that light field input data for objects that are closer to a light field modulation surface of a light field display can be transferred with priority access.

42. The method of claim 38 further comprising creating an ordered list of 3D planes representing objects in the identified portions of the light field input data, ordered by their distances to a light field modulation surface of the light field display system.