SELECTIVE CULLING OF MULTI-DIMENSIONAL DATA SETS

• US 20180063504A1
• Filed: 08/29/2017
• Published: 03/01/2018
• Est. Priority Date: 08/29/2016
• Status: Active Grant

• Current Assignee: Stratus Systems, Inc.
• Original Assignee: Stratusvr, Inc.
• Inventors: Haines, Russell T.
• Granted Patent: US 10,659,759 B2
• Time in Patent Office: Days
• CPC Class Codes: H04N 13/156 Mixing image signals
• CPC Class Codes: H04N 13/167 Synchronising or controlling image signals
• CPC Class Codes: H04N 13/178 Metadata, e.g. disparity information
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays
• CPC Class Codes: H04N 13/383 for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
• CPC Class Codes: H04N 5/60 for the sound signals
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

Method and system for selective culling of multi-dimensional data sets. Selective culling system introduces a feedback control method for remote users (e.g., client-side) to customize the selection, transmission, and display of datasets (e.g., on server-side) or any collection or subset of datasets.

Claims

1. A computer-implemented method for selectively culling data sets at data source for a viewer of a display system, comprising:

(a) determining a control command based at least on requested data sets, field of interest of the viewer, and available bandwidth;

(b) receiving, at a server, the control command, wherein the control command comprises encoding of instructions on which one or more subsets among the data sets are to be culled;

(c) selectively culling, at the server, one or more subsets among the requested data sets according to the transmitted control command; and

(d) serving, to the display system, the selectively culled data set.

2. The method of claim 1, wherein the data sets comprise video and audio content.

3. The method of claim 2, wherein the data sets further comprise sensor data.

4. The method of claim 1, wherein the control command is further based at least on display system information, wherein the display system information includes display type, display resolution, display size, display frame rate, and distance from the user to the display system.

5. The method of claim 4, wherein the control command is further based at least on metatags and metadata embedded or associated with the requested data sets.

6. The method of claim 4, wherein the size of the control command is 128 bytes or less.

7. The method of claim 2, wherein the video and audio content comprises two-dimensional, three-dimensional, or virtual reality video and audio content.

8. The method of claim 7, wherein the control command information is further based at least on foveal area and foveal area target shape.

9. The method of claim 8, wherein the display system is a head-mounted display (HMD) system.

10. The method of claim 9, wherein the field of interest information is obtained by monitoring one or more sensors local to the viewer.

11. The method of claim 10, wherein the field of interest information is further adjusted based on user input methods, wherein the user input methods comprise joystick, touch display, and keyboard.

12. The method of claim 10, wherein the field of interest information is further adjusted based on a class of input methods comprising knobs, fader, and buttons.

13. The method of claim 10, wherein the one or more sensors are configured to track body movement, head movement, and eye movement of the viewer.

14. The method of claim 10, wherein the one or more sensors are further configured to track stereo view information for each eye of the viewer.

15. The method of claim 13, wherein the X, Y, and Z coordinate information of the eye, head, and body movements are tracked.

16. The method of claim 15, wherein the one or more sensors are configured to track the movements over time, thereby calculating acceleration and interpolation of the field of interest.

17. The method of claim 13, wherein the one or more subsets that are within the viewer'"'"'s field of interest are selectively culled.

18. The method of claim 17, wherein the amount of culling is configured to be proportional to the available or desired bandwidth between the server and the display system.

19. A system for selectively culling data sets at a data source for viewers of immersive display systems, comprising:

(a) an immersive display system comprising sensor modules;

(b) a processor that is communicatively coupled to a server system and adapted to execute stored instructions; and

(c) a memory device, communicatively coupled to the processor, wherein the memory device is configured to;

determine a control command based at least on requested data sets, field of interest of the viewer, and available bandwidth;

receive, at the server system, the control command, wherein the control command comprises encoding of instructions on which one or more subsets among the data sets are to be culled;

selectively cull, at the server system, one or more subsets among the requested data sets according to the transmitted control command; and

serve, to the display system, the selectively culled data set.

20. The system of claim 19, wherein the control command is further based at least on the immersive display system information, wherein the immersive display system information includes display type, display resolution, display size, display frame rate, and distance from the user to the display system.

21. The system of claim 19, wherein the data sets comprise video and audio content.

22. The system of claim 21, wherein the field of interest of the viewer is determined based at least on data obtained by the sensor modules.

23. The system of claim 22, wherein the sensor modules are configured to track body movement, head movement, eye movement of the viewer.

24. The system of claim 23, wherein the sensor modules are further configured to track stereo view information for each eye.

25. The system of claim 23, wherein the immersive display system is a head-mounted display (HMD).

26. The system of claim 19, wherein the amount of culling is configured to be proportional to the available or desired bandwidth between the server system and the immersive display system.