System and method of perception-based image generation and encoding
First Claim
1. In a computer-implemented method of rendering a sequence of images of a 3-D model space, wherein the model space comprises a plurality of graphics primitives, the method of rendering each of said graphics primitives at a spatial resolution that reflects an ability of a human visual system to spatially resolve said graphics primitive, the method comprising the steps, for each of said graphics primitives, of:
- a) determining an initial exposure time at which a graphics primitive becomes exposed in an image sequence;
b) storing the initial exposure time for said graphics primitive;
c) determining, for said graphics primitive, an exposure duration corresponding to a current frame of the image sequence by subtracting (1) a current time corresponding to said current frame from (2) said initial exposure time;
d) rendering, for said current frame, said graphics primitive at a spatial resolution that is proportional to the exposure duration of said graphics primitive if said exposure duration is substantially less than a known time required for the human visual system to maximally resolve a newly exposed target.
5 Assignments
0 Petitions
Accused Products
Abstract
A computer implemented method of image generation that employs adaptive, progressive, perception-based spatio-temporal importance sampling to reduce the cost of image generation. The method uses an adaptive approach to sampling which employs refinement criteria based on specific spatio-temporal limits of human vision. By using these refinement criteria the method produces an image with a spatio-temporal structure that is closely matched to the spatio-temporal limits of human vision. Using one sampling criteria the spatial sampling density is adjusted in proportion to the sampled region'"'"'s exposure duration in a manner that substantially reflects the visual systems'"'"'s increase in acuity as a function of exposure time. Using other criteria the spatial and temporal sampling frequencies of a region of the image stream are adjusted based on the measured or estimated retinal velocity of the sampled element in a manner that substantially reflects both dynamic visual acuity limits of vision and the critical temporal sampling frequency for the perception of smooth motion. The method includes image parallel, shared-memory multiprocessor implementations based on sample reprojection and primitive reprojection, the latter using a technique of adaptive rasterization. In these implementations the natural temporal image coherence and temporal visibility coherence of the image stream produce a temporal locality of data reference that enhances performance of the system. Because temporal image coherence and temporal visibility coherence increase the spatial resolving performance of the human visual system, the performance of the present method parallels the performance of human vision making performance degradations relatively invisible to the user.
558 Citations
16 Claims
-
1. In a computer-implemented method of rendering a sequence of images of a 3-D model space, wherein the model space comprises a plurality of graphics primitives, the method of rendering each of said graphics primitives at a spatial resolution that reflects an ability of a human visual system to spatially resolve said graphics primitive, the method comprising the steps, for each of said graphics primitives, of:
-
a) determining an initial exposure time at which a graphics primitive becomes exposed in an image sequence; b) storing the initial exposure time for said graphics primitive; c) determining, for said graphics primitive, an exposure duration corresponding to a current frame of the image sequence by subtracting (1) a current time corresponding to said current frame from (2) said initial exposure time; d) rendering, for said current frame, said graphics primitive at a spatial resolution that is proportional to the exposure duration of said graphics primitive if said exposure duration is substantially less than a known time required for the human visual system to maximally resolve a newly exposed target. - View Dependent Claims (2, 3, 6)
-
-
4. In a computer-implemented method of rendering a sequence of images of a 3-D model space, wherein the model space comprises a plurality of objects wherein each object is represented at a plurality of levels of detail, the method of rendering said objects at a level-of-detail that reflects an ability of a human visual system to spatially resolve said objects comprising the steps, for each of said objects, of:
-
a) determining an initial exposure time at which an object becomes exposed in an image sequence; b) storing, for said object, the initial exposure time; c) determining, for said object, an exposure duration corresponding to a current frame of said image sequence by subtracting a time corresponding to said current frame from said initial exposure time; d) rendering, for said current frame, said object by rendering a representation of said object having a level-of-detail that is proportional to the exposure duration of said object if the exposure duration is substantially less than a known time required for the human visual system to maximally resolve a newly exposed object. - View Dependent Claims (5)
-
-
7. In a computer-implemented method of encoding graphics information for an image sequence of a 3-D model space, the model space comprising graphics primitives representing a plurality of objects at a plurality of levels of detail, by determining and storing, for substantially each image of said image sequence, visibility event information comprising a set of graphics primitives that becomes newly visible during an image of said image sequence, the method of determining and storing newly visible graphics primitives that reflect an ability of a human visual system to spatially resolve newly exposed objects comprising the steps of:
-
a) determining, for a current frame of said image sequence, objects that are newly visible; and b) storing, for the current frame of said image sequence, newly visible graphics primitives from source objects represented at a low level-of-detail.
-
-
8. In a computer-implemented method of encoding graphics information for an image sequence of a 3-D model space, the model space comprising graphics primitives defining a plurality of objects, each represented at a plurality of levels of detail, by determining and storing, for each image of said image sequence, visibility event information comprising a set of graphics primitives that becomes newly visible during an image of said image sequence, the method of determining and storing newly visible primitives that reflect an ability of a human visual system to spatially resolve the corresponding newly exposed object comprising the steps of:
-
a) determining, for each of said plurality of objects in the 3-D model space, for each level-of-detail representation of said plurality of objects, a threshold exposure duration at which the human visual system cannot substantially distinguish between each level-of-detail representation and a next higher level-of-detail representation of said object; b) storing the threshold exposure duration for each of said level-of-detail representations of said plurality of objects; c) determining an initial exposure time at which an object becomes exposed in said image sequence; d) storing, for each exposed object, the initial exposure time; e) storing, for each exposed object, an indication of the level-of-detail representation of the object that was last selected to determine and encode newly visible graphics primitives of said object; f) determining, for each exposed object, an exposure duration of said object for a current frame of said image sequence by subtracting a time corresponding to the current frame from said initial exposure time; g) comparing, for each exposed object in the current frame of said image sequence, (1) the exposure duration of said exposed object to (2) the threshold exposure duration corresponding to the level-of-detail representation of the object that was last selected to determine and encode newly visible primitives; and h) replacing previously encoded graphics primitives visible in the current frame with graphics primitives from a higher level-of-detail source object representation if, the step g) of comparing, determines that the exposure duration for the source object exceeds the threshold exposure duration for the corresponding level-of-detail representation of the source object that was last selected to determine and encode newly visible graphics primitives.
-
-
9. In a computer-implemented method of rendering an image sequence of a 3-D model space, wherein the model space comprises graphics primitives defining a plurality of objects wherein each object is represented at a plurality of levels of detail, the method of rendering, by primitive reprojection, newly exposed objects at a level-of-detail that reflects an ability of a human visual system to spatially resolve said objects comprising the steps of:
-
a) transforming and rasterizing previously exposed graphics primitives; b) identifying regions in a current image of said sequence of images that may contain newly exposed graphics primitives; c) searching said regions of the current image for newly visible graphics primitives using an image-order, front-to-back visibility tracing in a 3-D model space containing objects represented at a low level-of-detail; and d) rasterizing the newly visible graphics primitives. - View Dependent Claims (10)
-
-
11. In a computer-implemented method of encoding graphics information for an image sequence of a 3-D model space, the model space comprising graphics primitives defining a plurality of objects wherein each object is represented at a plurality of levels of detail, by determining and storing, for each image of said image sequence. visibility event information comprising a set of graphics primitives that becomes newly visible during an image of said image sequence, the method of determining and storing newly visible graphics primitives by selecting said graphics primitives from an object having a level-of-detail that reflects an ability of a human visual system to spatially resolve said object, the method comprising the steps of:
-
a) determining, for each of said objects in the 3-D model space, for each level-of-detail representation of said objects, a threshold image-space velocity above which the human visual system cannot substantially distinguish between each level-of-detail representation and a next higher level-of-detail representation of said object; b) storing the threshold image-space velocity for each of said level-of-detail representations of objects; c) storing, for each object visible in a current frame of said image sequence, an indication of the level-of-detail representation of the object that was last selected to determine and encode newly visible graphics primitives of said object; d) determining, for each object visible in the current frame of said image sequence, the image-space velocity of the object; e) comparing, for each object visible in the current frame of said image sequence, (1) the image-space velocity of said exposed object to (2) said threshold image-space velocity corresponding to the level-of-detail representation of the object that was last selected to determine and encode newly visible primitives; and f) replacing previously encoded graphics primitives visible in the current frame with graphics primitives from a higher level-of-detail source object representation if the step g) of comparing determines that the image-space velocity for the source object exceeds the threshold exposure duration for the corresponding level-of-detail representation of the source object that was last selected to determine and encode newly visible graphics primitives.
-
-
12. A method of load-balancing a shared-memory multiprocessor that implements a computer-implemented method of rendering an image sequence, wherein each subprocessor of said multiprocessor is assigned to render at least one subimage region of an image, the method comprising the steps of:
-
tracking eye movements of a viewer in real-time; and assigning each subimage processor to a subimage region that corresponds to a specific region of a visual field of the viewer. - View Dependent Claims (13, 14, 15, 16)
-
Specification