Perspective altering display system

• US 10,063,848 B2
• Filed: 08/25/2008
• Issued: 08/28/2018
• Est. Priority Date: 08/24/2007
• Status: Active Grant

• Current Assignee: Videa, L.L.C. (Barry Schwab)
• Original Assignee: Videa, L.L.C. (Barry Schwab)
• Inventors: Posa, John G., Schwab, Barry H.
• Primary Examiner(s): HARRISON, CHANTE E
• Application Number:
  US12/197,635
• Publication Number:
  US 20090051699A1
• Time in Patent Office: 3,655 Days
• Field of Search: 345427, 345156, 345660, 345647-648, 345672, 345619, 345629, 345680, 345682, 348 46, 348 51, 348 1407, 348563, 348578, 348583, 715719
• CPC Class Codes: G09G 2340/10 Mixing of images, i.e. displayed pixel being the result of an operation, e.g. adding, on the corresponding input pixels
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 2340/125 wherein one of the images is motion video
• CPC Class Codes: H04N 13/275 from 3D object models, e.g. computer-generated stereoscopic image signals
• CPC Class Codes: H04N 13/279 the virtual viewpoint locations being selected by the viewers or determined by tracking
• CPC Class Codes: H04N 13/366 using viewer tracking
• CPC Class Codes: H04N 13/368 for two or more viewers
• CPC Class Codes: H04N 13/373 for tracking forward-backward translational head movements, i.e. longitudinal movements
• CPC Class Codes: H04N 21/42202 environmental sensors, e.g. for detecting temperature, luminosity, pressure, earthquakes
• CPC Class Codes: H04N 21/42222 Additional components integrated in the remote control device, e.g. timer, speaker, sensors for detecting position, direction or movement of the remote control, microphone or battery charging device
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/431 Generation of visual interfaces for content selection or interaction; Content or additional data rendering
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 2213/006 Pseudo-stereoscopic systems, i.e. systems wherein a stereoscopic effect is obtained without sending different images to the viewer's eyes

Abstract

The perception of a displayed image is altered for viewers moving relative to the position of the display system screen, thereby imparting a sense of three-dimensional immersion in the scene being displayed. A display generator generates a scene having foreground and background elements, and a display screen displaying the scene. A sensor detects the position of a viewer relative to the display screen, and a processor is operative to shift the relative position of the foreground and background elements in the displayed scene as a function of viewer position, such that the viewer'"'"'s perspective of the scene changes as the viewer moves relative to the display screen. The foreground and background elements may be presented in the form of multiple superimposed graphics planes, and/or a camera may be used to record the scene through panning at sequential angles. The system may be used to implement virtual windows, virtual mirrors and other effects without requiring viewers or users to modify behavior or wear glasses, beacons, etc.

Claims

1. A perspective-altering display system, comprising:

a display screen for displaying a scene having opposing side edges;

a sensor for detecting the movement of a viewer relative to the display screen;

at least one memory for storing foreground and background imagery;

wherein the memory has a capacity to store imagery that extends beyond one or both of the side edges of the display screen;

at least one processor interconnected to the memory and the sensor, the processor being operative to perform the following functions;

generate a scene based upon multiple superimposed graphics planes, including a foreground graphics plane containing the foreground imagery and a background graphics plane containing the background imagery, and

receive a signal from the sensor and shift the relative position of the foreground and background graphics planes in the displayed scene as a function of viewer movement;

wherein the multiple superimposed graphics planes include non-visible imagery beyond one of both of the side edges display screen; and

wherein the foreground graphics plane is shifted by way of a translation to a greater extent than the background graphics plane as a viewer moves from side to side relative to the display screen, causing the non-visible imagery to become visible.

2. The display system of claim 1, wherein at least one of the graphics planes includes prerecorded imagery.

3. The display system of claim 1, wherein at least one of the graphics planes includes imagery received through a transmission medium.

4. The display system of claim 1, wherein at least one of the graphics planes includes imagery received from a camera.

5. The display system of claim 1, wherein at least one of the graphics planes includes textual information.

6. The display system of claim 1, further including:

a camera that records the scene through panning at sequential angles; and

wherein the memory stores imagery obtained at the sequential angles for later recall as a function of user movement.

7. The display system of claim 6, further including:

an interpolator for filling in visual gaps in the scene.

8. The display system of claim 1, wherein:

the display screen is mounted on a wall having a backside; and

further including a camera mounted on the backside of the wall which pans as a viewer moves, thereby imaging a scene representative of the display being a virtual window through the wall.

9. The display system of claim 1, further including:

a camera with a zoom capability for recording the scene;

the sensor being operative to detect the viewer'"'"'s distance from the display screen; and

wherein the processor is further operative to zoom in the camera as the viewer moves closer to the display screen and zoom out the camera as the viewer moves away from the display screen.

10. The display system of claim 1, wherein:

the sensor is operative to detect the viewer'"'"'s distance from the display screen; and

the processor is operative to increase the resolution of the scene as the viewer moves closer to the display screen and decrease the resolution of the scene as the viewer moves away from the display screen.

11. The display system of claim 1, further including:

a camera with a tilt capability for recording the scene;

the sensor being operative to detect the viewer'"'"'s up/down movement relative to the display screen; and

wherein the processor is further operative to tilt the camera in response to the viewer'"'"'s up/down movement.

12. The display system of claim 1, further including:

a camera for recording the scene, the camera having a field of view including a viewer of the display screen; and

wherein the processor is further operative to flip the scene horizontally such that the display functions as a virtual mirror.

13. The display system of claim 1, further including:

a plurality of cameras for recording the scene; and

wherein the processor is further operative to construct a three-dimensional image for display on the screen.

14. The display system of claim 1, further including:

a plurality of cameras for recording the scene;

the processor being further operative to construct a three-dimensional image for display on the screen; and

a user control a viewer to select a specific camera or cameras to see how others would view the user from different perspectives.

15. The display system of claim 1, wherein the sensor is an infrared CCD (charge-coupled device) camera.

16. The display system of claim 1, wherein:

the sensor is a camera having an image sensor; and

an image of a person in front of the display is focused onto the image sensor as a spot or group of pixels, allowing the movements of the person to be tracked with no moving parts.

17. The display system of claim 1, wherein:

the sensor is a camera having at least a pan mount that tracks the movement of a viewer; and

the processor is operative to shift the relative position of the foreground and background elements as a function of the tracking.

18. The display system of claim 1, wherein:

the sensor is a camera having a pan/tilt mount that tracks the movement of a viewer;

the processor is operative to shift the relative position of the foreground and background elements as a function of the tracking.

19. The display system of claim 1, wherein:

the sensor is a camera having an auto-focus capability to track the movement of a viewer;

the processor is operative to shift the relative position of the foreground and background elements as a function of the tracking.

20. The display system of claim 1, wherein:

the sensor is operative to sense a plurality of individuals in front of the display screen; and

the processor is operative to shift the relative position of the foreground and background elements by favoring clusters of individuals.

21. The display system of claim 1, wherein:

the sensor is operative to sense a plurality of individuals in front of the display screen; and

the processor is operative to shift the relative position of the foreground and background elements by favoring moving individuals over stationary individuals.

22. The display system of claim 1, wherein:

the sensor is operative to sense a whether an individual is looking toward the display screen; and

the processor is operative to shift the relative position of the foreground and background elements by favoring individuals looking toward the display screen.

23. The display system of claim 1, further including one or more transducers for producing sounds associated with the scene;

and

the processor is operative to alter the reproduction of the sounds as a function of viewer movement.