Synthesizing three-dimensional surround visual field

• US 20070126864A1
• Filed: 03/28/2006
• Published: 06/07/2007
• Est. Priority Date: 12/05/2005
• Status: Abandoned Application

• Current Assignee: Seiko Epson Corporation (Seiko Group)
• Original Assignee: Seiko Epson Corporation (Seiko Group)
• Inventors: Tan, Kar-Han, Bhattacharjya, Anoop, Bhat, Kiran
• Application Number:
  US11/390,907
• Publication Number:
  US 20070126864A1
• Time in Patent Office: Days
• US Class Current: 348/51
• CPC Class Codes: H04N 21/4122 additional display device, e.g. video projector digital output for controlling a plurality of local displays G06F3/1423
• CPC Class Codes: H04N 21/4131 home appliance, e.g. lighting, air conditioning system, metering devices home automation data switching networks exchanging configuration information on appliance services H04L12/2807
• CPC Class Codes: H04N 21/42202 environmental sensors, e.g. for detecting temperature, luminosity, pressure, earthquakes
• CPC Class Codes: H04N 21/42222 Additional components integrated in the remote control device, e.g. timer, speaker, sensors for detecting position, direction or movement of the remote control, microphone or battery charging device
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/4318 by altering the content in the rendering process, e.g. blanking, blurring or masking an image region image enhancement or restoration in general G06T5/00
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/47 End-user applications
• CPC Class Codes: H04N 5/144 Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20
• CPC Class Codes: H04N 9/3179 Video signal processing therefor
• CPC Class Codes: H04N 9/74 for obtaining special effects H04N9/65 - H04N9/73 take precedence

Abstract

A surround visual field that has a characteristic or characteristics which relate to an audio/visual presentation is described. In one embodiment, the surround visual field is displayed in an area partially surrounding or surrounding the video content being displayed. This surround visual field may be comprised of a plurality of elements that further enhance the effect of the content being displayed. For example, one embodiment of the invention provides for elements within the surround visual field to move in relation to motion within the video content being displayed. Other characteristics of the video content may also be supplemented by the surround visual field or the surround visual field may be authored, at least in part, to correspond to the content itself. In embodiments, the surround visual field may be a rendering of a three-dimensional environment. In embodiments, one or more otherwise idle display areas may be used to display a surround visual field.

Claims

1. A surround visual field system comprising:

a surround visual field controller that obtains at least one control signal related to an input stream and generates a three-dimensional surround visual field environment comprising a plurality of elements, wherein at least one element within the three-dimensional surround visual field is affected by the control signal; and

a display device, communicatively coupled to the surround visual field controller, that displays the input stream in a first area and displays the surround visual field in a second area that at least partially surrounds the first area.

2. The system of claim 1 wherein the at least one control signal is obtained from at least one source selected from the group comprising:

video data, audio data, game controller, input device, remote control, sensor, and authored cue.

3. The system of claim 2 wherein the video data comprises information related to at least one selected from the group comprising:

color, location, motion, and content.

4. The system of claim 2 wherein a motion model is used to approximate motion from the at least one control signal.

5. The system of claim 4 wherein the motion model is an affine motion model.

6. The system of claim 1 wherein the display device comprises a first display device that displays the input stream in the first area and a second display device that displays the surround visual field in the second area.

7. The system of claim 2 wherein the surround visual field controller comprises a physics-based model and the at least one element is affected in a realistic manner.

8. The system of claim 2 wherein the surround visual field controller comprises utilizes an image-based method for rendering at least a portion of the surround visual field.

9. A method of generating a surround visual field that relates to an input stream, the method comprising:

extracting a control signal related to the input stream; and

generating a surround visual field based upon a three-dimensional environment comprising a plurality of elements, wherein at least one element within the three-dimensional environment is affected by the control signal.

10. The method of claim 9 further comprising the steps of:

displaying the input stream in a first area; and

displaying the surround visual field in a second area that at least partially surrounds the first area.

11. The method of claim 9 wherein the control signal is extracted from at least one source selected from the group comprising:

video data, audio data, game controller, input device, remote control, sensor, and authored cue.

12. The method of claim 9 wherein the step of generating a surround visual field based upon a three-dimensional environment comprising a plurality of elements, wherein at least one element within the three-dimensional environment is affected by the control signal comprises the steps of:

using a motion model to generate a motion field that approximates motion from the input stream;

decomposing the motion field into pan, tilt, and zoom components about an image center;

using the pan, tilt, and zoom components to control the direction of a virtual camera motion within the three-dimensional environment.

13. The method of claim 12 wherein the motion model is an affine motion model.

14. The method of claim 9 wherein the step of generating a surround visual field based upon a three-dimensional environment comprising a plurality of elements, wherein at least one element within the three-dimensional environment is affected by the control signal comprises the step of:

using a physics-based model to affect the at least one element in a realistic manner.

15. The method of claim 9 wherein the step of generating a surround visual field based upon a three-dimensional environment comprising a plurality of elements, wherein at least one element within the three-dimensional environment is affected by the control signal comprises the step of:

using an image-based method for rendering at least a portion of the surround visual field.

16. A computer-readable medium carrying one or more sequences of instructions which, when executed by one or more processors, cause the one or more processors to perform at least the steps of the method of claim 9.

17. A surround visual field controller comprising:

a control signal extractor, coupled to receive an input source related to an input stream displayed in a first area, that obtains at least one control signal from the input source; and

a surround visual field generator, coupled to the control signal extractor, that receives the at least one control signal and generates an effect on at least one element within the surround visual field in relation to a three-dimensional rendering of the surround visual field and the at least one control signal.

18. The surround visual field controller of claim 17 wherein the input source is at least one selected from the group comprising:

video data, audio data, game controller, input device, remote control, sensor, and authored cue.

19. The controller of claim 17 wherein the surround visual field generator utilizes physics-based models to animate the effect on the at least one element within the surround visual field.

20. The controller of claim 17 wherein the surround visual field generator utilizes authored data to generate the surround visual field.