REAL-TIME CHANGES TO A SPHERICAL FIELD OF VIEW

• US 20170244884A1
• Filed: 02/23/2016
• Published: 08/24/2017
• Est. Priority Date: 02/23/2016
• Status: Active Grant

• Current Assignee: RPX Corporation
• Original Assignee: Videostitch, Inc.
• Inventors: Burtey, Nicolas, Valente, Stphane, Fink, Alexander
• Granted Patent: US 9,992,400 B2
• Time in Patent Office: Days
• CPC Class Codes: G02B 2027/011 comprising device for correcting geometrical aberrations, distortion
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0101 characterised by optical features G02B27/0172 takes precedence
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: H04N 23/66 Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/73 by influencing the exposure time
• CPC Class Codes: H04N 5/265 Mixing

Abstract

A system includes a video source and a computing device. The video source may be configured to generate a plurality of video streams that capture a view of an environment. The computing device generally includes one or more processors configured to (i) perform a stitching operation on the plurality of video streams to generate a video signal representative of a spherical field of view of the environment, (ii) transmit a display signal that represents a projection of the video signal to be displayed to a user utilizing an immersive display, (iii) receive a plurality of commands from the user while the user observes the environment in the immersive display and (iv) adjust a plurality of parameters in one or more of the video source and the stitching operation in real time in response to the commands.

Claims

1. A system comprising:

a video source configured to generate a plurality of video streams that capture a view of an environment; and

a computing device comprising one or more processors configured to (i) perform a stitching operation on said plurality of video streams to generate a video signal representative of a spherical field of view of said environment, (ii) transmit a display signal that represents a projection of said video signal to be displayed to a user utilizing an immersive display, (iii) receive a plurality of commands from said user while said user observes said environment in said immersive display and (iv) adjust a plurality of parameters in one or more of said video source and said stitching operation in real time in response to said commands.

2. The system according to claim 1, wherein said video source comprises a camera (i) surrounded by said environment, (ii) configured to generate said plurality of video streams and (iii) configured to adjust one or more of said plurality of video streams in response to said parameters.

3. The system according to claim 1, wherein said video source comprises (i) one or more storage media configured to present said plurality of video streams and (ii) a processing circuit configured to adjust one or more of said plurality of video streams in response to said parameters.

4. The system according to claim 1, wherein said computing device is further configured to (i) receive a position signal that identifies where said user is currently observing in said spherical field of view and (ii) generate said display signal in response to said position signal.

5. The system according to claim 4, wherein said immersive display is configured to (i) generate said position signal and (ii) display said display signal to said user.

6. The system according to claim 1, wherein (i) said computing device is further configured to identify one or more of said plurality of video streams that contribute to said display signal when said commands related to said video source are received from said user and (ii) transfer said parameters to said video source to adjust said one or more video streams that are identified.

7. The system according to claim 1, wherein said computing device is further configured to (i) identify one or more of a plurality of transformations in said stitching operation that contribute to said display signal when said commands related to said stitching operation are received from said user and (ii) change said one or more transformations in response to said commands.

8. The system according to claim 1, wherein said computing device further comprises a sensor interface circuit configured to (i) receive said commands from said user and (ii) generate one or more electrical signals representative of said commands.

9. The system according to claim 8, wherein said computing device further comprises a stitch circuit configured to (i) perform said stitching operation to generate said video signal and (ii) instruct said one or more of said video source and said stitching operation to adjust said parameters in response to said electrical signals received from said sensor interface circuit.

10. The system according to claim 1, further comprising a display interface circuit configured to (i) generate said display signal from said video signal based on a position signal that identifies where said user is currently observing in said spherical field of view and (ii) generate one or more electrical signals representative of said position.

11. The system according to claim 10, wherein said computing device further comprises a stitch circuit configured to (i) perform said stitching operation to generate said video signal and (ii) identify one or more of said plurality of video streams that contribute to said display signal in response to said electrical signals received from said display interface circuit.

12. A method for immersive changes to a spherical field of view, comprising the steps of:

(A) generating a plurality of video streams that capture a view of an environment using a video source;

(B) performing a stitching operation on said plurality of video streams to generate a video signal representative of said spherical field of view of said environment;

(C) transmitting a display signal that represents a projection of said video signal to be displayed to a user utilizing an immersive display;

(D) receiving a plurality of commands from said user while said user observes said environment in said immersive display; and

(E) adjusting a plurality of parameters in one or more of said video source and said stitching operation in real time in response to said commands.

13. A system comprising:

an immersive display configured to display to a user a projection of a spherical field of view representative of an environment; and

a plurality of sensors configured to receive a plurality of commands from said user while said user observes said environment using said immersive display, wherein (i) a plurality of parameters in one or more of a video source and a stitching operation are adjusted in response to said commands, (ii) said user sees changes in real time to said environment through said immersive display in response to said commands and (iii) said stitching operation generates a video signal representative of said environment from a plurality of video streams generated by said video source.

14. The system according to claim 13, wherein (i) said immersive display is further configured to generate a position signal that identifies said projection of said spherical field of view currently observed by said user and (ii) said position signal conveys one or more of (a) a head orientation of said user relative to said spherical field of view and (b) an eye position said user relative to said spherical field of view.

15. The system according to claim 13, wherein said immersive display is further configured to play audio to said user as presented by said video source.

16. The system according to claim 13, wherein said sensors comprise one or more of (i) an eye tracking sensor, (ii) a hand gesture sensor, (iii) a tracking pad sensor, (iv) a rotating platform angle sensor, (v) a game pad sensor, (vi) a computer mouse and (vii) a speech recognition circuit.

17. The system according to claim 13, wherein said sensors control one or more of (i) an authoring tool, (ii) an insertion of one or more hot spot areas and (iii) an insertion of one or more action-triggering areas.

18. The system according to claim 13, wherein said parameters relate to one or more of (i) calibration, (ii) stitching distance, (iii) parallax, (iv) lens distortion, (v) spatial orientation of said video source, (vi) light sensitivity, (vii) exposure, (viii) white balance, (ix) contrast curve, (x) black level, (xi) white level, (xii) tone mapping, (xiii) focal length, (xiv) baseline distance between views, (xv) network settings, (xvi) filters, (xvii) global audio volume, (xviii) audio volume of a sound object and (ixx) a location of said sound object.

19. The system according to claim 13, further comprising a computing device comprising one or more processors configured to (i) perform said stitching operation on said plurality of video streams to generate said video signal representative of said spherical field of view of said environment, (ii) transmit a display signal that represents a projection of said video signal to be displayed to said user utilizing said immersive display, (iii) receive said plurality of commands from said user via said sensors while said user observes said environment in said immersive display and (iv) adjust said plurality of parameters in one or more of said video source and said stitching operation in real time in response to said commands.

20. The system according to claim 19, wherein said computing device comprises (i) a stitch circuit configured to generate said video signal, (ii) a display interface circuit configured to generate said display signal from said video signal and (iii) a sensor interface circuit configured to receive said commands from said user.