DEVICE FOR WATCHING REAL-TIME AUGMENTED REALITY AND METHOD FOR IMPLEMENTING SAID DEVICE

• US 20100045700A1
• Filed: 01/10/2008
• Published: 02/25/2010
• Est. Priority Date: 01/12/2007
• Status: Abandoned Application

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Total Immersion
• Inventors: Savary, Alan, Lefevre, Valentin, Livet, Nicolas
• Application Number:
  US12/522,948
• Publication Number:
  US 20100045700A1
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G02B 23/00 Telescopes, e.g. binoculars; Periscopes; Instruments for viewing the inside of hollow bodies; Viewfinders; Optical aiming or sighting devices
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: H04N 21/44008 involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 5/445 for displaying additional information H04N5/50 takes precedence

Abstract

The invention relates to a real-time augmented-reality watching device (300), which comprises an image sensor (335) such as a PTZ camera, a visualisation system (320) and a control interface (310). In this device, the camera is controlled by a control interface operated by the user. Once the user has received (610) orientation information on the desired line of sight from the control interface, the orientation information on the line of sight is transmitted to the camera (615), the camera being powered and capable of movement. Then the camera transmits the orientation of its line of sight (620). The camera transmits a video stream in parallel. The position at which must be inserted data, such as the representation of a virtual three-dimensional object, is determined from the data resulting from the calibration and the orientation of the line of sight of the camera received from the latter (640). The data is further inserted in real time into the video stream that is transmitted to the visualisation system (645). A calibration of the camera is preferably carried out during the device set up (600) in order to improve the device performance.

Claims

1. A method for a real time augmented reality observation device (300) comprising an image sensor (335), a viewing system (320) and an indirect command interface (310) with direct line of sight, said method comprising the following steps:

receiving a request including line of sight orientation information transmitted by said command interface (610);

transmitting said line of sight orientation information to said image sensor (615), said image sensor being mobile and motorized;

receiving from said image sensor the orientation of its line of sight (620);

receiving at least one image from said image sensor;

determining in said received image the position at which at least one item of data must be inserted, according to the orientation of the line of sight of said image sensor (640); and

inserting said at least one data item in real time into said received image at the position so determined (645).

2. Method according to claim 1, wherein said viewing system is mobile and mechanically connected to at least one element of said command interface.

3. Method according to claim 1, further comprising a phase of calibrating said image sensor (600).

4. Method according to claim 3, wherein said calibration step comprises the calibration of at least one of the parameters included in the set of parameters comprising correcting radial distortion of said image sensor, correcting roll of said image sensor, correcting the pan and tilt of the line of sight of said image sensor and the offset between the optical center and the rotation center of said image sensor.

5. Method according to claim 4, wherein said image sensor comprises a zoom function and the calibration of said at least one parameter is effected for a plurality of zoom factors.

6. Method according to claim 1, further comprising a step of colocation of said image sensor and the scene observed by said image sensor to determine the pose of said at least one data item to be inserted into said image received in said observed scene.

7. Method according to claim 1, wherein at least one data item to be inserted in said received image is dependent on the geographical position of said observation device.

8. Method according to claim 1, wherein that said at least one item of data to be inserted in said received image is a representation of a three-dimensional virtual model animated or not.

9. Method according to claim 1, wherein said orientation of the line of sight is defined with respect to two degrees of freedom and said image sensor comprises a zoom function.

10. Computer program stored on a computer readable storage medium, and including instructions adapted to execute each of the steps of the method according to claim 1.

14. Device according to claim 1, wherein said image sensor and/or said storage means is remote from said observation device.

15. Method according to claim 2, further comprising a phase of calibrating said image sensor (600).

11. An augmented reality observation device (300) comprising:

means for connection to an image sensor (335), a viewing system (320) and an indirect command interface (310) with direct line of sight;

means for receiving line of sight orientation information transmitted by said command interface;

means for controlling the orientation of the line of sight of said image sensor according to said orientation information received, said image sensor being mobile and motorized;

means for receiving the orientation of the line of sight of said image sensor;

means for receiving at least one image from said image sensor;

means for determining in said received image the position at which at least one item of data must be inserted, according to the orientation of the line of sight of said image sensor; and

means for inserting in real time into said received image said at least one data item at said position so determined.

12. Device according to claim 11, wherein said viewing system is mobile and mechanically connected to at least part of said command interface.

13. Device according to claim 11, further comprising means for transmitting said received image comprising said at least one item of data.

16. Device according to claim 12, further comprising means for transmitting said received image comprising said at least one item of data.