Vision-based augmented reality system using invisible marker

• US RE45,031 E1
• Filed: 08/31/2012
• Issued: 07/22/2014
• Est. Priority Date: 07/30/2004
• Status: Active Grant

• Current Assignee: Industry-University Cooperation Foundation Hanyang University
• Original Assignee: Industry-University Cooperation Foundation Hanyang University
• Inventors: Park, Jong-Il, Park, Han-Hoon
• Primary Examiner(s): Ye, Lin
• Assistant Examiner(s): COWAN, EUEL K
• Application Number:
  US13/602,057
• Time in Patent Office: 690 Days
• Field of Search: None
• US Class Current: 348/162
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

A vision-based augmented reality system using an invisible marker indicates an invisible marker on a target object to be tracked, such that it can rapidly and correctly track the target object by detecting the invisible marker. The augmented reality system includes a target object including an infrared marker drawn by an invisible infrared light-emitting material; a visible-ray camera for capturing an image of the TO; an infrared-ray camera for capturing an image of the IM included in the TO image; an optical axis converter for allowing the infrared-ray camera and the visible-ray camera to have the same viewing point; an image processing system for rendering a prepared virtual image to the TO image to generate a new image.

Claims

1. A vision-based augmented reality system using an invisible marker, comprising:

a target object (TO) including an infrared marker (IM) drawn by an invisible infrared light-emitting material;

a visible-ray camera for capturing an image of the TO;

an infrared-ray camera for capturing an image of the IM included in the TO image;

an optical axis converter for transmitting a visible ray received from the TO to the visible-ray camera, transmitting an infrared ray received from the TO to the infrared-ray camera, and allowing the infrared-ray camera and the visible-ray camera to have the same viewing point;

an image processing system for receiving the infrared marker image from the infrared-ray camera, receiving the TO image from the visible-ray camera, separating the infrared marker image and the TO image from each other, real-time monitoring a position and pose of the IM associated with the infrared-ray camera, real-time tracking a position and pose of the TO, and rendering a prepared virtual image to the TO image based on the tracked position and pose of the TO to generate a new image; and

an output unit for displaying the image received from the image processing system on a screen.

2. The system according to claim 1, wherein the visible-ray camera includes a color compensation filter for passing visible-ray light.

3. The system according to claim 1, wherein the infrared-ray camera includes an infrared pass filter for passing infrared-ray light to recognize the IM.

4. The system according to claim 1, wherein the optical axis converter is indicative of a cold mirror, which is arranged between the visible-ray camera and the infrared-ray camera, transmits the infrared ray generated from the TO to the infrared-ray camera, reflects the visible ray generated from the TO on the visible-ray camera, and allows the viewing point of the infrared-ray camera to coincide with that of the visible-ray camera.

5. The system according to claim 1, wherein the optical axis converter is indicative of a prism, which refracts the visible ray received from the TO in the direction of the visible-ray camera, refracts the infrared ray received from the TO in the direction of the infrared-ray camera, and allows the viewing point of the infrared-ray camera to coincide with that of the visible-ray camera.

6. A vision-based augmented reality system using an invisible marker, comprising:

a visible-ray camera for capturing target object (TO) including an infrared marker (IM) drawn by an invisible infrared light-emitting material;

an infrared-ray camera for capturing the IM included in the TO; and

an image processing system for receiving an image of the infrared marker from the infrared-ray camera, receiving an image of the TO from the visible-ray camera, separating the image of the infrared marker and the image of the TO from each other, monitoring position and pose of the IM associated with the infrared-ray camera, tracking position and pose of the TO based on position and pose of the IM, and rendering a virtual image to the TO image.

7. The system of claim 6, further comprising an optical axis converter for changing viewing point of at least one of the visible-ray camera and the infrared-ray camera.

8. The system of claim 7, wherein the optical axis converter changes the viewing point of at least one of the visible-ray camera and the infrared-ray camera so that the infrared-ray camera and the visible-ray camera have the same viewing point.

9. The system of claim 8, wherein the optical axis converter includes a cold mirror, which transmits infrared ray generated from the TO on the infrared-ray camera, and reflects visible-ray generated from the TO to the visible-ray camera.

12. A vision-based augmented reality system using an invisible marker, comprising:

an infrared-ray camera for capturing a target object (TO) having an infrared marker drawn by an invisible infrared light-emitting material;

a visible-ray camera for capturing the TO; and

an image processing system for monitoring position and pose of the IM from an image of the infrared-ray camera, tracking position and pose of the TO based on the position and the pose of the IM, rendering a virtual image based on the tracked position and pose,

wherein the image processing system separates an image of the visible-ray camera and an image of the infrared-ray camera from each other in receiving the image of the visible-ray camera and the image of the infrared-ray camera, and renders the virtual image on an image captured by the visible-ray camera.

10. The system of claim 12, further comprising an output unit for displaying the rendered virtual image.

11. The system of claim 10, wherein the output unit includes at least one of a head mount display, a 3-dimensional glass and a glass-type HMD.

13. The system of claim 12, further comprising an optical axis converter for changing viewing point of at least one of the visible-ray camera and the infrared-ray camera.

14. The system of claim 13, wherein the optical axis converter changes the viewing point of at least one of the visible-ray camera and the infrared-ray camera so that the infrared-ray camera and the visible-ray camera have the same viewing point.

17. A vision-based augmented reality system using an invisible marker, comprising:

a visible-ray camera for capturing a target object (TO) including an infrared marker (IM) drawn by an invisible infrared light-emitting material;

an infrared-ray camera for capturing the IM included in the TO;

an optical axis converter for changing viewing point of at least one of the visible-ray camera and the infrared-ray camera; and

an image processing system for rendering a virtual image using an image captured by the infrared-ray camera, an image captured by the visible-ray camera and IM;

wherein the image processing system receives an image of the infrared marker from the infrared-ray camera, receives an image of the TO from the visible-ray camera, separates the image of the infrared marker and the image of the TO from each other, monitors position and pose of the IM associated with the infrared-ray camera, tracks position and pose of the TO based on position and pose of the IM, and renders the virtual image to the TO image.

15. The system of claim 17, wherein the optical axis converter changes the viewing point of at least one of the visible-ray camera and the infrared-ray camera so that the infrared-ray camera and the visible-ray camera have the same viewing point.

16. The system of claim 17, wherein the optical axis converter includes a cold mirror, which transmits infrared ray generated from the TO on the infrared-ray camera, and reflects visible-ray generated from the TO to the visible-ray camera.

18. A display device, comprising:

an infrared-ray camera for capturing a target object (TO) having an infrared marker drawn by an invisible infrared light-emitting material;

a visible-ray camera for capturing the TO;

an image processing system for monitoring position and pose of the IM from an image of the infrared-ray camera, tracking position and pose of the TO based on the position and the pose of the IM, rendering a virtual image based on the tracked position and pose, and

an output unit for displaying the rendered virtual image;

wherein the image processing system separates an image of the visible-ray camera and an image of the infrared-ray camera from each other in receiving the image of the visible-ray camera and the image of the infrared-ray camera, and renders the virtual image on an image captured by the visible-ray camera.