Vision-based augmented reality system using invisible marker

• US 8,854,470 B2
• Filed: 02/21/2014
• Issued: 10/07/2014
• Est. Priority Date: 07/30/2004
• Status: Expired due to Fees

• Current Assignee: Industry-University Cooperation Foundation Hanyang University
• Original Assignee: Industry-University Cooperation Foundation Hanyang University
• Inventors: Park, Jong-Il, Park, Han-Hoon
• Primary Examiner(s): Moe, Aung S
• Assistant Examiner(s): COWAN, EUEL K
• Application Number:
  US14/187,110
• Publication Number:
  US 20140232749A1
• Time in Patent Office: 228 Days
• Field of Search: None
• US Class Current: 348/162
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: H04N 23/20 for generating image signals from infrared radiation only
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

A vision-based augmented reality system using an invisible marker indicates an invisible marker on a target object to be tracked, such that it can rapidly and correctly track the target object by detecting the invisible marker. The augmented reality system includes a target object (TO) including an infrared marker (IM) drawn by an invisible infrared light-emitting material; a visible-ray camera (110) for capturing an image of the TO; an infrared-ray camera (120) for capturing an image of the IM included in the TO image; an optical axis converter for allowing the infrared-ray camera (120) and the visible-ray camera (110) to have the same viewing point; an image processing system (140) for rendering a prepared virtual image to the TO image to generate a new image.

Claims

1. A vision-based augmented reality system using an invisible marker, comprising:

an infrared-ray camera for capturing a target object (TO) having an infrared marker (IM) drawn by an invisible infrared light-emitting material; and

an image processing system for monitoring position and pose of the IM from an image of the infrared-ray camera, tracking position and pose of the TO based on the position and the pose of the IM, rendering a virtual image on the TO.

2. The system of claim 1, further comprising an output unit for displaying the rendered virtual image.

3. The system of claim 2, wherein the output unit includes at least one of a head mount display, a 3-dimensional glass and a glass-type HMD.

4. The system of claim 1, further comprising a visible-ray camera for capturing the TO.

5. The system of claim 4, wherein an image of the visible-ray camera and an image of the infrared-ray camera are obtained separately.

6. A display device, comprising:

an infrared-ray camera for capturing a target object (TO) having an infrared marker (IM) drawn by an invisible infrared light-emitting material;

an image processing system for monitoring position and pose of the IM from an image of the infrared-ray camera, tracking position and pose of the TO based on the position and the pose of the IM, rendering a virtual image on the TO and an output unit for displaying the rendered virtual image.

7. The device of claim 6, further comprising a visible-ray camera for capturing the TO.

8. The device of claim 7, wherein an image of the visible-ray camera and an image of the infrared-ray camera are obtained separately.

9. A method for vision-based augmented reality using an invisible marker, comprising the steps of:

(a) capturing target object (TO) including an infrared marker (IM) drawn by an invisible infrared light-emitting material using a visible-ray camera;

(b) capturing the IM included in the TO using an infrared-ray camera;

(c) obtaining an image of the visible-ray camera and an image of the infrared-ray camera separately, monitoring position and pose of the IM associated with the infrared-ray camera, tracking position and pose of the TO based on position and pose of the IM, and rendering a virtual image to the TO image.

10. The method of claim 9, further comprising optical axis converting step of changing viewing point of at least one of the visible-ray camera and the infrared-ray camera.

11. The method of claim 10, wherein the optical axis converting step comprises the step of changing the viewing point of at least one of the visible-ray camera and the infrared-ray camera so that the infrared-ray camera and the visible-ray camera have the same viewing point.

12. The method of claim 11, wherein a cold mirror which reflects infrared ray generated from the TO on the infrared-ray camera and transmits infrared-ray generated from the TO to the visible-ray camera is used in the optical axis converting step.

13. A method for vision-based augmented reality using an invisible marker, comprising the steps of:

(a) capturing a target object (TO) having an infrared marker (IM) drawn by an invisible infrared light-emitting material using an infrared-ray camera;

(b) monitoring position and pose of the IM from an image of the infrared-ray camera, tracking position and pose of the TO based on the position and the pose of the IM, rendering a virtual image based on the TO.

14. The method of claim 13, further comprising the step of displaying the rendered virtual image.

15. The method of claim 14, wherein the rendered virtual image is displayed using at least one of a head mount display, a 3-dimensional glass and a glass-type HMD.

16. The method of claim 13, further comprising the capturing the TO using a visible-ray camera.

17. The method of claim 16, wherein an image of the visible-ray camera and an image of the infrared-ray camera are obtained separately.

18. The method of claim 17, wherein the virtual image is rendered on the image of the visible-ray camera.

19. The method of claim 17, further comprising the optical axis converting step of changing viewing point of at least one of the visible-ray camera and the infrared-ray camera.

20. The method of claim 19, wherein the optical axis converting step comprises the step of changing the viewing point of at least one of the visible-ray camera and the infrared-ray camera so that the infrared-ray camera and the visible-ray camera have the same viewing point.