METHOD AND APPARATUS FOR EXHIBITING MIXED REALITY BASED ON PRINT MEDIUM

• US 20120320092A1
• Filed: 06/13/2012
• Published: 12/20/2012
• Est. Priority Date: 06/14/2011
• Status: Abandoned Application

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: Shin, Hee Sook, Lee, Dong Woo, Lim, Jeong Mook, Jeong, Hyun Tae, Shin, Sungyong
• Application Number:
  US13/495,560
• Publication Number:
  US 20120320092A1
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386

Abstract

An apparatus for exhibiting mixed reality based on a print medium includes a command identification module and a content reproduction module. The command identification module identifies a hand gesture of a user performed on a printer matter in the print medium to recognize a user input command corresponding to the hand gesture. The content reproduction module provides a digital content corresponding to the printed matter onto a display area on the print medium.

Claims

1. An apparatus for exhibiting mixed reality based on a print medium, comprising:

a command identification module configured to identify a hand gesture of a user performed on a printer matter in the print medium to recognize a user input command corresponding to the hand gesture; and

a content reproduction module configured to provide a digital content corresponding to the printed matter onto a display area on the print medium.

2. The apparatus of claim 1, wherein the command identification module comprises:

a pattern image output unit configured to generate a pattern image on the printed medium, the hand gesture causes a change in the pattern image;

an image acquiring unit configured to capture an image of the surface of the printed medium, wherein the captured image has the pattern image included therein, and wherein the hand gesture causes the change in the pattern image; and

a command recognizing unit configured to detect the pattern image caused by the hand gesture to recognize the user input command corresponding to the hand gesture.

3. The apparatus of claim 1, wherein the pattern image includes an image in a grid form projected onto the print medium at a preset period.

4. The apparatus of claim 3, wherein the pattern image includes an infrared image invisible to the user.

5. The apparatus of claim 2, wherein the command identification module further comprises a command model database that stores a plurality of command models corresponding to hand gestures representative of user input commands;

and

wherein the command recognition unit is configured to match the hand gesture with the command models to find out a command model corresponding to the hand gesture.

6. The apparatus of claim 1, wherein the command identification module further comprises an environment recognizing unit that is configured to analyze the captured image of the print medium to find the display area appropriate for presenting the digital content.

7. The apparatus of claim 6, wherein the environment recognizing unit is further configured to collect display environment information from the captured image of the print medium, the display environment information including at least one of information related to size, brightness, flat state or distorted state of the display area.

8. The apparatus of claim 7, further comprising a content management module that is configured to format the digital content based on the display environment information of the display area and provide the digital content to the content reproduction module.

9. The apparatus of claim 7, wherein the content reproduction module comprises an image correction unit that is configured to correct the image of the digital content based on the display environment information.

10. A method for exhibiting mixed reality based on a print medium, comprising:

generating a pattern image onto the print medium, wherein a hand gesture of a user is interacted with a printed matter in the printed medium to produce a user input command;

identifying the hand gesture causing a change in the pattern image to recognize the user input command; and

projecting digital content corresponding to the printed matter onto a display area of the print medium depending on the user input command.

11. The method of claim 10, wherein said generating a pattern image onto the print medium comprises projecting an image in a grid form onto the print medium at a preset period.

12. The method of claim 10, wherein said identifying the hand gesture comprises:

capturing an image of the print medium, the captured image including the pattern image;

detecting the change in the pattern image caused by the hand gesture; and

matching the hand gesture with a plurality of command models to find out a command model corresponding to the user input command.

13. The method of claim 12, wherein the pattern image includes an infrared image invisible to the user.

14. The method of claim 10, further comprising:

analyzing the captured image to find a display area appropriate for reproducing the digital content on the print medium.

15. The method of claim 12, further comprising:

collecting display environment information including at least one of information related to size, brightness, flat state and distorted state of the display area.

16. The method of claim 14, further comprising:

formatting the digital content based on the collected display environment information.

17. The method of claim 14, further comprising:

correcting an image of the digital content reproduced on the display area based on the collected display environment information.