Position Tracking Device, Position Tracking Method, Position Tracking Program and Mixed Reality Providing System

• US 20080267450A1
• Filed: 05/25/2006
• Published: 10/30/2008
• Est. Priority Date: 06/14/2005
• Status: Abandoned Application

• Current Assignee: The University of Electro-Communications
• Original Assignee: The University of Electro-Communications
• Inventors: Nakamura, Akihiro, Inami, Masahiko, Nii, Hideaki, Sugimoto, Maki
• Application Number:
  US11/922,256
• Publication Number:
  US 20080267450A1
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: A63H 17/395 steered by program
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 2207/30248 Vehicle exterior or interior
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

The present invention has a simpler structure than before and is designed to precisely detect the position of a real environment'"'"'s target object on a screen. The present invention generates a special marker image MKZ including a plurality of areas whose brightness levels gradually change in X and Y directions, displays the special marker image MKZ on the screen of a liquid crystal display 2 such that the special marker image MKZ faces an automobile-shaped robot 3, detects, by using sensors SR1 to SR4 provided on the automobile-shaped robot 3 for detecting the change of brightness level of position tracking areas PD1A, PD2A, PD3 and PD4 of the special marker image MKZ in the X and Y directions, the change of brightness level, and then detects the position of the automobile-shaped robot 3 on the screen of the liquid crystal display 2 by calculating, based on the change of brightness level, the change of relative coordinate value between the special marker image MKZ and the automobile shaped robot 3.

Claims

1-14. -14. (canceled)

15. A mixed reality representation device comprising:

a computer graphics image generation means for generating a virtual environment'"'"'s computer graphics image to be displayed on a display means;

a state recognition means for recognizing the state of a real environment'"'"'s target object that is placed such that the virtual environment'"'"'s computer graphics image displayed on the display means and the real environment'"'"'s target object overlap with one another; and

an interlocking means for displaying the virtual environment'"'"'s computer graphics image in accordance with the state of the real environment'"'"'s target object by changing, in accordance with the state of the real environment'"'"'s target object recognized by the state recognition means, the virtual environment'"'"'s computer graphics image.

16. The mixed reality representation device according to claim 15, wherein

the state recognition means including an image pickup means takes, by using the image pickup means, an image of position or motion of the real environment'"'"'s target object and recognizes, based on a result of taking the image, the state of the real environment'"'"'s target object.

17. The mixed reality representation device according to claim 15, wherein

the state recognition means includes:

an index image generation means for generating an index image including a plurality of areas whose brightness levels gradually change in first and second directions and displaying the index image on the display means such that the index image faces the real environment'"'"'s target object;

a brightness level detection means provided on the real environment'"'"'s target object for detecting the change of brightness level of the areas of the index image in the first and second directions; and

a position detection means for recognizing the state of the real environment'"'"'s target object by detecting the position of the real environment'"'"'s target object on the display means after calculating, based on the result of detection by the brightness level detection means, the change of relative coordinate value between the index image and the real environment'"'"'s target object.

18. The mixed reality representation device according to claim 17, wherein

the position detection means detects, based on the brightness levels of the areas of the index image detected by the brightness level detection means in accordance with the real environment'"'"'s target object movement on the display means, the position.

19. The mixed reality representation device according to claim 17, wherein

there is a brightness-level reference area on the index image, and the position tracking means detects, based on the brightness levels of the areas and reference area of the index image detected by the brightness level detection means in accordance with the real environment'"'"'s target object movement on the display means, the position on the display means when the mobile object rotates.

20. The mixed reality representation device according to claim 17, wherein

the index image generation means generates the index image including the plurality of areas whose brightness levels gradually change in the first direction and the second direction perpendicular to the first direction and displays the index image on the display means such that the index image faces the real environment'"'"'s target object.

21. The mixed reality representation device according to claim 17, wherein

the position detection means detects, based on the change of the number of the added brightness levels of the areas of the index image detected by the brightness level detection means in accordance with the real environment'"'"'s target object movement on the display means, the height of the real environment'"'"'s target object on the display means.

22. The mixed reality representation device according to claim 17, wherein

the index image generation means changes the brightness level linearly and gradually.

23. The mixed reality representation device according to claim 17, wherein

the index image generation means changes the brightness level nonlinearly and gradually.

24. The mixed reality representation device according to claim 15, comprising

a manipulation means for manipulating the real environment'"'"'s target object, wherein

the state recognition means recognizes, in accordance with the manipulation of the manipulation means, the state of the real environment'"'"'s target object.

25. The mixed reality representation device according to claim 15, wherein

the interlocking means generates, when the virtual environment'"'"'s computer graphics image is being changed in accordance with the state recognized by the state recognition means, a predetermined virtual object model that is an image to be added in accordance with the state recognized by the state recognition means, and adds the virtual object model such that the virtual object model moves with the real environment'"'"'s target object.

26. The mixed reality representation device according to claim 15, wherein

the computer graphics image generation means including a half mirror projects, by using the half mirror, the virtual environment'"'"'s computer graphics image such that the real environment'"'"'s target object placed at a predetermined position and the virtual environment'"'"'s computer graphics image overlap with one another.

27. A mixed reality representation method for displaying a virtual environment'"'"'s computer graphics image such that a real environment'"'"'s target object and the virtual environment'"'"'s computer graphics image overlap with one another, comprising:

a computer graphics image generation step of generating the virtual environment'"'"'s computer graphics image to be displayed on a display means;

a display step of displaying on the display means the virtual environment'"'"'s computer graphics image generated by the computer graphics image generation step;

a state recognition step of recognizing the state of a real environment'"'"'s target object that is placed such that the virtual environment'"'"'s computer graphics image displayed on the display means and the real environment'"'"'s target object overlap with one another; and

an image interlocking step of displaying the virtual environment'"'"'s computer graphics image in accordance with the state of the real environment'"'"'s target object by changing, in accordance with the state of the real environment'"'"'s target object recognized by the state recognition step, the virtual environment'"'"'s computer graphics image.

28. The mixed reality representation method according to claim 27, wherein

the state recognition step includes:

an index image generation step of generating an index image including a plurality of areas whose brightness levels gradually change in first and second directions and displaying the index image on the display means such that the index image faces the real environment'"'"'s target object;

a brightness level detection step of detecting, by using a brightness level detection means provided on the real environment'"'"'s target object, the change of brightness level of the areas of the index image in the first and second directions; and

a position detection step of recognizing the state of the real environment'"'"'s target object by detecting the position of the real environment'"'"'s target object on the display means after a position detection means calculates, based on the result of detection by the brightness level detection step, the change of relative coordinate value between the index image and the real environment'"'"'s target object.

29. A mixed reality representation device comprising:

a computer graphics image generation means for generating a virtual environment'"'"'s computer graphics image to be displayed on a display means;

a computer graphics image change detection means for detecting the change of the virtual environment'"'"'s computer graphics image displayed on the display means; and

an interlocking means for getting the real environment'"'"'s target object to move with the virtual environment'"'"'s computer graphics image by supplying an operation control signal to control the motion of the real environment'"'"'s target object that is placed such that the virtual environment'"'"'s computer graphics image and the real environment'"'"'s target object overlap with one another, in accordance with the result of detection by the computer graphics image change detection means.

30. The mixed reality representation device according to claim 29, wherein

the computer graphics image change detection means including an image pickup means takes, by using the image pickup means, the computer graphics image, and supplies, based on a result of taking the image, the operation control signal to the real environment'"'"'s target object.

31. The mixed reality representation device according to claim 29, wherein

the computer graphics image change detection means includes:

an index image generation means for generating an index image including a plurality of areas whose brightness levels gradually change in first and second directions and displaying the index image on the display means such that the index image faces the real environment'"'"'s target object;

a brightness level detection means provided on the real environment'"'"'s target object for detecting the change of brightness level of the areas of the index image in the first and second directions; and

a image change detection means for detecting the change of the virtual environment'"'"'s computer graphics image by detecting the position of the real environment'"'"'s target object on the display means after calculating, based on the result of detection by the brightness level detection means, the change of relative coordinate value between the index image and the real environment'"'"'s target object.

32. The mixed reality representation device according to claim 29, wherein:

the computer graphics image change detection means includes a display control means to control the computer graphics image that the computer graphics image generation means generates; and

the operation control signal is supplied to the real environment'"'"'s target object in accordance with a signal output from the display control means.

33. A mixed reality representation method for displaying a virtual environment'"'"'s computer graphics image such that a real environment'"'"'s target object and the virtual environment'"'"'s computer graphics image overlap with one another, comprising:

a computer graphics image generation step of generating the virtual environment'"'"'s computer graphics image to be displayed on a display means;

a display step of displaying on the display means the virtual environment'"'"'s computer graphics image generated by the computer graphics image generation step;

a computer graphics image change detection step of detecting the change of the virtual environment'"'"'s computer graphics image displayed on the display means; and

an image interlocking step of getting the real environment'"'"'s target object to move with the virtual environment'"'"'s computer graphics image by supplying an operation control signal to control the motion of the real environment'"'"'s target object that is placed such that the virtual environment'"'"'s computer graphics image and the real environment'"'"'s target object overlap with one another, in accordance with the result of detection by the computer graphics image change detection step.

34. The mixed reality representation method according to claim 33, wherein

the computer graphics image change detection step includes:

an index image generation step of generating an index image including a plurality of areas whose brightness levels gradually change in first and second directions and displaying the index image on the display means such that the index image faces the real environment'"'"'s target object;

a brightness level detection step of detecting, by using a brightness level detection means provided on the real environment'"'"'s target object, the change of brightness level of the areas of the index image in the first and second directions; and

a image change detection step of detecting the change of the virtual environment'"'"'s computer graphics image by detecting the position of the real environment'"'"'s target object on the display means after a position detection means calculates, based on the result of detection by the brightness level detection step, the change of relative coordinate value between the index image and the real environment'"'"'s target object.