Method and input device for controlling the position of an object to be graphically displayed in virtual reality

• US 6,545,663 B1
• Filed: 04/18/2000
• Issued: 04/08/2003
• Est. Priority Date: 04/19/1999
• Status: Expired due to Fees

• Current Assignee: Deutsches Zentrum Fur Luft
• Original Assignee: Deutsches Zentrum Fã¼R Luft- Und Raumfahrt E.V.
• Inventors: Arbter, Klaus, Hirzinger, Gerd, Landzettel, Klaus
• Primary Examiner(s): CHANG, KENT WU
• Application Number:
  US09/551,124
• Time in Patent Office: 1,085 Days
• Field of Search: 345/156, 345/157, 345/158, 345/784, 345/785, 345/851, 345/863, 345/684
• US Class Current: 345/158
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour

Abstract

In virtual reality, the position of a graphically-displayed object (1) in a three-dimensional view on a display device (3) of a graphics workstation is controlled by way of a human-machine interface, in which the hand (5) of an operator can displace and rotate an input medium in an arbitrary spatial direction. One or more stationary electronic cameras records or record one or more unique marking symbols (9) provided on the underside of a material, easy-to-handle model object (6) serving as an input medium, the underside facing the camera(s). The unique marking symbols (9) are evaluated with a computer according to image-processing methods such that the spatial position of the model object and the recognition thereof are the result of the evaluation.

The result of the evaluation is transmitted to a graphics unit (graphics workstation, PC graphics). The associated object is displayed, in the position of the model object as predetermined by the operator'"'"'s hand, as virtual reality on the display device (3) of the graphics unit. The invention can be used, for example, in the advertising field for displaying advertised products in desired positions.

Claims

1. An input device using a human-machine interface to control the position of an image representing a vehicle type offered by an auto dealer or another advertised product that is graphically displayed in virtual reality on a display device of a graphics unit, said device comprising:

at least one stationary electronic camera;

an input medium that includes at least one material model object representing the vehicle type or other advertised product to be displayed, said material model object being actively movable in all six degrees of freedom by an operator'"'"'s hand, thus being easy for the operator'"'"'s hand to manipulate, and being provided with at least one unique marking symbol on a side that faces said camera, wherein said camera records said marking symbol into the form of camera images; and

a computer connected to evaluate the camera images in accordance with image processing methods such that information representing the spatial position of said material model object is transmitted by said computer to the graphics unit, whereby the image of the associated vehicle type or product is correspondingly displayed in a position corresponding to the position of said material model object relative to said camera as virtual reality on said display device of said graphics unit.

2. The input device according to claim 1, wherein said marking symbol provided on said material model object also contains an identification of the respective advertised vehicle type or product to be displayed as virtual reality, said identification likewise being recorded by said camera and evaluated according to image-processing methods, so that material model objects provided with different identifications can be distinguished, and their individual positions can be indicated, and the result of this evaluation is transmitted to the graphics unit where the image of the associated advertised vehicle type or product that has been recognized on the basis of its identification is displayed as virtual reality in the position corresponding to the position of the respective material model object relative to said camera, as predetermined by the operator'"'"'s hand, on said display device.

3. The input device according to claim 2, wherein said material model object has an underside to which is applied a respective one of a plurality of marking symbols, each with a different orientation, so that a single material model object can be associated with different advertised vehicle types or products in virtual reality corresponding to the identification of the respective advertised vehicle type or product provided on said marking symbols.

4. The input device according to claim 2, wherein said computer is used to perform the camera image evaluation related to the position of said marking symbol and thus of said material model object, as well as of the identification provided on said material model object.

5. The input device according to claim 4, wherein said computer includes a device for image processing and a computer graphics device, said image processing device includes elements for recording images, segmenting elements for separating the image of said marking symbol from all other image parts, elements for object identification through evaluation of a color, shape or code associated with said marking symbol, and elements for calculating back to the spatial material model object position from the camera image or the camera images of said marking symbols, and said computer graphics device includes elements for storing virtual 3-D objects, and elements for calculating the views of advertised vehicle types or products to be displayed as virtual reality, as a function of a respective observer'"'"'s position.

6. The input device according to claim 1, wherein said material model object has an underside which faces said camera and on which said marking symbol is provided.

7. The input device according to claim 6, wherein said marking symbol provided on the underside of said material model object is flat.

8. The input device according to claim 7, wherein said marking symbol provided on the underside of said material model object is at least one letter, number and/or abstract symbol.

9. The input device according to claim 8, wherein said marking symbol is colored, or is reflective, or is a barcode.

10. The input device according to claim 1, further comprising a plate beneath which said camera is located and above which the operator'"'"'s hand holds said material model object with an underside of said material model object facing said camera, wherein said plate has one or more of the following properties:

transparency or translucence, clearness or matte finish, coloring or absence of color, coating or no coating.

11. The input device according to claim 10, wherein said at least one camera comprises two cameras having respective optical axes that are inclined toward one another, are oriented at a diagonal with respect to said plate, and lie in two vertical planes extending perpendicular to one another.

12. The input device according to claim 10, further comprising a housing of an operating panel in which said electronic camera is disposed, said housing also accommodating a lighting service, and said housing having a top side where said plate is located.

13. The input device according to claim 12, wherein said plate supports said material model object in a detection region of said camera for comfortable guidance of said material model object if only three degrees of freedom of the movement on said plate are to be controlled, which degrees of freedom include translational displacement on said plate and rotation about a vertical axis extending perpendicular to said plate.

14. The input device according to claim 12, wherein said lighting service emits multispectral (white), infrared or ultraviolet light.

15. The input device according to claim 14, wherein said plate is designed for sharpening contrast by filtering with colored lighting.

16. The input device according to claim 15, wherein said plate is red-transparent, or is designed as an infrared filter, and the lighting is infrared lighting.

17. The input device according to claim 1, wherein said computer is used to perform the camera image evaluation related to the position of said marking symbol and thus of said material model object.

18. The input device according to claim 17, wherein said computer includes a device for image processing and a computer graphics device, said image processing device includes elements for recording images, segmenting elements for separating the image of said marking symbol from all other image parts, elements for object identification through evaluation of a color, shape or code associated with said marking symbol, and elements for calculating back to the spatial material model object position from the camera image or the camera images of said marking symbols, and said computer graphics device includes elements for storing virtual 3-D objects, and elements for calculating the views of advertised vehicle types or products to be displayed as virtual reality, as a function of a respective observer'"'"'s position.

19. The input device according to claim 1, wherein said electronic camera has an interface that is suitable for electronic image transmission to a digital computer.