FLOATING IMAGE INTERACTION DEVICE AND ITS PROGRAM

• US 20100110384A1
• Filed: 03/31/2008
• Published: 05/06/2010
• Est. Priority Date: 03/30/2007
• Status: Active Grant

• Current Assignee: National Institute of Information and Communications Technology
• Original Assignee: National Institute of Information and Communications Technology
• Inventors: Maekawa, Satoshi
• Granted Patent: US 8,985,774 B2
• Time in Patent Office: Days
• US Class Current: 353/10
• CPC Class Codes: G02B 30/56 by projecting aerial or floating images
• CPC Class Codes: G02B 5/124 plural reflecting elements forming part of a unitary plate or sheet
• CPC Class Codes: G02B 5/136 plural reflecting elements forming part of a unitary body G02B5/124 takes precedence
• CPC Class Codes: G03B 15/00 Special procedures for taking photographs; Apparatus therefor
• CPC Class Codes: G03B 35/18 by simultaneous viewing
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042

Abstract

A floating image interaction device includes an imaging optical system projecting a floating real image of an object and an user object detecting means for obtaining information about the user accessing the floating real image; means to realize interactive response by the real image or by haptic feedback to actions by the user; and related computer programs.

Claims

1. A floating image interaction device, comprising:

an imaging optical system including a refracting surface whereas paths of light rays are bent at a plane; and

a user object detecting means,

whereby for an object to be projected placed to one side of said refracting surface, said imaging optical system forms a real image of said object on the opposite side of said refracting surface with front and back being reversed,

and said user object detecting means capable of detecting the position, or the position and shape of user objects approaching said real image.

2. A floating image interaction device according to claim 1, further comprising:

a real image modifying means wherein said position, or position and shape of said objects as detected by said user object detecting means is used to modify said real image.

3. A floating image interaction device according to claim 1, further comprising an object actuator means, wherein said position, or position and shape of said user objects as detected by said user object detecting means is used to exert force on said user object.

4. A floating image interaction device according to claim 3, wherein said object actuator means comprises a mechanical link mechanism.

5. A floating image interaction device according to claim 3, wherein said object actuator means comprises an air flow generating device.

6. A floating image interaction device according to claim 1, wherein said imaging optical system is realized by an imaging optical device comprising of a plurality of unit optical elements, said unit optical elements comprising of at least one specular surface reflecting light and arranged perpendicularly or nearly perpendicularly to an optical device plane corresponding to said refracting surface, whereas for an object to be projected placed to one side of said optical device plane, its real image will be formed on the opposite side of said optical device plane by said plurality of unit optical elements.

7. A floating image interaction device according to claim 6, wherein said imaging optical device comprises of a plurality of unit optical elements realized by dihedral corner reflectors consisting of pairs of mutually perpendicular specular surfaces, thereby functioning as a dihedral corner reflector array.

8. A floating image interaction device according to claim 1, wherein said imaging optical system comprises of a half-mirror having a half-mirror surface corresponding to said refracting surface, and a retroreflector array comprising of a set of unit retroreflectors, whereas said retroreflectors are located so as to recursively reflect light reflected by, or transmitted through, said half-mirror.

9. A floating image interaction device according to claim 1, wherein said user object detecting means is located in a fixed position with respect to said refracting surface.

10. A floating image interaction device according to claim 1, wherein said user object detecting means comprises of at least one camera, and an image processing device capable of determining the position, or the position and shape of said object from images recorded by said camera.

11. A floating image interaction device according to claim 10, wherein said camera is located on the same side of said refracting surface as said object to be projected.

12. A floating image interaction device according to claim 11, wherein said camera is located in such a position so that it can observe said object to be projected directly through said refracting surface.

13. A floating image interaction device according to claim 11, wherein a half-mirror is placed between said refracting surface and said object to be projected, and furthermore said camera is located in such a position so that it can observe the real image of said object to be projected as formed by said imaging optical system, and reflected by said half-mirror.

14. A floating image interaction device according to claim 6, wherein said user object detecting means comprises of at least one camera, and an image processing device capable of determining the position, or the position and shape of said user object from images recorded by said camera, and said camera is located in such a position so that it can observe light emitted from said user object and reflected only once by said unit optical elements.

15. A floating image interaction device according to claim 10, wherein said camera is located in such a position so that it can observe directly said user object without the light passing through said refracting surface.

16. A floating image interaction device according to claim 10, wherein said camera is located on the same side of said refracting surface as said real image.

17. A floating image interaction device according to claim 10, wherein at least one of said cameras is a distance detecting camera.

18. A computer program stored on a computer readable medium to be used by the floating image interaction device according to claim 1, comprising:

a user object detection procedure; and

a user object information generating procedure,

wherein said user object detection procedure detects an information of the position, or position and shape of said user object by using said user object detecting means, and determines the position or the position and shape of said user object based on said detected information,

and wherein said user object information generating procedure receives said information from said user object detection procedure about the position, or the position and shape of said user object, and generates information for an appropriate interaction program in order to control said object to be projected according to the position or the position and shape of said user object.