SYSTEM FOR COLOCATING A TOUCH SCREEN AND A VIRTUAL OBJECT, AND DEVICE FOR MANIPULATING VIRTUAL OBJECTS IMPLEMENTING SUCH A SYSTEM

• US 20140204079A1
• Filed: 06/15/2012
• Published: 07/24/2014
• Est. Priority Date: 06/17/2011
• Status: Active Grant

• Current Assignee: Immersion Corporation, Inria Institut National de Recherche EN Informatique et EN Automatique
• Original Assignee: Immersion Corporation, Inria Institut National de Recherche EN Informatique et EN Automatique
• Inventors: De La Riviere, Jean-Baptiste, Chartier, Christophe, Hachet, Martin, Bossavit, Benoit, Casiez, Gery
• Granted Patent: US 9,786,090 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G02B 30/35 using reflective optical elements in the optical path between the images and the observer
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: H04N 13/156 Mixing image signals

Abstract

A system (10) for displaying at least one virtual object includes a secondary screen (20) for displaying the virtual object, a primary screen (30), an optical element for overlaying images displayed on the secondary screen (20) with images displayed on the primary screen (30), and a pointing surface combined with the primary screen (30) for detecting the contact of one or more physical pointing elements. A device (90) for manipulating at least one virtual object includes calculation elements for generating images of the virtual object displayed on the system (10) from information output from the system (10) in accordance with the actions of the operator (100).

Claims

1. A system (10) for visualizing at least one virtual object (200) comprising a secondary screen (20) for displaying images of the virtual object (200), characterized in that said secondary screen is integrated into a workstation of said system which furthermore comprises:

a primary screen (30) for displaying images and/or text;

optical means for simultaneous visualization, by an operator (100) of the workstation, of images displayed on the secondary screen (20) and images and/or text displayed on the primary screen (30);

a detection device, associated with the primary screen (30), for detecting contact points of one or more physical elements for pointing on a pointing surface (32) of said detection device;

said detection device being located between the secondary screen (20) and the primary screen (30).

2. The system as claimed in claim 1, in which an image of a virtual object (200) displayed on the secondary screen (20) appears visually, for an operator (100) using the workstation, overlaid and above the primary screen (30) and the detection device.

3. The system as claimed in claim 2, in which an optical path between an observation point (13) and a point on the secondary screen (20) observed according to a given direction has a length less than the length of an optical path between said observation point and a point on the primary screen (30) observed according to the same direction.

4. The system as claimed in claim 2, in which the optical means mainly comprise a transparent, semi-reflective panel (40), disposed between the secondary screen (20) and the primary screen (30) in such a way that an operator (100) of the workstation visualizes images of the virtual object (200) displayed on the secondary screen (20) through reflection on a surface of the transparent panel (40) and visualizes images and/or text displayed on the primary screen (30) by way of transparency through the transparent panel (40).

5. The system as claimed in claim 1, comprising a system for determining (60) a position, in a reference system of the workstation, of an observation point (13) of an operator (100) using the workstation.

6. The system as claimed in claim 1, comprising one or more systems for observing (50) stereoscopic images when such stereoscopic images are displayed on the secondary screen (20) and/or on the primary screen (30).

7. A device (90) for the manipulation of at least one virtual object by an operator (100), comprising calculation means (91) and a system (10) as claimed in claim 1, in which device the calculation means (91) generate the image of the virtual object (200) displayed on the secondary screen (20) according to contact information received from the detection device associated with the primary screen (30).

8. The device (90) for the manipulation of at least one virtual object as claimed in claim 7, in which symbolic controls (210, 210a, 210b) are calculated and displayed on the primary screen (30) to provide the actions which can be carried out on the detection device to work on the representation of a virtual object (200) or of elements (201a, 201b) of a virtual object displayed on the secondary screen (20).

9. The device (90) for the manipulation of at least one virtual object as claimed in claim 8, in which the symbolic controls (210, 210a, 210b) are displayed on the primary screen (30) according to the apparent position for the operator (100) of the image of the virtual object (200) to retain the apparent relative positions of said symbolic controls in relation to the virtual object to which they relate.

10. The device for the manipulation of at least one virtual object as claimed in claim 9, comprising a system for determining (60) a position, in a reference system of the workstation, of an observation point (13) of an operator (100) using the workstation, in which an image or images of the virtual object (200) displayed on the secondary screen (20) and an image or images displayed on the primary screen (30) are moved relatively in relation to one another according to a measured or calculated position of the observation point (13) in order to retain the same relative positions between said images seen by the operator (100).

11. The device for the manipulation of at least one virtual object as claimed in claim 7, in which said device generates stereoscopic images from a virtual object (200) displayed on the secondary screen (20).

12. The device for the manipulation of at least one virtual object as claimed in claim 11, in which the stereoscopic images are generated in such a way that the virtual object (20), observed using a system (50) for observing stereoscopic images, appears to the operator (100) in front of his hands and the images of the primary screen.

13. The device for the manipulation of at least one virtual object as claimed in claim 7, in which said device generates, from a virtual object (200) displayed on the secondary screen (20), images obtained from a three-dimensional representation.

14. The device for the manipulation of at least one virtual object as claimed in claim 7, in which said device displays information extracted from databases on the primary screen (30).