System for the remote assistance of an operator during a work stage

• US 7,333,073 B2
• Filed: 12/28/2004
• Issued: 02/19/2008
• Est. Priority Date: 12/30/2003
• Status: Expired due to Fees

• Current Assignee: C.R.F. Societa Consortile Per Azioni, Qualcomm, Inc.
• Original Assignee: Cefla Societa' Cooperativa
• Inventors: Repetto, Piermario, Bernard, Stefano, Liotti, Luca, Carvignese, Cosimo, Finizio, Roberto, Pallaro, Nereo
• Primary Examiner(s): Hjerpe; Richard
• Assistant Examiner(s): Said; Mansour M.
• Application Number:
  US11/022,657
• Publication Number:
  US 20050156816A1
• Time in Patent Office: 1,148 Days
• Field of Search: 345 11- 13, 345 7- 9, 348/53, 349/11, 351/157, 359/11, 359/14, 359630-633
• US Class Current: 345/8
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted

Abstract

Interface system for assisting an operator during a work stage comprising a support structure wearable by the operator, with a transparent screen placed in front of the operator'"'"'s eyes to permit him to see a portion of a background; a virtual image generator for producing an optical signal directed towards the operator'"'"'s retina so as to form a virtual image at a predetermined distance from the operator'"'"'s eyes and superimposed on the background; a recording device integral with the operator'"'"'s head, to record part of the operator'"'"'s visual field and make available a signal representative of the visual field; a processing device for processing the signal from the recording device and generating a visual information signal of use to the operator for carrying out the work stage; and a device of reception/transmission for sending the signal from the recording device to the processing device, and rendering the information signal to the virtual image generator.

Claims

1. Interface system for assisting an operator during a work stage, comprising:

a support structure wearable by the operator, on which is arranged at least one transparent screen element suitable for being placed in front of at least one of the operator'"'"'s eyes to permit the operator to see at least one portion of the background;

virtual image generating means suitable for producing an optical signal directed towards the operator'"'"'s retina in such a way as to form a virtual image at a predetermined distance from the operator'"'"'s eyes and superimposed on said background;

recording means integral with the operator'"'"'s head, suitable for recording at least part of the operator'"'"'s visual field and making available a signal representative of said visual field;

processing means suitable for processing said signal from the recording means and generating a visual information signal of use to the operator for carrying out said work stage; and

means of reception/transmission suitable for sending said signal from the recording means to said processing means and as a consequence rendering said information signal to said virtual image generating means, in such a way as to provide the operator with visual information relating to said work stage superimposed on said visual field of the operator.

2. System according to claim 1, in which said processing means are arranged in a remote control station.

3. System according to claim 2, in which said processing means are controlled by a second operator.

4. System according to claim 3, in which said system comprises means for recording and means for rendering an audio signal, said means being integrated on one side in said support structure, and arranged on the other in said control station in order to put said first and second operators into audio communication.

5. System according to claim 1, in which said processing means are at least integrated in part in said support structure.

6. System according to claim 5, in which said processing means are integrated in part in said support structure and in part in a second unit wearable by the operator.

7. System according to claim 1, in which said visual information signal is generated by the processing means on the basis of the information recorded by recording means integral with the operator'"'"'s head, comparing this information with data present in a database.

8. System according to claim 7 in which said database is integrated at least in part in the support structure.

9. System according to claim 7 in which said database is integrated at least in part in a remote base station.

10. System according to claim 7 in which said database is integrated at least in part in a second wearable unit different from the support structure.

11. System according to claim 1, in which said visual information signal is generated by the processing means on the basis of the information recorded by recording means integral with the operator'"'"'s head and additional measuring instruments operatively connected to said processing means.

12. System according to claim 1, in which at least part of the virtual image associated with said information signal is made available to the operator in a manner integral with the operator'"'"'s head.

13. System according to claim 1, also comprising a measuring unit, preferably inertial, of the position and/or orientation of the operator'"'"'s head, integrated in said support structure, in which at least part of the virtual image associated with said information signal is made available to the operator in a manner integral with the background observed by the operator, the alignment between the virtual image and the background being obtained from measurement of the coordinates of the operator'"'"'s head, read by said measuring unit.

14. System according to claim 1, in which at least part of the virtual image associated with said information signal is made available to the operator in an manner integral with the surrounding environment, the alignment between the virtual image and the background being obtained through the recognition of particulars present in the operator'"'"'s visual field caught by said recording means.

15. System according to claim 14, in which at least part of the virtual image associated with said visual information is coupled with a detail of the visual field caught by the recording means.

16. System according to claim 15, in which said particular of the visual field caught by the recording means is selected by a second operator positioned in a remote control station.

17. System according to claim 15, comprising means for tracking the position of the pupil of the eye, in which said particular of the visual field caught by the recording means is selected by the operator himself by fixing his gaze on said particular, the direction of the gaze being recorded by said tracking means.

18. System according to claim 1, comprising means for tracking the position of the pupil of the eye, in which at least part of the virtual image associated with said information signal is made available to the operator in a manner integral with his gaze, the direction of the gaze being recorded by said tracking means.

19. System according to claim 1, in which the rendering distance of said virtual image is close to the distance between the operator'"'"'s eyes and the object on which the operator is carrying out the work.