FEEDBACK BASED REMOTE MAINTENANCE OPERATIONS

• US 20160350595A1
• Filed: 09/13/2015
• Published: 12/01/2016
• Est. Priority Date: 05/31/2015
• Status: Active Grant

• Current Assignee: Fieldbit Ltd. (Help Lightning, Inc.)
• Original Assignee: Fieldbit Ltd. (Help Lightning, Inc.)
• Inventors: Solomin, Shay, Rapoport, Alex, Meiron, Evyatar
• Granted Patent: US 10,339,382 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 18/2178 based on feedback of a supervisor
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 10/7784 based on feedback from supervisors
• CPC Class Codes: G06V 20/20 in augmented reality scenes

Abstract

A method for feedback based maintenance wherein the method may include transmitting to an intermediate computerized system a first image that was acquired, at an acquisition point of time, by a head mounted display (HMD) mounted on a head of a first user; transmitting the first image from the intermediate computerized system and towards a computerized device of a second user; displaying on the computerized device of the second user the first image; receiving feedback, in response to the displaying and by the intermediate computerized system; generating, by at least one of the computerized device of the second user and the intermediate computerized system, an augmented image that comprises the first image and the feedback; transmitting to the HMD the augmented image; adjusting, by the HMD the augmented image to provide an amended augmented image that corresponds to visual information as viewed by the HMD at a display point in time; and displaying, by the HMD, the amended augmented image.

Claims

1. A method for feedback based maintenance, the method comprises:

transmitting to an intermediate computerized system a first image that was acquired, at an acquisition point of time, by a head mounted display (HMD) mounted on a head of a first user;

transmitting the first image from the intermediate computerized system and towards a computerized device of a second user;

displaying on the computerized device of the second user the first image;

receiving feedback, in response to the displaying, by the computerized system;

generating, by at least one of the computerized device of the second user and the intermediate computerized system, an augmented image that comprises the first image and the feedback;

transmitting to the HMD the augmented image;

adjusting, by the HMD the augmented image to provide an amended augmented image that corresponds to visual information as viewed by the HMD at a display point in time; and

displaying, by the HMD, the amended augmented image.

2. The method according to claim 1 comprising receiving by the intermediate computerized system a feedback request from the first user to provide feedback related to first image and transmitting the feedback request to the computerized device of the second user.

3. The method according to claim 1 comprising receiving, by the intermediate computerized system, the feedback at an absence of a feedback request from the first user to provide feedback related to first image.

4. The method according to claim 1 wherein the receiving of the feedback comprises receiving the feedback from the second user.

5. The method according to claim 1 wherein the receiving of the feedback is preceded by relaying the first image from the computerized device of the second user to a computerized device of a third user;

and receiving the feedback from the third user.

6. The method according to claim 1 wherein the transmitting of the first image comprises transmitting to the intermediate computerized system a video stream that comprises the first image.

7. The method according to claim 1 wherein the intermediate computerized system is a cloud environment server.

8. The method according to claim 1 comprising broadcasting the first image to multiple computerized devices of multiple users.

9. The method according to claim 8 further comprising receiving feedback only from a single user of the multiple users.

10. The method according to claim 8 further comprising receiving feedback only from at least two users of the multiple users.

11. The method according to claim 1 wherein the HMD is a see-through binocular head mounted display.

12. The method according to claim 11 wherein the adjusting, by the HMD the augmented image to provide an amended augmented image is preceded by performing a calibration process based upon a certain image of an object and feedback from the first user.

13. A non-transitory computer readable medium that once executed by a computer causes the computer to execute the steps of:

transmitting to an intermediate computerized system a first image that was acquired, at an acquisition point of time, by a head mounted display (HMD) mounted on a head of a first user;

transmitting the first image from the intermediate computerized system and towards a computerized device of a second user;

displaying on the computerized device of the second user the first image;

receiving feedback, in response to the displaying and by the intermediate computerized system;

generating, by at least one of the computerized device of the second user and the intermediate computerized system, an augmented image that comprises the first image and the feedback;

transmitting to the HMD the augmented image;

adjusting, by the HMD the augmented image to provide an amended augmented image that corresponds to visual information as viewed by the HMD at a display point in time; and

displaying, by the HMD, the amended augmented image.

14. A method for feedback based maintenance, the method comprises:

transmitting to an intermediate computerized system a first image that was acquired, at an acquisition point of time, by first user device that differs from a head mounted display (HMD) mounted on a head of the first user;

transmitting the first image from the intermediate computerized system and towards a computerized device of a second user;

displaying on the computerized device of the second user the first image;

receiving feedback, in response to the displaying and by the intermediate computerized system;

generating, by at least one of the computerized device of the second user and the intermediate computerized system, an augmented image that comprises the first image and the feedback;

transmitting to the HMD the augmented image;

adjusting, by the HMD the augmented image to provide an amended augmented image that corresponds to visual information as viewed by the HMD at a display point in time; and

displaying, by the HMD, the amended augmented image.