Method and device for providing a multiple source display and a remote visual inspection system specially adapted for use with the device

• US 6,031,566 A
• Filed: 12/27/1996
• Issued: 02/29/2000
• Est. Priority Date: 12/27/1996
• Status: Expired due to Term

• Current Assignee: Olympus America Incorporated (Olympus Corporation)
• Original Assignee: Olympus America Incorporated (Olympus Corporation)
• Inventors: Leo, Joseph K.
• Primary Examiner(s): Tung, Bryan
• Assistant Examiner(s): VO, TUNG T
• Application Number:
  US08/774,919
• Time in Patent Office: 1,159 Days
• Field of Search: 348/82, 348/65, 348/66, 348/67, 348/74, 348/72, 348/161, 348/162, 348/164, 348/61, 348/143, 348/165, 600/109, 600/110, 324/220, 324/222, 382/103, 382/128
• US Class Current: 348/82
• CPC Class Codes: G02B 7/28 Systems for automatic generation of focusing signals
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

A method and device for displaying information from multiple sources, in a way that permits rapid assimilation and interpretation of the information. The information may originate from the inspection, e.g., via a remote visual inspection system, of a work piece. The method and device processes (i) frames of video data, each frame including data corresponding to an array of pixels, including video of a sensor, and (ii) data output by the sensor by (a) locating the sensor in one of the frames of video data to define an anchor location, (b) accepting the data output by the sensor, (c) converting the data output by the sensor to either a numeric value or a graphical representation, and (d) in the one of the frames, modifying pixels data in a vicinity of the anchor location based on the numeric value or graphical representation to form a modified frame.

Claims

1. A method for processing

video data which includes an image of a sensor, and

data output by the sensor, the method comprising steps of:

a) accepting analog video data;

b) converting the analog video data to frames, each frame including data corresponding to an array of pixels;

c) locating the image of the sensor in one of the frames to define an anchor location;

d) accepting the data output by the sensor;

e) digitizing the data output by the sensor to form digitized sensor data;

f) converting the digitized sensor data to one of a numeric value and a graphical representation; and

g) in the one of the frames, modifying pixels in a vicinity of the anchor location based on the one of numeric value and graphical representation to form a modified frame.

2. The method of claim 1 further comprising a step of:

h) displaying the modified frame.

3. The method of claim 1 further comprising a step of:

h) recording the modified frame.

4. The method of claim 1 further comprising a step of:

h) storing a file including the modified frame.

5. The method of claim 1 wherein the step of locating the video of the sensor in the one of the frames to define an anchor location includes sub-steps of:

i) checking the data corresponding to the array of pixels to determine whether a pixel within the array of pixels has data corresponding to an attribute of the sensor; and

ii) determining the anchor location at a location of a pixel which has data corresponding to the attribute of the sensor.

6. The method of claim 5 wherein the attribute is a particular color.

7. The method of claim 1 wherein the step of locating the video of the sensor in one of the frames to define an anchor location includes sub-steps of:

i) segmenting the frame into a plurality of sub-images;

ii) within one of the sub-images, checking data corresponding to each pixel within the sub-image to determine whether a pixel within the sub-image has data corresponding to an attribute of the sensor;

iii) if the sub-image has a pixel having data corresponding to the attribute of the sensor, determining the anchor location at a location of the pixel which has data corresponding to the attribute of the sensor; and

iv) if the sub-image does not have a pixel having data corresponding to the attribute of the sensor, continuing processing with a next one of the sub-images.

8. The method of claim 7 wherein the attribute is a particular color.

9. The method of claim 7 wherein, if it is known that the video of the sensor is more likely located within a particular sub-image than other sub-images, the particular sub-image is the one of the sub-images.

10. The method of claim 1 further comprising steps of:

selecting the graphical representation from a group of candidate graphical representations to define a selected graphical representation; and

selecting parameters for the selected graphical representation,

wherein, the step of converting the digitized sensor data to one of a numeric value and a graphical representation uses the selected graphical representation and the parameters.

11. The method of claim 1 wherein the graphical representation is one of a cloud, a halo, and an arrow.

12. The method of claim 1 wherein the sensor data includes data of a first type and data of a second type, and wherein the step of converting the digitized sensor data to one of a numeric value and a graphical representation includes sub-steps of:

varying a first attribute of the graphical representation based on the data of the first type; and

varying a second attribute of the graphical representation based on the data of the second type.

13. The method of claim 12 wherein each of the first and second attributes of the graphical representation are selected from a group consisting of:

size, length, diameter, inner diameter, outer diameter, color, brightness, sharpness and orientation.

35. The method of claim 1 wherein the data output by the sensor is non-video data.

36. The method of claim 1 wherein the image of the sensor included in the video data is within a field of view of an image capturing device.

14. A method for processing

frames of video data, each frame including data corresponding to an array of pixels, including an image of a sensor, and

data output by the sensor, the method comprising steps of:

a) locating the image of the sensor in one of the frames of video data to define an anchor location;

b) accepting the data output by the sensor;

c) converting the data output by the sensor to one of a numeric value and a graphical representation; and

d) in the one of the frames, modifying pixels data in a vicinity of the anchor location based on the one of numeric value and graphical representation to form a modified frame.

15. The method of claim 14 further comprising a step of:

e) displaying the modified frame.

16. The method of claim 14 further comprising a step of:

e) recording the modified frame.

17. The method of claim 14 further comprising a step of:

e) storing a file including the modified frame.

18. The method of claim 14 wherein the step of locating the sensor in the one of the frames to define an anchor location includes sub-steps of:

i) checking the data corresponding to the array of pixels to determine whether a pixel within the array of pixels has an data corresponding to attribute of the sensor; and

ii) determining the anchor location at a location of a pixel which has data corresponding to the attribute of the sensor.

19. The method of claim 18 wherein the attribute is a particular color.

20. The method of claim 14 wherein the step of locating the sensor in the one of the frames to define an anchor location includes sub-steps of:

i) segmenting the frame into a plurality of sub-images;

ii) within one of the sub-images, checking data corresponding to each pixel within the sub-image to determine whether a pixel within the sub-image has data corresponding to an attribute of the sensor;

iii) if the sub-image has a pixel having data corresponding to the attribute of the sensor, determining the anchor location at a location of the pixel which has data corresponding the attribute of the sensor; and

iv) if the sub-image does not have a pixel having data corresponding to the attribute of the sensor, continuing processing with a next one of the sub-images.

21. The method of claim 20 wherein the attribute is a particular color.

22. The method of claim 20 wherein, if it is known that the video of the sensor is more likely located within a particular sub-image than other sub-images, the particular sub-image is the one of the sub-images.

23. The method of claim 14 further comprising steps of:

selecting the graphical representation from a group of candidate graphical representations to define a selected graphical representation; and

selecting parameters for the selected graphical representation,

wherein, the step of converting the digitized sensor data to one of a numeric value and a graphical representation uses the selected graphical representation and the parameters.

24. The method of claim 14 wherein the graphical representation is one of a cloud, a halo, and an arrow.

25. The method of claim 14 wherein the sensor data includes data of a first type and data of a second type, and wherein the step of converting the digitized sensor data to one of a numeric value and a graphical representation includes sub-steps of:

varying a first attribute of the graphical representation based on the data of the first type; and

varying a second attribute of the graphical representation based on the data of the second type.

26. The method of claim 25 wherein each of the first and second attributes of the graphical representation are selected from a group consisting of:

size, length, diameter, inner diameter, outer diameter, color, brightness, sharpness and orientation.

27. The method of claim 14, wherein the frames of video data include the field of view of an image capturing device, the method further comprising steps of:

e) fixing the field of view of the image capturing device;

f) moving the sensor with respect to the image capturing device;

g) converting the data output by the sensor to one of a numeric value and a graphical representation;

h) in at least one of the frames, modifying pixels in a vicinity of the anchor location based on the one of numeric value and graphical representation to form at least one modified frame;

i) sampling, in response to one of a user selection and a clock, the one of the numeric value and graphical representation replacing the pixels in the vicinity of the anchor location such that subsequent frames also include the one of the numeric value and graphical representation at and around the anchor location; and

j) generating an inspection profile by repeating steps (f) through (i).

37. The method of claim 14 wherein the data output by the sensor is non-video data.

38. The method of claim 14 wherein the image of the sensor included in the video data is within a field of view of an image capturing device.

28. A device for inspecting an object, the device comprising:

a) a video generation device, having a field of view, for generating video frames of the object, each video frame including data corresponding to an array of pixels;

b) a sensor, located within the field of view of the video generation device, for detecting a physical phenomenon related to an attribute of the object;

c) a signal generation device, for generating a signal based on the physical phenomenon detected by the sensor;

d) a converter, receiving the signal from the signal generation device, for converting the signal to one of a numeric value and graphical representation;

e) an anchor generation device, receiving frames from the video generation device, for determining the location of an image of the sensor within a frame and generating an anchor at the location; and

f) a video processing device, receiving the anchor from the anchor generation device, the frames from the video frame generation device, and the one of the numeric representation and graphical representation from the converter, for modifying pixels in a vicinity of the anchor in at least one of the frames, based on the one of the numeric value and graphical representation.

29. The device of claim 28 wherein the video generation device is a remote visual inspection device.

30. The device of claim 28 wherein the video generation device is one of (a) a video camera, (b) a borescope coupled with a video camera, (c) a fiberscope coupled with a video camera, and (d) a videoimagescope provided with a camera control unit.

31. The device of claim 30 wherein the sensor is provided through a working channel in the video generation device.

32. The device of claim 28 wherein the video generation device includes a 90 degree tip adapter and a fixed probe.

33. The device of claim 29 wherein the remote visual inspection device includes an insertion tube having a proximal end and a distal end, and including a working channel extending from the proximal end to the distal end of the insertion tube, and

wherein the sensor passes through the working channel,

the device further comprising a biasing system for biasing the sensor away from the distal end of the insertion tube.

34. The device of claim 28 wherein the sensor is one of an eddy current probe and an ultrasound probe.