Method and system for the acquisition of data and for the display of data

• US 20070236366A1
• Filed: 01/24/2007
• Published: 10/11/2007
• Est. Priority Date: 07/25/2004
• Status: Abandoned Application

• Current Assignee: Israel Aerospace Industries Limited (Government of Israel)
• Original Assignee: Israel Aerospace Industries Limited (Government of Israel)
• Inventors: Gur, Joshua, Seror, Ianiv
• Application Number:
  US11/657,118
• Publication Number:
  US 20070236366A1
• Time in Patent Office: Days
• US Class Current: 340/945
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G06V 2201/02 Recognising information on displays, dials, clocks

Abstract

A method and system are provided for acquiring data from an instrument panel or the like by obtaining images of the panel and optically identifying readings of the instruments in the image. The readings are in the form of geometrically identifiable forms having a meaning according to predetermined criteria. A coded data stream is created representative of the identified readings and transmitted to another location. The received data stream is then encoded and displayed on a virtual image of the instrument panel. The data stream can also be stored.

Claims

1. A method for acquiring data from an instrument panel having at least one instrument display wherein a value of a parameter being monitored via the display is associated with at least an angular disposition of an indicator with respect to the display comprising:

—

(a) providing a first image of said at least one angular display comprising a plurality of image elements;

(b) transforming at least a portion of said first image into a corresponding R-θ

image, wherein each said image element of the at least portion of said first image is relocated in said R-θ

image to a position with respect to a first axis and a second axis according to the radius (R) and angular disposition (θ

), respectively, of said image element with respect to a center and datum angle, respectively, in said first image;

(c) identifying said parameter value from said R-θ

image by identifying an image corresponding to at least a part of said indicator therein and determining a position of said indicator with respect to said R-θ

image;

(d) providing a coded data stream representative of said parameter value;

(e) at least one of transmitting and recording said coded data stream.

2. A method according to claim 1, wherein step (a) comprises providing a global image of said panel having a plurality of said displays, and further comprising dividing the said global image into a corresponding plurality of said first images, each corresponding to a region of interest (ROI), wherein each said ROI comprising one said display, and wherein steps (b) to (e) may be performed for each said ROI.

3. A method according to claim 2, wherein at least one said ROI comprises said indicator in the form of an image of a dial that points to the said parameter value, and step (c) comprises identifying a position of said dial in said R-θ

image with respect to said second axis.

4. A method according to claim 3, wherein said coded data stream comprises digital values representative of said angular disposition.

5. A method according to claim 3, further comprising serially compiling said coded data stream for each said ROI into a data package.

6. A method according to claim 1, further comprising a calibrating procedure for determining a position of said center in said first image prior to step (a).

7. A method according to claim 6, wherein said calibrating procedure includes the sub steps:

—

(i) providing a first calibration image of said at least one angular display;

(ii) analyzing said first calibration image to identify a center of rotation with respect to an angular rotation of said indicator.

8. A method according to claim 7, wherein sub step (ii) comprises:

—

(ii-a) analyzing said first calibration image to identify a periphery of the display that is nominally circular about said center;

(ii-b) if said periphery is circular within a predefined tolerance determining and providing the location of said center;

(ii-c) if said periphery is not circular;

(A) determining a calibration transformation required for elements of the said first calibration image such that said periphery in said calibration image is transformed to a circular shape in a second calibration image created via said calibration transformation; and

(B) determining a location of a transformed center of the circular shape in the second calibration image.

9. A method according to claim 8, wherein said second calibration image is used as said first image in step (a).

10. A method according to claim 1, wherein steps (a) to (d) are applied to a plurality of said instrument displays of instruments that monitor at least engine conditions of at least one engine, and further comprising the step of analyzing said data stream with respect to reference data to obtain a measure of engine performance.

11. A method according to claim 10, wherein said analyzing step comprises comparing actual engine data as provided via said data stream with reference engine data obtained at substantially the same operating conditions to provide a deviation therebetween, and generating an alert when said deviation exceeds a predetermined threshold.

12. A method according to claim 1, further comprising the step of recording said coded data streams in a crash proof device.

13. A method according to claim 1, wherein step (c) comprises transmitting said coded data streams by means of a radio signal.

14. A method according to claim 1, wherein said parameter includes at least any one of airspeed, altitude, pitch, roll, yaw, turn rate, vertical speed, horizontal situation (compass heading), engine rpm, oil status, fuel status, oil temperature, Mach number, chronological time.

15. A method according to claim 1, further comprising the step of providing at least one second image of an external environment.

16. A method according to claim 15, further comprising the step of providing at least one of:

attitude data, GPS data, DGPS data, altitude data, voice data.

17. A method according to claim 16, further comprising the steps:

—

(A) providing a virtual image corresponding to the said external environment corresponding to said at least one said second image;

(B) comparing said at least one second image with said corresponding virtual image;

(C) identifying differences between the images in step (B).

18. A method according to claim 17, further comprising providing digital data representative of said differences in step (C) and optionally displaying said digital data.

19. A method according to claim 18, further comprising including at least one of said attitude data, GPS data, DGPS data, altitude data, voice data, said digital data representative of said differences in step (C), in said coded data stream.

20. A method according to claim 1, wherein said image element comprises at least one pixel.

21. A method according to claim 1, further comprising providing an image of a user that is facing said panel, analyzing said image and comparing with images of authorized users, and further comprising the step of generating an alert if the said user image does not match at least one authorized user image.

44. A computer readable medium storing instructions for programming a processing means of a system to perform a method as defined in claim 1.

22. A method for displaying data comprising:

—

(i) at least one of receiving and reading a coded data stream representative of an image of said at least one display of an instrument panel;

(ii) creating an image of said at least one readout based on said corresponding said coded data stream;

(iii) displaying said image in the context of an image representative of said panel.

23. A method according to claim 22, wherein said coded data stream is created by imaging said at least one instrument panel display to provide a first image thereof, and manipulating said first image to provide said coded data stream, wherein said coded data stream is representative of at least one value of a parameter being monitored at the display.

24. A method according to claim 22, wherein said coded data stream is created according to a method for acquiring data from an instrument panel having at least one instrument angular display wherein a value of a parameter being monitored at the display is associated with at least an angular disposition with respect to the display comprising:

—

(a) providing a first image of said at least one angular display

(b) transforming at least a portion of said first image into a corresponding R-θ

image, wherein each said image element of the at least portion of said first image is relocated in said R-θ

image to a position with respect to a first axis and a second axis according to the radius (R) and angular disposition (θ

), respectively, of said image element with respect to a center and datum angle, respectively, in said first image;

(c) identifying said parameter value from said R-θ

image by identifying an image corresponding to at least a part of said indicator therein and determining a position of said indicator with respect to said R-θ

image;

(d) providing said coded data stream, wherein said data stream is representative of said parameter value;

(e) at least one of transmitting and recording said coded data stream.

25. A method according to claim 24, wherein at least one said coded data stream relates to a dial of a dial-type instrument, and step (ii) comprises creating an image of a dial at an angular disposition of said dial with respect to a datum, said angular position being correlated with said coded data stream.

26. A method according to claim 24, wherein said coded data stream comprises digital values representative of said angular disposition.

27. A method according to claim 23, wherein said parameter includes at least one of airspeed, altitude, pitch, roll, yaw, turn rate, vertical speed, horizontal situation (compass heading), engine rpm, oil status, fuel status, oil temperature, Mach number, chronological time.

28. A method according to claim 22, further comprising the step of displaying at least one of said attitude data, GPS data, DGPS data, altitude data, voice data, said digital data representative of said differences in step (C), in said coded data stream.

29. A method according to claim 22, comprising

providing a flight simulator program having capabilities of simulating a flight and of displaying instruments displays indicative of said flight from a vantage point of a user, wherein inputs for driving the instruments displays are normally provided by manipulation of suitable flight controls by said user;

adapting at least one of said flight simulator program and said coded data stream such that said inputs for said instruments displays are provided by said coded data stream.

45. A computer readable medium storing instructions for programming a processing means of a system to perform a method as defined in claim 22.

47. A computer readable medium storing instructions for programming a processing means of a system to perform a method as defined in claim 45.

30. A system for the acquisition of data from an instrument panel comprising at least one instrument angular display wherein a value of a parameter being monitored at the display is associated with at least an angular disposition of an indicator with respect to the display, comprising:

—

(a) at least one first camera for providing a first image of said at least one display;

(b) processing system for processing said first image of said at least one display to provide a coded data stream representative of said image via a method comprising;

—

(I) transforming at least a portion of said first image into a corresponding R-θ

image, wherein each said predefined image element of the at least portion of said first image is relocated in said R-θ

image to a position with respect to a first axis and a second axis according to the radius (R) and angular disposition (θ

), respectively, of said predefined image element with respect to a center and datum angle, respectively, in said first image;

(II) identifying said parameter value from said R-θ

image;

(III) providing said coded data stream, wherein said coded data stream is representative of said parameter value;

(c) at least one of transmitting apparatus and recording apparatus for transmitting and recording, respectively, said coded data stream.

31. A system according to claim 30, wherein said panel comprises a plurality of said displays, and said processing system is adapted for dividing the said image into a corresponding plurality of regions of interest (ROI), each said ROI comprising one said display, wherein said processing system processes said first image of each said readout to provide a corresponding plurality of coded data streams representative of said images.

32. A system according to claim 31, comprising at least one fiducial marker provided on said panel for aligning the said ROI with respect to an image of said marker.

33. A system according to claim 32, wherein said fiducial comprises a white outer annular portion circumscribing a dark central portion.

34. A system according to claim 31, wherein said at least one first camera and said panel are comprised in an aircraft cockpit.

35. A system according to claim 34, further comprising a crash proof device operatively connected to said processing system for recording said coded data streams therein.

36. A system according to claim 34, wherein said transmission apparatus comprises a suitable radio transmitter.

37. A system according to claim 30, further comprising at least one second camera for obtaining images of an external environment.

38. A system according to claim 37, further comprising at least one of:

attitude data module, GPS system, DGPS system, altitude module, voice compression module.

39. A system for displaying data comprising:

—

(i) at least one of data receiving apparatus and reading apparatus for receiving and reading, respectively, a coded data stream representative of an image of said at least one readout of an instrument panel;

(ii) processing apparatus for creating an image of said at least one readout based on said corresponding said coded data stream;

(iii) displaying apparatus for displaying said image in the context of an image representative of said panel.

40. A system according to claim 39, wherein said coded data stream is created by imaging said at least one instrument panel display to provide a first image thereof, and manipulating said first image to provide said coded data stream, wherein said coded data stream is representative of at least one value of a parameter being monitored at the display.

41. A system according to claim 39, wherein said coded data stream is created according to a method for acquiring data from an instrument panel having at least one instrument angular display wherein a value of a parameter being monitored at the display is associated with at least an angular disposition with respect to the display comprising:

—

(a) providing a first image of said at least one angular display

(b) transforming at least a portion of said first image into a corresponding R-θ

image, wherein each said image element of the at least portion of said first image is relocated in said R-θ

image to a position with respect to a first axis and a second axis according to the radius (R) and angular disposition (θ

), respectively, of said image element with respect to a center and datum angle, respectively, in said first image;

(c) identifying said parameter value from said R-θ

image by identifying an image corresponding to at least a part of said indicator therein and determining a position of said indicator with respect to said R-θ

image;

(d) providing said coded data stream, wherein said data stream is representative of said parameter value;

(e) at least one of transmitting and recording said coded data stream.

42. A system according to claim 39, wherein said processing apparatus is adapted for at least one of receiving and reading a data package comprising a plurality of said coded data streams, each representative of an image of one of a plurality of displays of said instrument panel.

43. A system according to claim 42, wherein said processing apparatus is adapted for dividing the data package into a corresponding plurality of coded data streams, and said display apparatus is adapted for displaying each image corresponding to a coded data stream in a window of said panel image corresponding to the position of the corresponding readout of said instrument panel.

46. A method for providing a simulation in a first location of an instrument status at a second location, comprising:

(a) providing an image of an instrument having said instrument status at said second location;

(b) analyzing said image to provide a measure of said instrument status;

(c) providing a coded data stream representative of said measure of said instrument status;

(d) transmitting said coded data stream to said first location

(e) receiving said coded data stream at said first location;

(f) simulating said instrument status by reconstructing and displaying a virtual image corresponding to said instrument based on said coded data stream, such that said virtual image comprises a representation of said instrument status.

48. A system for providing a simulation in a first location of an instrument status at a second location, comprising:

(g) at least one first camera for providing an image of said instrument at said second location;

(h) first processing system for analyzing said image to provide a measure of said instrument status, and for providing a coded data stream representative of said measure of said instrument status;

(i) transmitting apparatus for transmitting said coded data stream to said first location;

(j) receiving apparatus at said first location for receiving said coded data stream;

(k) second processing system for simulating said instrument status by reconstructing a virtual image corresponding to said instrument based on said coded data stream, such that said virtual image comprises a representation of said instrument status, and first display apparatus for displaying said virtual image.

49. A system according to claim 48, further comprising a second display apparatus coupled to a flight simulation processor for displaying at said second location computer generated images of said instrument, and wherein said at least one camera is focused on said second display.