Method and apparatus for performing field data collection

• US 6,711,535 B2
• Filed: 06/12/2001
• Issued: 03/23/2004
• Est. Priority Date: 09/16/1996
• Status: Expired due to Term

• Current Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Original Assignee: Datria Systems Incorporated (Honeywell International Inc.)
• Inventors: Yotka, John Anthony, Ford, Terry Edward, Turek, Richard James Jr.
• Primary Examiner(s): EDOUARD, PATRICK NESTOR
• Application Number:
  US09/879,348
• Publication Number:
  US 20010053970A1
• Time in Patent Office: 1,015 Days
• Field of Search: 704/1, 704/9, 704/270, 704/276, 342/357.06, 342/357.09, 342/357.13, 455/456, 701/208, 701/213
• US Class Current: 704/1
• CPC Class Codes: G01C 21/3629 Guidance using speech or audio output, e.g. text-to-speech text to speech systems per se G10L13/00
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

A data collection and automatic database population system which combines global positioning system (GPS), speech recognition software, radio frequency (RF) communications, and geographic information system (GIS) to allow rapid capture of field data, asset tracking, and automatic transfer of the data to a GIS database. A pre-defined grammar allows observations to be continuously captured along with GPS location and time, and stored on the field mobile unit. A mobile unit'"'"'s location is tracked in real time or post processed through wireless RF transmission of location information between the mobile unit and a central processing station. The captured data is electronically transferred to a central processing station for quality assurance and automatic population of the GIS database. The system provides for automatic correlation of field data with other GIS database layers. Tools to generate predefined or user defined reports, work orders, and general data queries allow exploitation of the GIS database.

Claims

1. A method for performing field data collection, the method comprising:

(a) capturing a verbal observation as audio data;

(b) time-stamping said audio data;

(c) capturing location data contemporaneously with said verbal observation with a location receiver device;

(d) time-stamping said location data;

(e) interpreting said audio data using semantic information stored in a reference grammar;

(f) creating a recognized text data record from said interpreted audio data;

(g) merging said location data and said recognized text data record to form a time-based observation data object; and

(h) converting said location data into a time-based track data object.

2. The method according to claim 1 wherein said location data comprises at least one of a latitude, a longitude, and an altitude.

3. The method according to claim 1 wherein a user performs the field data collection with:

a mobile unit in communication with said location receiver device for capturing said location data, wherein said location receiver device comprises a global positioning satellite receiver;

a microphone in communication with a system sound card in said mobile unit for capturing said verbal observations by said user; and

a central processing unit clock in said mobile unit for time-stamping said audio data and said location data.

4. The method according to claim 3 further comprising:

repeating steps (a) through (h) for a plurality of verbal observations yielding a data store of a plurality of time-based track data objects which represent said mobile unit'"'"'s historical travel path, and a data store of a plurality of time-based observation data objects which represent a time sequence in which said plurality of time-based observation data objects were observed.

5. The method according to claim 4 wherein said merging step (g) further comprises:

interpreting a phrase comprised of at least two of said recognized text data records utilizing a reference observation semantic.

6. The method according to claim 5 wherein said interpreting a phrase step further comprises:

detecting semantic error conditions.

7. The method according to claim 4 further comprising:

associating said plurality of time-based track data objects with said plurality of time-based observation data objects to produce a tour object which comprises a correlated location, observation, and time data set.

8. The method according to claim 3 further comprising:

capturing an external sensor data contemporaneously with said captured audio data;

time-stamping said external sensor data; and

correlating said captured audio data with said captured external sensor data.

9. The method according to claim 8 wherein said external sensor data comprises at least one of a switch on/off state, an analog value, a digital photo data, a laser ranging data, and a sensor digital data collected by a device capable of transmitting said sensor digital data from said device to said mobile unit.

10. The method according to claim 3 further comprising:

during the field data collection, transporting said mobile unit by at least one of a vehicle, a bicycle, a backpack, and a carrying by hand.

11. The method according to claim 3 further comprising:

performing quality assurance functions by said user through interactive displays presented on said mobile unit.

12. The method according to claim 3 further comprising:

transferring said time-based observation data object and said time-based track data object from said mobile unit to a processing computer for storage in a geographic information system database associated with said processing computer.

13. The method according to claim 12 further comprising:

routing said transferred time-based observation data object to a proper layer of said geographic information system database.

14. The method according to claim 12 further comprising:

detecting inconsistencies between said time-based observation data object and a reference data store.

15. The method according to claim 1 further comprising:

correcting said location data in real-time utilizing a reference Differential Global Positioning Service (DGPS) data.

16. The method according to claim 1 further comprising:

correcting said location data utilizing a reference Differential Global Positioning Service (DGPS) data in a post-processing mode.

17. The method according to claim 1 wherein said location receiver device comprises a global positioning satellite receiver.

18. The method according to claim 1 wherein said interpreting step (e) further comprises:

detecting word level interpretation error conditions.

19. The method according to claim 1 wherein said time-based observation data object comprises at least one of an asset observation, a condition observation, a defect observation, and a repair observation.

20. The method according to claim 19 wherein said asset observation and said defect observation further comprise a spatial type observation, wherein said spatial type observation comprises at least one of a discrete observation, a linear observation, and a polygonal observation.

21. The method according to claim 20 wherein said discrete observation of said spatial type observation represents an item at a discrete location.

22. The method according to claim 20 wherein said linear observation of said spatial type observation represents an item running from a start position to a stop position.

23. The method according to claim 20 wherein said polygonal observation of said spatial type observation represents an area where said time-based observation data objects are observed.

24. The method according to claim 20 further comprising:

tracking said linear observations and said polygonal observations interleaved with said discrete observations.

25. An apparatus for performing field data collection, the apparatus comprising:

a mobile computer, said mobile computer comprising,

a central processing unit;

a system sound card connectable to said central processing unit;

a microphone in communication with said system sound card, wherein said microphone and said system sound card cooperatively capture a verbal observation as audio data;

a central processing unit clock connectable to said central processing unit; and

a memory connectable to said central processing unit, wherein said central processing unit loads into said memory a reference grammar containing semantic information; and

a location receiver device in communication with said mobile computer for capturing location data contemporaneously with said verbal observation;

wherein said central processing unit clock time-stamps said captured audio data and time-stamps said location data received from said location receiver device;

and further wherein said central processing unit;

interprets said audio data using said reference grammar containing semantic information;

creates a recognized text data record;

merges said location data and said recognized text data record to form a time-based observation data object; and

converts said location data into a time-based track data object.

26. The apparatus according to claim 25 wherein said location receiver device captures location data comprising at least one of a latitude, a longitude, and an altitude.

27. The apparatus according to claim 25 further comprising:

at least one external device for capturing and sending external sensor data to said mobile computer, wherein;

said central processing unit clock time-stamps said captured external sensor data; and

said central processing unit correlates said captured audio data with said captured external sensor data.

28. The apparatus according to claim 27 wherein said at least one external device comprises at least one of a digital camera and a laser ranging device.

29. The apparatus according to claim 25 wherein said mobile computer further comprises:

a storage device connectable to said central processing unit, wherein said storage device stores;

a plurality of time-based track data objects which represent said mobile computer'"'"'s historical travel path; and

a plurality of time-based observation data objects which represent a time sequence in which said plurality of time-based observation data objects were captured.

30. The apparatus according to claim 29 wherein said central processing unit loads into said memory a reference observation semantic that:

interprets a phrase comprised of at least two of said recognized text data records; and

detects semantic error conditions.

31. The apparatus according to claim 29 wherein said mobile computer further comprises:

a transfer means for transferring said plurality of time-based observation data objects and said plurality of time-based track data objects from said mobile computer to a processing computer for storage in a geographic information system database associated with said processing computer.

32. The apparatus according to claim 31 wherein said transfer means comprises at least one of a RF wireless transmitter and a wired electronic connection.

33. The apparatus according to claim 29 wherein said central processing unit associates said plurality of time-based track data objects with said plurality of time-based observation data objects to produce a tour object which comprises a correlated location, observation, and time data set.

34. The apparatus according to claim 25 wherein said location receiver device comprises a global positioning satellite receiver.

35. The apparatus according to claim 25 further comprising:

a means for transporting said mobile computer while performing field data collection.

36. The apparatus according to claim 35 wherein said means for transporting comprises at least one of a vehicle, a bicycle, a backpack, and a carrying by hand.

37. The apparatus according to claim 25 wherein said mobile computer further comprises:

an interactive display connectable to said central processing unit for enabling a user to perform quality assurance functions.