Method and system for RFID-assisted imaging

• US 8,668,136 B2
• Filed: 03/01/2012
• Issued: 03/11/2014
• Est. Priority Date: 03/01/2012
• Status: Active Grant

• Current Assignee: Trimble Navigation Limited (Trimble Inc.)
• Original Assignee: Trimble Navigation Limited (Trimble Inc.)
• Inventors: Ahern, Brian, Schoner, Bernd
• Primary Examiner(s): MARSHALL, CHRISTLE I
• Application Number:
  US13/410,084
• Publication Number:
  US 20130228620A1
• Time in Patent Office: 740 Days
• Field of Search: 235/375
• US Class Current: 235/375
• CPC Class Codes: G01C 11/02 Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
• CPC Class Codes: G01C 15/002 Active optical surveying means optical plumbing G01C15/105
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06K 7/10297 arrangements for handling protocols designed for non-contact record carriers such as RFIDs NFCs, e.g. ISO/IEC 14443 and 18092 cryptographic protocols H04L9/00; network security protocols H04L63/00; real-time communication protocols in data switching networks H04L65/00; network protocols for data switching network services H04L67/00
• CPC Class Codes: G06Q 10/20 Administration of product repair or maintenance
• CPC Class Codes: H04N 1/00342 with a radio frequency tag transmitter or receiver storage of additional information in an RFID tag attached to an image bearing paper sheet H04N1/32138
• CPC Class Codes: H04N 1/32128 attached to the image data, e.g. file header, transmitted message header, information on the same page or in the same computer file as the image for information embedded in the image H04N1/32144
• CPC Class Codes: H04N 13/178 Metadata, e.g. disparity information
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00
• CPC Class Codes: H04N 2201/0084 Digital still camera
• CPC Class Codes: H04N 2201/3205 of identification information, e.g. name or ID code H04N2201/3209 takes precedence
• CPC Class Codes: H04N 2201/3225 of data relating to an image, a page or a document

Abstract

A method includes acquiring imaging data of a scene using an imaging tool. The method also includes extracting radio frequency identification (RFID) data stored in an RFID tag associated with the scene. The method further includes associating the RFID data with the imaging data.

Claims

1. A method comprising:

acquiring imaging data of a scene using an imaging tool, wherein an object is depicted within the scene;

processing the imaging data to create a three-dimensional representation of the scene, including a three-dimensional representation of the object;

extracting radio frequency identification (RFID) data stored in an RFID tag associated with the scene, wherein the RFID data comprises information that indicates what the object depicted within the scene is; and

associating the RFID data with the imaging data to identify a position of the object within the three-dimensional representation of the scene.

2. The method of claim 1 further comprising storing the RFID data and the imaging data.

3. The method of claim 2 wherein the RFID data is stored as metadata along with the imaging data.

4. The method of claim 1 wherein the imaging data is associated with one or more objects in the scene and the RFID tag is associated with at least one of the one or more objects.

5. The method of claim 1 wherein extracting RFID data comprises detecting the RFID tag using an RFID reader.

6. The method of claim 1 wherein acquiring imaging data and detecting the RFID tag are performed concurrently.

7. The method of claim 1 further comprising retrieving the RFID data in relation to the imaging data.

8. The method of claim 1 wherein the three-dimensional representation of the object is a point cloud representation of the object.

9. The method of claim 1 wherein the imaging tool comprises a time-of-flight laser rangefinder.

10. The method of claim 1 wherein the imaging tool comprises one or more cameras.

11. The method of claim 1 wherein the RFID reader is co-located with the imaging tool.

12. The method of claim 11 wherein the imaging tool has a first field of view, and the RFID reader has a second field of view, and wherein the second field of view at least partially overlaps with the first field of view.

13. The method of claim 12 further comprising varying the second field of view of the RFID reader, thereby facilitating a unique identification of the object.

14. The method of claim 13 wherein the second field of view of the RFID reader is varied statically or dynamically.

15. The method of claim 1 wherein the imaging tool and the RFID reader reside in a hand-held device.

16. The method of claim 1 wherein the imaging tool comprises a triangulation laser scanner.

17. The method of claim 1 wherein the RFID data includes reference information about the object.

18. The method of claim 1 wherein the RFID data includes a reference to a computer-added design (CAD) model of the object.

19. The method of claim 1 wherein the RFID tag is embedded in an optical marker, wherein:

the optical marker is used as a reference point in creating the three-dimensional representation of the scene; and

the RFID data stored in the RFID tag includes an identification of the optical marker.

20. The method of claim 19 wherein the imaging data include two or more images of the object acquired from at least two different perspectives, each of the two or more images including the optical marker, and the method further comprises:

identifying the optical marker using the RFID data; and

creating the three-dimensional representation of the object from the two or more images using the optical marker as the reference point.

21. The method of claim 20 further comprising, before identifying the optical marker:

determining a distance from the RFID reader to the optical marker using the RFID reader;

determining an angle of view of the optical marker with respect to the imaging tool using the imaging tool; and

determining a location of the optical marker based on the determined distance using the RFID reader and the determined angle of view using the imaging tool.

22. A system comprising:

an imaging tool operable to acquire imaging data of a scene, wherein;

an object is depicted within the scene; and

the imaging data is processed to create a three-dimensional representation of the scene, including a three-dimensional representation of the object; and

an RFID reader operable to extract RFID data stored in an RFID tag associated with the scene, wherein;

the RFID data comprises information that indicates what the object depicted within the scene is; and

the RFID data is associated with the imaging data to identify a position of the object within the three-dimensional representation of the scene.

23. The system of claim 22 wherein the RFID reader is further operable to detect the RFID tag.

24. The system of claim 22 further comprising:

a memory; and

a processor operable to;

associate the RFID data with the imaging data; and

store the RFID data as metadata along with the imaging data in the memory.

25. The system of claim 24 wherein the three-dimensional representation of the scene is a 3D point cloud.

26. The system of claim 22 wherein the RFID data is associated with the imaging data and is retrievable via the association with the imaging data.

27. The system of claim 22 wherein the RFID data includes a metadata tag associated with the imaging data.

28. The system of claim 22 wherein the imaging data includes a point cloud representing the object.

29. The system of claim 22 wherein the imaging tool comprises a laser scanner using a time-of-flight laser rangefinder.

30. The system of claim 22 wherein the imaging tool comprises one or more cameras.

31. The system of claim 22 wherein the RFID reader and the imaging tool are co-located.

32. The system of claim 31 wherein the imaging tool has a first field of view, and the RFID reader has a second field of view, and wherein the second field of view at least partially overlaps with the first field of view.

33. The system of claim 32 wherein the second field of view of the RFID reader is statically or dynamically variable.

34. The system of claim 32 wherein the imaging tool comprises a triangulation laser scanner.

35. The system of claim 22 wherein the RFID data references a CAD model of the object.

36. The system of claim 22 wherein the RFID tag is embedded in an optical marker, wherein:

the optical marker is used as a reference point in creating the three-dimensional representation of the scene; and

the RFID data stored in the RFID tag includes an identification of the optical marker.

37. The system of claim 36 wherein the imaging data include two or more images of the object acquired from at least two different perspectives, each image including the optical marker, and a processor is configured to:

identify the optical marker using the RFID data; and

create a three-dimensional point cloud from the two or more images using the optical marker as a reference point.

38. The system of claim 36 wherein a processor is configured to determine a location of the optical marker by:

determining a distance from the RFID reader to the optical marker using the RFID reader;

determining an angle of view from the optical marker with respect to the imaging tool using the imaging data; and

determining the location of the optical marker based on the determined distance from the RFID reader and the determined angle of view based on the imaging data.

39. A method of collecting information comprising;

acquiring imaging data of a plurality of objects using an imaging tool;

creating a three-dimensional representation of the plurality of objects based on the imaging data; and

extracting RFID data stored in an RFID tag, wherein;

the RFID data comprises, or comprises a reference to, a three-dimensional model;

the three-dimensional model was created before acquiring the imaging data of the plurality of objects; and

the three-dimensional model is a representation of at least one object of the plurality of objects.

40. The method of claim 39 further comprising storing the RFID data and the three-dimensional representation.

41. The method of claim 39 further comprising identifying the at least one object using the RFID data.

42. The method of claim 39 further comprising detecting the RFID tag prior to extracting RFID data stored in the RFID tag.

43. The method of claim 39 wherein the three-dimensional model comprises a point cloud.

44. The method of claim 43 further comprising:

receiving a search query related to the at least one object;

identifying the three-dimensional model of the at least one object using the stored RFID data, wherein the three-dimensional model was created before acquiring the imaging data; and

displaying the three-dimensional model of the at least one object.

45. The method of claim 39 wherein the RFID data further includes reference information for the at least one object.

46. The method of claim 44 wherein the search query includes an identification of the at least one object.

47. The method of claim 44 wherein the search query comprises a link provided on a user interface.

48. The method of claim 39 further comprising:

displaying the three-dimensional representation of the plurality of objects;

identifying the at least one object using the stored RFID data; and

displaying a list comprising the at least one object along with the three-dimensional representation of the plurality of objects.

49. The method of claim 48 further comprising:

receiving a user selection of the at least one object in the list; and

in response to receiving the user selection;

retrieving the RFID data associated with the at least one object; and

displaying at least a portion of the retrieved RFID data.

50. The method of claim 48 wherein:

additional RFID data is extracted from a plurality of RFID tags, the plurality of RFID tags comprising the RFID tag,

the additional RFID data identifies objects in a subset of the plurality of objects, and

the method further comprises linking the subset of the plurality of objects and the additional RFID data with a database, wherein the database includes reference information related to each object of the subset of the plurality of objects.

51. The method of claim 50 further comprising:

receiving a user selection of an object of the subset of the plurality of objects; and

in response to receiving the user selection;

retrieving reference information from the database based on the user selection; and

displaying at least a portion of the reference information retrieved from the database.

52. The method of claim 51 wherein the reference information includes lifecycle information of the selected object.

53. The method of claim 39 wherein the three-dimensional model is a CAD model of the at least one object, and the method further comprises:

displaying the three-dimensional representation of the plurality of objects;

retrieving the CAD model corresponding to the at least one object; and

overlaying the CAD model on the displayed three-dimensional representation of the plurality of objects.

54. A method of constructing a 3D point cloud, the method comprising:

obtaining a first image including at least a scene from a first perspective, wherein the first image includes an optical marker;

receiving RFID data stored on an RFID tag associated with the optical marker;

identifying a physical location of the optical marker using the RFID data stored on the RFID tag;

obtaining a second image including at least the scene from a second perspective, wherein the second image includes the optical marker; and

constructing the 3D point cloud using the first image, the second image, and the optical marker as a reference point in both the first image and the second image.

55. The method of claim 54 wherein obtaining the first image and the second image comprises using an imaging tool and receiving RFID data comprises using an RFID tag reader integrated with the imaging tool.

56. The method of claim 54 wherein the scene comprises a plurality of assets.

57. The method of claim 56 wherein each of the plurality of assets are associated with an RFID tag including information related each of the plurality of assets.

58. The method of claim 54 wherein the optical marker is co-located with the RFID tag.

59. The method of claim 54 wherein the optical marker is located a predetermined distance from the RFID tag and the RFID tag includes information related to the predetermined distance.