Fusing RFID and vision for surface object tracking

• US 8,847,739 B2
• Filed: 08/04/2008
• Issued: 09/30/2014
• Est. Priority Date: 08/04/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wilson, Andrew D., Olwal, Alex
• Primary Examiner(s): Mehmood, Jennifer
• Assistant Examiner(s): ALAM, MIRZA F
• Application Number:
  US12/185,174
• Publication Number:
  US 20100026470A1
• Time in Patent Office: 2,248 Days
• Field of Search: 715/863, 345/173, 345/179, 345/158, 345/156, 340/572.3, 340/572.7, 340/539.26, 340/666, 340/584, 340/593, 340/594, 340/601, 340/602, 340/689, 340/572.1, 340/572.4, 340/572.5, 340/988, 340/10.1, 340/10.2, 340/10.3, 340/10.31, 340/10.4, 340/10.41, 340/10.42, 340/825.69, 340/825.71, 340/825.73, 340/825.31, 340/825.72, 340/539, 340/825.32, 340/10.5, 340/5.61, 340/5.72
• US Class Current: 340/10.52
• CPC Class Codes: G06F 18/25 Fusion techniques
• CPC Class Codes: G06F 3/046 by electromagnetic means
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: G06V 10/80 Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level multimodal speaker identification or verification G10L17/10
• CPC Class Codes: H04Q 2213/13095 PIN / Access code, authentication

Abstract

The claimed subject matter provides a system and/or a method that facilitates detecting and identifying objects within surface computing. An interface component can receive at least one surface input, the surface input relates to at least one of an object, a gesture, or a user. A surface detection component can detect a location of the surface input utilizing a computer vision-based sensing technique. A Radio Frequency Identification (RFID) tag can transmit a portion of RFID data, wherein the RFID tag is associated with the surface input. A Radio Frequency Identification (RFID) fusion component can utilize the portion of RFID data to identify at least one of a source of the surface input or a portion of data to associate to the surface input.

Claims

1. A system that facilitates detecting and identifying objects within surface computing, comprising:

an interface component that captures a surface input of a source on a surface, the source comprising a Radio Frequency Identification (RFID) tag and one or more of an object, a gesture, and a portion of a user;

a surface detection component that;

detects a location of the surface input utilizing a computer vision-based sensing technique or a non-RFID localization technique, and associates a vision time stamp to at least one of addition, removal, or movement of the surface input; and

a Radio Frequency Identification (RFID) fusion component that;

associates an RFID time stamp to a detection of the RFID tag,

receives a portion of RFID data from the RFID tag of the source of the surface input, and

utilizes the portion of RFID data to identify the source of the surface input and a portion of data to associate to the surface input, wherein at least one of the vision time stamp and the RFID time stamp are matched based at least in part upon a correlation in time, the correlation relates to closeness in detection times for the RFID time stamp and the vision time stamp.

2. The system of claim 1, further comprising an RFID component that receives, translates, and communicates the portion of RFID data to the RFID fusion component, the RFID component comprises one or more of:

an RFID reader, an RFID writer, an RFID printer, a printer, a reader, a writer, an RFID transmitter, an antenna, a sensor, a real-time device, an RFID receiver, a real-time sensor, a device extensible to a web service, and a real-time event generation system.

3. The system of claim 1, wherein the surface detection component utilizes a Frame Difference Algebra (FDA) to detect a change between a first still frame and a second still frame.

4. The system of claim 3, wherein the FDA comprises a set of image processing operations in which the at least one of addition, removal, or movement of the surface input is detected.

5. The system of claim 1, further comprising a real time tracking component that employs a frame-to-frame correspondence tracking for the surface input between a previous frame and a subsequent frame, the surface input detected being tagged with an identification (ID).

6. The system of claim 5, wherein the real time tracking component ascertains the frame-to-frame correspondence by one or more of:

computing a time-based correlation and computing a distance of the surface input to a disparate surface input, wherein a shape in the subsequent frame inherits the ID of a closest shape in the previous frame.

7. The system of claim 1, further comprising an image network that provides the portion of data to associate with the surface input identified based upon the portion of RFID data received from the RFID tag.

8. The system of claim 7, the portion of data is at least one of an image, a photo, a portion of audio, a portion of video, a portion of a graphic, a portion of text, or a function.

9. The system of claim 1, wherein the RFID data defines the surface input as an object comprising:

a query object, the query object being a detected surface input associated with a keyword;

a container object, the container object being a detected surface input that is a physical handle to a collection of data;

or

an operator object, the operator object being a detected surface input that executes a function upon activation.

10. A computer-implemented method comprising:

under control of one or more processors configured with executable instructions;

sensing an object, comprising a Radio Frequency Identification (RFID) tag, on a surface utilizing a vision-based detection system;

ascertaining a shape or a location of the object utilizing the vision-based detection system;

receiving a portion of data from the RFID tag of the object, the received portion of data describing a functionality or an identity associated with the object; and

displaying the received portion of data on the surface on which the object is sensed, wherein the received portion of data from the RFID tag defines a surface input of the object as an object type comprising;

a query object, the query object being a detected surface input associated with a keyword;

a container object, the container object being a detected surface input that is a physical handle to a collection of data;

or

an operator object, the operator object being a detected surface input that executes a function upon activation.

11. The method of claim 10, further comprising utilizing the received portion of data to detect the shape of the object or the location of the object on the surface.

12. A computer-implemented method comprising:

under control of one or more processors configured with executable instructions;

detecting a corporeal object, comprising a Radio Frequency Identification (RFID) tag, on a surface;

in response to detecting the corporeal object on the surface, receiving RFID data from the RFID tag of the detected corporeal object; and

displaying a portion of the received RFID data on the surface, wherein the received RFID data defines a surface input of the detected corporeal object as an object type comprising;

a query object, the query object being a detected surface input associated with a keyword;

a container object, the container object being a detected surface input that is a physical handle to a collection of data;

or

an operator object, the operator object being a detected surface input that executes a function upon activation.

13. The method of claim 12, wherein the surface input of the corporeal object comprises the query object associated with the keyword, and the method further comprises:

in response to detecting the corporeal object on the surface, retrieving the keyword from a data store;

obtaining an image based on the retrieved keyword; and

displaying the obtained image around the detected corporeal object on the surface.

14. The method of claim 12, further comprising:

detecting a physical interaction between the detected corporeal object and a user;

manipulating the displayed portion of the received RFID data on the surface based on the detected physical interaction.

15. The method of claim 12, wherein the displayed portion of the received RFID data comprises an image of the corporeal object.

16. The method of claim 12, wherein the corporeal object further comprises at least one other RFID tag that has a different geometry or orientation from the RFID tag from which the RFID data is received, and wherein the method further comprises detecting an orientation of the corporeal object based on the at least one other RFID tag and the RFID tag from which the RFID data is received.

17. The method of claim 12, further comprising adding a time-stamped event in a data store in response to detecting an appearance or disappearance of the RFID tag of the corporeal object.