Systems and methods for tracking object association over time

• US 9,269,159 B2
• Filed: 06/05/2014
• Issued: 02/23/2016
• Est. Priority Date: 06/05/2014
• Status: Active Grant

• Current Assignee: Promethean Limited (Best Assistant Education Group Co.)
• Original Assignee: Promethean Limited (Best Assistant Education Group Co.)
• Inventors: Velozo, Steven C., Decker, Alex H., Hillier, Jason C.
• Primary Examiner(s): Liew, Alex
• Application Number:
  US14/297,513
• Publication Number:
  US 20150356746A1
• Time in Patent Office: 628 Days
• Field of Search: 382/103
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/188 Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

A system and method for tracking association of two or more objects over time, according to various embodiments, is configured to determine the association based at least in part on an image. The system may be configured to capture the image, identify two or more objects of interest within the image, determine whether the two or more objects are associated in the image, and store image association data for the two or more objects. In various embodiments the system is configured to create a timeline of object association over time for display to a user.

Claims

1. A computer system for tracking an association between two or more objects over time, the system comprising an imaging device comprising one or more cameras and one or more processors, wherein the imaging device is configured to:

a. capture a first image at a first time;

b. at least partially in response to capturing the first image, perform an object detection and identification analysis to detect and identify two or more objects of interest in the first image;

c. at least partially in response to identifying the two or more objects of interest in the first image, determine, based at least in part on one or more object association criteria, whether the two or more objects are associated in the first image;

d. at least partially in response to determining that the two or more objects are associated in the first image, store object association data for the two or more objects at the first time;

e. capture a second image at a second time;

f. at least partially in response to capturing the second image, detect, based at least in part on one or more change criteria, one or more changes between the first image and the second image;

g. at least partially in response to detecting the one or more changes between the first image and the second image, perform the object detection and identification analysis to detect and identify the two or more objects of interest in the second image;

h. at least partially in response to identifying the two or more objects of interest in the second image, determine, based at least in part on the one or more object association criteria, whether the two or more objects are associated in the second image; and

i. at least partially in response to determining that the two or more objects are associated in the second image, store object association data for the two or more objects at the second time.

2. The computer system of claim 1, further comprising a server operatively coupled to the first imaging device, wherein:

a. the first imaging device is configured to transmit at least a portion of the first image to the server; and

b. the server is configured to;

i. receive the at least a portion of the first image from the first imaging device;

ii. at least partially in response to receiving the at least a portion of the first image, perform a second object detection and identification analysis to identify two or more objects of interest in the first image;

iii. at least partially in response to identifying the two or more objects of interest in the first image, determining, based at least in part on the one or more object association criteria, whether the two or more objects of interest in the first image are associated in the first image; and

iv. at least partially in response to determining that the two or more objects are associated in the first image, storing object association information for the two or more objects for the first time.

3. The computer system of claim 2, wherein:

a. the first imaging device is configured to transmit at least a portion of the second image to the server; and

b. the server is further configured to;

i. receive the at least a portion of the second image from the first imaging device;

ii. at least partially in response to receiving the at least a portion of the second image, perform the second object detection and identification analysis to identify two or more objects of interest in the second image;

iii. at least partially in response to identifying the two or more objects of interest in the second image, determining, based at least in part on the one or more object association criteria, whether the two or more objects of interest in the second image are associated in the second image; and

iv. at least partially in response to determining that the two or more objects are associated in the second image, storing object association information for the two or more objects for the second time.

4. The computer system of claim 1, wherein:

a. performing object detection and identification analysis comprises using one or more facial recognition techniques to identify one or more of the two or more objects; and

b. the one or more of the two or more objects comprise one or more faces.

5. The computer system of claim 4, wherein performing object detection analysis comprises recognizing one or more of the two or more objects based at least in part on one or more substantially unique markings on the one or more of the two or more objects.

6. The computer system of claim 5, wherein the one or more substantially unique marking comprise one or more machine-readable indicia selected from a group consisting of:

a. three dimensional barcode;

b. a two dimensional barcode;

c. matrix barcode;

d. optical character recognition; and

e. hand writing recognition.

7. The computer system of claim 1, wherein the computer system is further configured to, at least partially in response to determining that the two or more objects are not associated in the second image, store object disassociation data for the two or more objects at the second time.

8. The computer system of claim 1, wherein the computer system is further configured to individually track each of the two or more objects over time.

9. The computer system of claim 2, wherein the computer system is further configured to synchronize the imaging device object association data with the server object association data.

10. The computer system of claim 2, wherein the object detection and identification analysis comprises a low resolution object detection and identification algorithm and the second object detection and identification analysis comprises a high resolution object detection and identification algorithm.

11. A computer-implemented method of tracking object association over a period time comprising:

a. capturing, by a first imaging device, a plurality of images each taken at a particular time of the period of time;

b. performing, by the first imaging device, object detection and identification analysis to identify two or more objects of interest in a first particular one of the plurality of images;

c. detecting, by the first imaging device, based at least in part on one or more change criteria, one or more changes between the first particular one of the plurality of images and a second particular one of the plurality of images;

d. at least partially in response to detecting the one or more changes between the first particular one of the plurality of images and the second particular one of the plurality of images, performing, by the first imaging device, object detection and identification analysis to identify the two or more objects of interest in the second particular one of the plurality of images;

e. at least partially in response to identifying the two or more objects of interest in the second particular one of the plurality of images, determining, by the first imaging device, based at least in part on one or more object association criteria, whether the two or more objects are associated in the second particular one of the plurality of images;

f. at least partially in response to determining that the two or more objects are associated in the second particular one of the plurality of images, storing, by the first imaging device, object association data for the two or more objects for the particular time at which the first imagining device captured the second particular one of the plurality of images.

12. The computer-implemented method of claim 11, wherein capturing the plurality of images comprises capturing the plurality of images from a video feed taken by the first imaging device.

13. The computer-implemented method of claim 11, wherein the two or more objects comprise:

a. a first student; and

b. a document comprising a plurality of questions, the plurality of questions having a plurality of related answer choices.

14. The computer-implemented method of claim 13, the method further comprising:

a. determining, by the first imaging device, an indicated answer choice for one or more of the plurality of questions; and

b. scoring, by the first imaging device, the one or more of the plurality of questions based at least in part on an answer key associated with the document and the indicated answer choice for the one or more of the plurality of questions.

15. The computer-implemented method of claim 14, the method further comprising:

a. determining, by the first imaging device, a substantially current grade for the document based at least in part on the scored one or more questions; and

b. displaying the substantially current grade to one or more teachers of the first student.

16. The computer-implemented method of claim 15, the method further comprising:

a. performing, by the first imaging device, object detection and identification analysis to identify two or more objects of interest in a third particular one of the plurality of images;

b. detecting, by the first imaging device, based at least in part on one or more change criteria, one or more changes between the third particular one of the plurality of images and the first particular one of the plurality of images, the one or more changes comprising completion, by the first student, of one or more additional questions on the document;

c. determining, by the first imaging device, an indicated answer choice for the one or more additional questions; and

d. scoring, by the first imaging device, the one or more additional questions based at least in part on the answer key indicated answer choice for one or more additional questions.

17. The computer-implemented method of claim 16, the method further comprising:

a. determining, by the first imaging device, a substantially updated grade for the document based at least in part on the scored one or more additional questions; and

b. displaying the substantially updated grade the one or more teachers of the first student.