Systems and methods for machine learning enhanced by human measurements

• US 10,303,982 B2
• Filed: 07/25/2018
• Issued: 05/28/2019
• Est. Priority Date: 06/28/2013
• Status: Active Grant

• Current Assignee: President and Fellows of Harvard College (Harvard Corporation)
• Original Assignee: President and Fellows of Harvard College (Harvard Corporation)
• Inventors: Cox, David, Scheirer, Walter, Anthony, Samuel, Nakayama, Ken
• Primary Examiner(s): Lee, Jonathan S
• Application Number:
  US16/044,822
• Publication Number:
  US 20180357515A1
• Time in Patent Office: 307 Days
• CPC Class Codes: G06F 18/2178 based on feedback of a supervisor
• CPC Class Codes: G06F 18/2413 based on distances to training or reference patterns
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

In various embodiments, training objects are classified by human annotators, psychometric data characterizing the annotation of the training objects is acquired, a human-weighted loss function based at least in part on the classification data and the psychometric data is computationally derived, and one or more features of a query object are computationally classified based at least in part on the human-weighted loss function.

Claims

1. A system for data classification and identification, the system comprising:

a database of training objects, the database comprising a storage medium populated with stored computer records specifying, for each of a plurality of training objects, psychometric data pertaining thereto;

a computer processor;

a classification device for receiving query objects different from the training objects;

a penalization module, executable by the computer processor, for deriving a human-weighted loss function based at least in part on the psychometric data, the loss function comprising penalties for misclassification, magnitudes of the penalties increasing with increasing deviation from the classification data; and

a classification module, executable by the computer processor, for classifying features of query objects based at least in part on the human-weighted loss function.

2. The system of claim 1, wherein the human-weighted loss function is based at least in part on a hinge loss function.

3. The system of claim 1, wherein the psychometric data comprises at least one of (i) response time for classifying one or more features, (ii) accuracy of feature classification, or (iii) presentation time of one or more training objects.

4. The system of claim 1, wherein the training objects comprise digital images, digital video, and/or data representative of digital images and/or digital video.

5. The system of claim 4, wherein features of the training objects comprise human figures, portions of human figures, and/or visual attributes related to human figures.

6. The system of claim 4, wherein features of the training objects comprise trees, cars, buses, buildings, and/or streets.

7. The system of claim 1, wherein the classification module is configured to classify features of query objects comprising digital images, digital video, and/or data representative of digital images and/or digital video.

8. The system of claim 7, wherein the features comprise human figures, portions of human figures, and/or visual attributes related to human figures.

9. The system of claim 7, wherein the features comprise trees, cars, buses, buildings, and/or streets.

10. The system of claim 1, further comprising a display module, executable by the computer processor, for displaying training objects, features of training objects, query objects, and/or features of query objects.

11. The system of claim 1, wherein the classification device comprises a camera or other image- and/or video-capture device.

12. The system of claim 1, wherein the classification device comprises a microphone or other audio-capture device.

13. The system of claim 1, wherein the classification device comprises a communication interface to a server and/or database.

14. The system of claim 1, wherein the classification module is configured to detect obscured or occluded human figures or portions thereof within query objects.

15. The system of claim 1, wherein the classification module is configured to detect trees, cars, buses, buildings, and/or streets within query objects.

16. The system of claim 1, wherein the classification device is disposed in or on a driverless or semiautonomous vehicle.

17. The system of claim 1, wherein the computer processor is disposed in or on a driverless or semiautonomous vehicle.

18. The system of claim 1, wherein the psychometric data comprises one or more weighted or unweighted psychophysical behavioral measures and/or one or more event-related potentials.

19. The system of claim 1, wherein, for at least some of the training objects, the psychometric data characterizes annotation and/or classification of the training objects by one or more human annotators.