METHOD AND SYSTEM FOR DESCRIBING AND ORGANIZING IMAGE DATA

• US 20120287272A1
• Filed: 07/27/2012
• Published: 11/15/2012
• Est. Priority Date: 12/18/2009
• Status: Active Grant

• Current Assignee: Toyota Motor Corporation
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Uehara, Yasuo
• Granted Patent: US 8,405,722 B2
• Time in Patent Office: Days
• US Class Current: 348/143
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/587 using geographical or spatial information, e.g. location
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G08B 13/19613 Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion

Abstract

Methods and systems for describing and organizing image data. An analysis system includes a sensor, a processing unit, a machine, a database, and/or a display. The analysis system is connected to an electronic device. The sensor generates sensor data which is imported by the processing unit. The processing unit analyzes the sensor data to determine features of the sensor data such as the subject, the time, the location, the object, the event, and/or the action in the sensor data. The processing unit creates entries for the sensor data in a features index. The processing unit receives a query for an item and uses the features index to determine search results including the item, such as a location and a time of the item. The processing unit can also determine whether a machine has completed its mission using the features in the sensor data.

Claims

1. A method for finding an item comprising:

importing sensor data;

identifying a subject in the sensor data;

identifying a time in the sensor data;

identifying a location in the sensor data;

identifying an object in the sensor data;

identifying an event in the sensor data;

identifying an action in the sensor data;

creating an entry in the feature index for the sensor data from the subject, the time, the location, the object, the event, and the action;

receiving a query for an item;

searching the feature index to determine whether the item is contained in the sensor data; and

displaying the result of the search.

2. The method of claim 1 further comprising analyzing the query for the item to determine whether the result of the search will be more specific than a predetermined specificity, and asking for a more detailed query when the result of the search will not be more specific than the predetermined specificity.

3. The method of claim 1 wherein the sensor data includes image data.

4. The method of claim 1 wherein the sensor data includes video data.

5. The method of claim 1 wherein the sensor data includes global positioning system data.

6. The method of claim 1 wherein the result includes a time and location of the item.

7. A method for finding an item comprising:

importing sensor data;

identifying a subject in the sensor data;

identifying a time in the sensor data;

identifying a location in the sensor data;

identifying an object in the sensor data;

identifying an event in the sensor data;

identifying an action in the sensor data;

creating an entry in the feature index for the sensor data from the subject, the time, the location, the object, the event, and the action;

receiving a first query for a first item;

searching, at a first iteration frequency, the feature index to determine whether the first item is contained in the sensor data;

displaying a result of the search for the first item;

analyzing the first query for the first item to determine whether the result of the search for the first item will be more specific than a predetermined specificity, and requesting a more detailed query when the result of the search for the first item will not be more specific than the predetermined specificity;

receiving a second query for a second item;

searching, at a second iteration frequency slower than the first iteration frequency, the feature index to determine whether the second item is contained in the sensor data; and

displaying a result of the search for the second item.

8. The method of claim 7 wherein the sensor data includes image data.

9. The method of claim 7 wherein the sensor data includes video data.

10. The method of claim 7 wherein the sensor data includes global positioning system data.

11. The method of claim 7 wherein the result of the search for the first item includes a time and a location of the first item or the result of the search for the second item includes a time and a location for the second item.

12. A method for operating a machine comprising:

receiving a mission state of the machine;

receiving a current state of the machine;

analyzing the mission state of the machine and the current state of the machine;

continuing to operate the machine when the mission state of the machine and the current state of the machine are substantially different; and

ceasing to operate the machine when the mission state of the machine and the current state of the machine are substantially the same.

13. The method of claim 12 wherein the mission state and the current state, each includes a subject, a time, a location, an object, an event, and an action.

14. The method of claim 12 wherein the analysis of the mission state of the machine and the current state of the machine includes comparing the subject, the time, the location, the object, the event, and the action of the mission state of the machine to the subject, the time, the location, the object, the event, and the action of the current state of the machine.

15. The method of claim 12 further comprising importing sensor data, wherein the current state of the machine is based on the sensor data.