Method and system for describing and organizing image data

• US 8,405,722 B2
• Filed: 07/27/2012
• Issued: 03/26/2013
• Est. Priority Date: 12/18/2009
• Status: Expired due to Fees

• Current Assignee: Toyota Motor Corporation
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Uehara, Yasuo
• Primary Examiner(s): NGUYEN, KIM T
• Application Number:
  US13/560,782
• Publication Number:
  US 20120287272A1
• Time in Patent Office: 242 Days
• Field of Search: 348/143, 348/154
• US Class Current: 348/143
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/587 using geographical or spatial information, e.g. location
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G08B 13/19613 Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion

Abstract

Methods and systems for describing and organizing image data. An analysis system includes a sensor, a processing unit, a machine, a database, and/or a display. The analysis system is connected to an electronic device. The sensor generates sensor data which is imported by the processing unit. The processing unit analyzes the sensor data to determine features of the sensor data such as the subject, the time, the location, the object, the event, and/or the action in the sensor data. The processing unit creates entries for the sensor data in a features index. The processing unit receives a query for an item and uses the features index to determine search results including the item, such as a location and a time of the item. The processing unit can also determine whether a machine has completed its mission using the features in the sensor data.

Claims

1. A method for finding an item comprising:

importing sensor data;

identifying a subject in the sensor data;

identifying a time in the sensor data;

identifying a location in the sensor data;

identifying an object in the sensor data;

identifying an event in the sensor data;

identifying an action in the sensor data;

creating an entry in the feature index for the sensor data from the subject, the time, the location, the object, the event, and the action;

receiving a first query for a first item;

searching, at a first iteration frequency, the feature index to determine whether the first item is contained in the sensor data;

displaying a result of the search for the first item;

analyzing the first query for the first item to determine whether the result of the search for the first item will be more specific than a predetermined specificity, and requesting a more detailed query when the result of the search for the first item will not be more specific than the predetermined specificity;

receiving a second query for a second item;

searching, at a second iteration frequency slower than the first iteration frequency, the feature index to determine whether the second item is contained in the sensor data; and

displaying a result of the search for the second item.

2. The method of claim 1 wherein the sensor data includes image data.

3. The method of claim 1 wherein the sensor data includes video data.

4. The method of claim 1 wherein the sensor data includes global positioning system data.

5. The method of claim 1 wherein the result of the search for the first item includes a time and a location of the first item or the result of the search for the second item includes a time and a location for the second item.