Information processing system, information processing method, information processing apparatus and control method and control program thereof, and communication terminal and control method and control program thereof

• US 9,792,528 B2
• Filed: 01/30/2013
• Issued: 10/17/2017
• Est. Priority Date: 01/30/2012
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Nomura, Toshiyuki, Yamada, Akio, Iwamoto, Kota, Mase, Ryota
• Primary Examiner(s): Park, Chan
• Assistant Examiner(s): Thomas, Mia M
• Application Number:
  US14/375,524
• Publication Number:
  US 20150010237A1
• Time in Patent Office: 1,721 Days
• CPC Class Codes: G06F 16/7837 using objects detected or recognised in the video content
• CPC Class Codes: G06F 16/7854 using shape G06F16/7837 takes precedence
• CPC Class Codes: G06F 18/213 Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
• CPC Class Codes: G06T 2207/30112 Baggage; Luggage; Suitcase
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/7715 Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

A search object and m-number of first local features respectively constituted by a feature vector of 1 to i dimensions of local areas of m-number of feature points in an image of the search object are stored, feature points are extracted from the image, second local features respectively constituted by a feature vector of 1 dimension to j dimensions are generated with respect to local areas of n-number of feature points, a smaller number of dimensions among the number of dimensions i of the first local features and the number of dimensions j of the second local features is selected, and an existence of the search object in the image in the video is recognized when a prescribed ratio of the m-number of first local features up to the selected number of dimensions corresponds to the n-number of second local features up to the selected number of dimensions.

Claims

1. An information processing system, comprising a processor and a memory, which includes:

a first local feature storing unit configured to store, in association with each other, a search object and m-number of first local features which are respectively constituted by a feature vector of 1 dimension to i dimensions and which are generated for each of m-number of local areas including each of m-number of feature points in an image of the search object;

a second local feature generating unit configured to extract an n-number of feature points from an image of a video captured by a first imaging unit, and that generates n-number of second local features respectively constituted by a feature vector of 1 dimension to j dimensions for n-number of local areas including each of the n-number of feature points; and

a recognizing unit that is configured to select a smaller number of dimensions among the number of dimensions i of the feature vector of the first local feature and the number of dimensions j of the feature vector of the second local feature, and that recognizes that the search object exists in the image in the video when determining that a prescribed ratio or more of the m-number of first local features constituted by a feature vector up to the selected number of dimensions corresponds to the n-number of second local features constituted by a feature vector up to the selected number of dimensions.

2. The information processing system according to claim 1, further comprising a first local feature generating unit configured to extract an m-number of feature points from an image of the search object and that generates the m-number of first local features which are respectively constituted by a feature vector of 1 dimension to i dimensions.

3. The information processing system according to claim 2, further comprising a second imaging unit configured to capture an image of the search object, wherein

the first local feature generating unit is configured to generate the m-number of first local features based on the image of the search object captured by the second imaging unit.

4. The information processing system according to claim 3, further comprising an informing unit configured to inform a recognition result obtained from the recognizing unit.

5. The information processing system according to claim 4, further comprising a first communication terminal configured for generating a local feature of the search object, a second communication terminal configured for searching the search object based on the local feature, and an information processing apparatus configured to communicate with the first communication terminal and the second communication terminal, wherein

the first communication terminal includes the second imaging unit, the first local feature generating unit, and the informing unit and is configured to transmit the m-number of first local features from the first communication terminal to the information processing apparatus,

the second communication terminal includes the first imaging unit and the second local feature generating unit and is configured to transmit the n-number of second local features from the second communication terminal to the information processing apparatus, and

the information processing apparatus includes the first local feature storing unit and the recognizing unit and is configured to transmit a recognition result of the recognizing unit from the information processing apparatus to the first communication terminal.

6. The information processing system according to claim 5, wherein

the second local feature generating unit includes an accuracy adjusting unit configured to adjust an accuracy of the second local feature, and

the recognizing unit is configured to confirm the search object based on a second local feature generated by the second local feature generating unit by adjusting to a higher accuracy.

7. The information processing system according to claim 4, wherein

the second local feature generating unit includes an accuracy adjusting unit configured to adjust an accuracy of the second local feature, and

the recognizing unit is configured to confirm the search object based on a second local feature generated by the second local feature generating unit by adjusting to a higher accuracy.

8. The information processing system according to claim 3, wherein

the second local feature generating unit includes an accuracy adjusting unit configured to adjust an accuracy of the second local feature, and

the recognizing unit is configured to confirm the search object based on a second local feature generated by the second local feature generating unit by adjusting to a higher accuracy.

9. The information processing system according to claim 2, wherein

the search object is a lost article or a stolen article,

the first local feature storing unit is configured to store a first local feature generated by the first local feature generating unit from an image of the lost article or the stolen article to be searched, and

the recognizing unit is configured to recognize that the lost article or the stolen article exists in the image in the video when determining that a prescribed ratio or more of the m-number of first local features corresponds to the n-number of second local features.

10. The information processing system according to claim 2, wherein

the search object is a person,

the first local feature storing unit is configured to store a first local feature generated by the first local feature generating unit from an image of the person to be searched, and

the recognizing unit is configured to recognize that the person exists in the image in the video when determining that a prescribed ratio or more of the m-number of first local features corresponds to the n-number of second local features.

11. The information processing system according to claim 2, wherein

the search object is a duplicate,

the first local feature storing unit is configured to store a first local feature generated by the first local feature generating unit from an original image, and

the recognizing unit is configured to recognize that the original image exists in the image in the video when determining that a prescribed ratio or more of the m-number of first local features corresponds to the n-number of second local features.

12. The information processing system according to claim 1, wherein

the second local feature generating unit includes accuracy adjusting unit that is configured to adjust an accuracy of the second local feature, and

the recognizing unit is configured to confirm the search object based on a second local feature generated by the second local feature generating unit by adjusting to a higher accuracy.

13. The information processing system according to claim 1, wherein the first local feature and the second local feature are each generated by dividing a local area including a feature point extracted from an image into a plurality of sub-areas and generating a feature vector of a plurality of dimensions constituted by a histogram in a gradient direction in the plurality of sub-areas.

14. The information processing system according to claim 13, wherein the first local feature and the second local feature are each generated by removing a dimension at which a correlation between adjacent sub-areas is higher among the generated feature vector of a plurality of dimensions.

15. The information processing system according to claim 13, wherein the plurality of dimensions of the feature vector are arranged to circle the local area once for every predetermined number of dimensions so that dimensions can be selected starting from a first dimension in a descending order of contributions to the feature point and in accordance with an increase in accuracy that is required with respect to the local feature.

16. The information processing system according to claim 15, wherein the second local feature generating unit is configured to generate, corresponding to a correlation of the search objects to be searched, the second local feature with a smaller number of dimensions for a search object that has a lower correlation with another search object.

17. The information processing system according to claim 15, wherein the first local feature storing unit is configured to store, corresponding to a correlation of the search objects to be searched, the first local feature with a smaller number of dimensions for a search object that has a lower correlation with another search object.

18. The information processing system according to claim 2, wherein

the second local feature generating unit includes an accuracy adjusting unit configured to adjust an accuracy of the second local feature, and

the recognizing unit is configured to confirm the search object based on a second local feature generated by the second local feature generating unit by adjusting to a higher accuracy.

19. An information processing method in an information processing system including a first local feature storing unit that is configured to store, in association with each other, a search object and m-number of first local features which are respectively constituted by a feature vector of 1 dimension to i dimensions and which are generated for each of m-number of local areas including each of m-number of feature points in an image of the search object, the method comprising the steps of:

extracting n-number of feature points from an image of a video captured by first imaging unit and generating n-number of second local features respectively constituted by a feature vector of 1 dimension to j dimensions for n-number of local areas including each of the n-number of feature points; and

selecting a smaller number of dimensions among the number of dimensions i of the feature vector of the first local feature and the number of dimensions j of the feature vector of the second local feature, and recognizing that the search object exists in the image in the video when determining that a prescribed ratio or more of the m-number of first local features constituted by a feature vector up to the selected number of dimensions corresponds to the n-number of second local features constituted by a feature vector up to the selected number of dimensions.