Information processing method and device

• US 10,075,695 B2
• Filed: 12/03/2014
• Issued: 09/11/2018
• Est. Priority Date: 07/03/2014
• Status: Active Grant

• Current Assignee: Lenovo Company Limited (Lenovo Group Ltd.)
• Original Assignee: Lenovo Company Limited (Lenovo Group Ltd.)
• Inventors: Shen, Hao
• Primary Examiner(s): Patel, Jitesh
• Application Number:
  US14/559,067
• Publication Number:
  US 20160005142A1
• Time in Patent Office: 1,378 Days
• CPC Class Codes: G06F 18/211 Selection of the most significant subset of features
• CPC Class Codes: G06F 18/28 Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/771 Feature selection, e.g. selecting representative features from a multi-dimensional feature space
• CPC Class Codes: G06V 10/772 Determining representative reference patterns, e.g. averaging or distorting patterns; Generating dictionaries
• CPC Class Codes: H04N 13/20 Image signal generators

Abstract

An information processing method and device are disclosed. The information processing method is applied to an information processing device in which a 3D map and a spatial topological structure management-based feature library created in advance for a certain environment are contained, and different users in the certain environment can determine their location. The method includes acquiring a first image taken by a first user; extracting one or more first feature points in the first image to obtain first feature descriptors; obtaining 3D locations of the first feature points based on 3D location of the first user, the first image, and the feature library; determining feature descriptors to be updated based on 3D location of the first user, the 3D locations of the first feature points, the first feature descriptors corresponding to the first feature points, and existing feature descriptors in the feature library; and updating the feature library.

Claims

1. An information processing method applied to an information processing device in which a 3D map and a spatial topological structure management-based feature library created in advance for a certain environment is contained, and different users in the certain environment are able to determine their location in accordance with images taken by themselves and the feature library, the method comprising:

acquiring a first image taken by a first user;

extracting one or more first feature points in the first image to obtain first feature descriptors for characterizing the first feature points;

obtaining 3D locations of the first feature points based on a 3D location of the first user, the first image, and the feature library;

determining feature descriptors to be updated based on the 3D location of the first user, the 3D locations of the first feature points, the first feature descriptors corresponding to the first feature points, and existing feature descriptors in the feature library; and

updating the feature library based on the feature descriptors to be updated,

wherein the step of determining feature descriptors to be updated comprises;

selecting one of the first feature points in the first image;

determining a minimum spatial node crossed by a line between the selected first feature point and the location of the first user;

judging whether any feature descriptors of the feature library are contained within the minimum spatial node; and

determining all of the feature descriptors contained within the minimum spatial node as feature descriptors to be deleted if the judgement is positive, wherein the minimum spatial node is a space resolution in the certain space environment, and the certain space environment is divided by the minimum spatial node,

wherein the step of determining feature descriptors to be updated further comprises;

determining respective first feature descriptors in the first image as feature descriptors to be added;

acquiring a plurality of strength information of the feature descriptors to be added; and

among the feature descriptors to be added, removing feature descriptors of which the plurality of strength information are less than a first threshold; and

wherein the step of determining feature descriptors to be updated further comprises;

determining respective first feature descriptors in the first image as feature descriptors to be added;

calculating a similarity degree between the feature descriptors to be added and the feature descriptors in the feature library at the corresponding minimum spatial node, wherein they are considered to be similar if the similarity degree greater than a second threshold; and

among the feature descriptors to be added, removing feature descriptors which are considered similar to the feature descriptors in the feature library.

2. The information processing method of claim 1, wherein the step of obtaining 3D locations of the first feature points based on 3D location of the first user, the first image, and the feature library further comprises:

obtaining 3D locations of the first feature points under a local coordinate system based on the first image;

transforming the 3D locations of the first feature points from the local coordinate to a global coordinate system based on the 3D location of the first user.

3. The information processing method of claim 1, further comprises:

acquiring a second image taken by a second user;

extracting second feature points in the second image to obtain second feature descriptors for characterizing the second feature points;

obtaining a 3D location of the second feature points based on a location of the second user and the second image;

determining feature descriptors to be updated based on a 3D location of the second user, the 3D locations of the second feature points, the feature descriptors corresponding to the second feature points, and existing feature descriptors in the feature library;

judging whether the feature descriptors to be updated at the same minimum spatial node are obtained in the first and second images;

calculating a correlation degree between the feature descriptors to be updated in the first and second images at the same minimum spatial node if the judgment is positive, and determining the feature descriptors to be updated in the first and second images as the final feature descriptors to be updated only when the correlation degree is greater than a third threshold; and

determining them not as final feature descriptors to be updated if the judgment is negative.

4. An information processing device for creating in advance a 3D map and a spatial topological structure management-based feature library for a certain environment, such that different users in the certain environment are able to determine their locations in accordance with image taken by themselves and the feature library, the device comprising:

a storage for storing the 3D map and the feature library;

a processor

wherein the storage stores instructions that can be executed by the processor to;

acquire a first image taken by a first user;

extract one or more first feature points in the first image to obtain first feature descriptors for characterizing the first feature points;

obtain 3D locations of the first feature points based on location of the first user, the first image, and the feature library; and

determine feature descriptors to be updated based on 3D location of the first user, the 3D locations of the first feature points, the first feature descriptors corresponding to the first feature points, and existing feature descriptors in the feature library, and for updating the feature library based on the feature descriptors to be updated,

wherein the step of determining feature descriptors to be updated comprises;

selecting one of the first feature points in the first image; and

determining a minimum spatial node crossed by the line between the first feature points and the location of the first user, and for judging whether any feature descriptor in the feature library is contained within the minimum spatial nodes;

determining all of the feature descriptors contained within the minimum spatial node as feature descriptors to be deleted if the judgement is positive, and wherein the minimum spatial node is a space resolution in the certain space environment, and the certain space environment is divided by the minimum spatial node,

wherein the step of determining feature descriptors to be updated comprises;

determining respective first feature descriptors in the first image as the feature descriptors to be added;

acquiring a plurality of strength information of feature descriptors to be added; and

removing the feature descriptors of which the plurality of the strength information are less than a first threshold among the feature descriptors to be added; and

wherein the step of determining feature descriptors to be updated comprises;

determining respective first feature descriptors in the first image as feature descriptors to be added;

calculating a similarity degree between the feature descriptor to be added and a feature point in the feature library at the corresponding minimum spatial node, wherein they are considered to be similar if the similarity degree is greater than a second threshold; and

removing the feature descriptors with the similar feature descriptors in the feature library.

5. The information processing device of claim 4, wherein the step of obtaining 3D locations of the first feature points based on 3D location of the first user, the first image, and the feature library further comprises:

obtaining 3D locations of the first feature points under a local coordinate system based on the first image;

transforming the 3D locations of the first feature points from the local coordinate to a global coordinate system based on the 3D location of the first user.

6. The information processing device of claim 4, wherein the storage stores instructions which can be executed by the processor to further achieve the following steps:

acquire a second image taken by a second user;

extract one or more second feature points in the second image to obtain second feature descriptors for characterizing the second feature points;

obtain 3D locations of the second feature points based on location of the second user, the second image, and the feature library;

determine feature descriptors to be updated based on the 3D location of the second user, the 3D locations of the second feature points, the second feature descriptors corresponding to the second feature points, and existing feature descriptors in the feature library, and

make a judgment on whether the feature descriptors to be updated at the same minimum spatial node are obtained in the first and second images;

calculate a correlation degree between the feature descriptors to be updated in the first and second images at the same minimum spatial node if the judgment is positive, and determine the feature descriptors to be updated in the first and second images as the final feature descriptors to be updated only when the correlation degree is greater than a third threshold; and

determine they are not final feature descriptors to be updated if the judgment is negative.