IMAGE PROCESSING DEVICE AND METHOD

• US 20130148849A1
• Filed: 10/12/2012
• Published: 06/13/2013
• Est. Priority Date: 12/07/2011
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Nakai, Takehiro, Fujimoto, Katsuhito, Ozawa, Noriaki, Moteki, Atsunori, Hara, Nobuyuki, Murase, Taichi
• Granted Patent: US 9,213,897 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/443 by matching or filtering
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 30/142 using hand-held instruments; Constructional details of the instruments

Abstract

An image processing device that accesses a storage unit that stores a feature point of a recognition-target object, the device includes an obtaining unit mounted with a user and configured to obtain image data in a direction of a field of view of the user; a recognizing unit configured to recognize the recognition-target object included in the image data by extracting a feature point from the image data and associating the extracted feature point and the feature point of the recognition-target object stored in the storage unit with each other; a calculating unit configured to calculate a location change amount of the feature point corresponding to the recognition-target object recognized by the recognizing unit from a plurality of the image data obtained at different times and calculate a motion vector of the recognition-target object from the location change amount; and a determining unit configured to determine a movement.

Claims

1. An image processing device that accesses a storage unit that stores a feature point of a recognition-target object, the device comprising:

an obtaining unit mounted with a user and configured to obtain image data in a direction of a field of view of the user;

a recognizing unit configured to recognize the recognition-target object included in the image data by extracting a feature point from the image data and associating the extracted feature point and the feature point of the recognition-target object stored in the storage unit with each other;

a calculating unit configured to calculate a location change amount of the feature point corresponding to the recognition-target object recognized by the recognizing unit from a plurality of the image data obtained at different times and calculate a motion vector of the recognition-target object from the location change amount; and

a determining unit configured to determine a movement of the user based on the motion vector.

2. The image processing device according to claim 1,

wherein the recognizing unit calculates a location change amount of the recognition-target object or an area change amount of the recognition-target object from the plurality of image data and, based on the location change amount or the area change amount, recognizes an object the user is gazing at.

3. The image processing device according to claim 1,

wherein the calculating unit specifies a location of the recognition-target object included in the image data from the feature point of the recognition-target object and calculates, as the location change amount of the feature point, a location change amount of at least four endpoints containing the recognition-target object the location of which is specified.

4. The image processing device according to claim 1,

wherein the determining unit has stored a model histogram in which a movement of the user and a motion vector are associated with each other, and determines the movement of the user by comparing similarity between a histogram of motion vectors generated by accumulating a predetermined amount of motion vectors calculated by the calculating unit and the model histogram.

5. The image processing device according to claim 1,

wherein the storage unit has further stored auxiliary information regarding the recognition-target object associated with the movement of the user.

6. The image processing device according to claim 5, further comprising:

a display unit configured to display the auxiliary information regarding the recognition-target object associated with the movement of the user determined by the determining unit.

7. An image processing method comprising:

obtaining, from an obtaining unit mounted with a user, image data in a direction of a field of view of the user;

extracting a feature point from the image data;

reading a feature point of a recognition-target object stored in a storage unit;

recognizing an object included in the image data by associating the extracted feature point and the feature point of the recognition-target object read from the storage unit;

calculating, by a processor, a location change amount at the feature point of the recognized recognition-target object from a plurality of the image data obtained at different times and calculating a motion vector of the recognition-target object from the location change amount; and

determining a movement of the user based on the motion vector.

8. The image processing method according to claim 7,

wherein, in the recognizing, a location change amount of the recognition-target object or an area change amount of the recognition-target object is calculated from the plurality of image data and, based on the location change amount or the area change amount, an object the user is gazing at is recognized.

9. The image processing method according to claim 7,

wherein, in the calculating, a location of the recognition-target object included in the image data is specified from the feature point of the recognition-target object, and a location change amount of at least four endpoints containing the recognition-target object the location of which is specified is calculated as the location change amount of the feature point.

10. The image processing method according to claim 7,

wherein, in the determining, the movement of the user is determined by comparing similarity between a model histogram in which a movement of the user and a motion vector are associated with each other and a histogram of motion vectors generated by accumulating a predetermined amount of motion vectors calculated in the calculating.

11. The image processing method according to claim 7,

wherein the storage unit has further stored auxiliary information regarding the recognition-target object associated with the movement of the user.

12. The image processing method according to claim 11, further comprising:

displaying the auxiliary information regarding the recognition-target object associated with the movement of the user determined in the determining.

13. A computer-readable storage medium storing an image processing program that causes a computer to execute a process comprising:

obtaining, from an obtaining unit mounted with a user, image data in a direction of a field of view of the user;

extracting a feature point from the image data;

reading a feature point of a recognition-target object stored in a storage unit;

recognizing an object included in the image data by associating the extracted feature point and the feature point of the recognition-target object read from the storage unit;

calculating a location change amount at the feature point of the recognized recognition-target object from a plurality of the image data obtained at different times and calculating a motion vector of the recognition-target object from the location change amount; and

determining a movement of the user based on the motion vector.

14. The computer-readable storage medium according to claim 13,

wherein, in the recognizing, a location change amount of the recognition-target object or an area change amount of the recognition-target object is calculated from the plurality of image data and, based on the location change amount or the area change amount, an object the user is gazing at is recognized.

15. The computer-readable storage medium according to claim 13,

wherein, in the calculating, a location of the recognition-target object included in the image data is specified from the feature point of the recognition-target object, and a location change amount of at least four endpoints containing the recognition-target object the location of which is specified is calculated as the location change amount of the feature point.

16. The computer-readable storage medium according to claim 13,

wherein, in the determining, the movement of the user is determined by comparing similarity between a model histogram in which a movement of the user and a motion vector are associated with each other and a histogram of motion vectors generated by accumulating a predetermined amount of motion vectors calculated in the calculating.

17. The computer-readable storage medium according to claim 13,

wherein the storage unit has further stored auxiliary information regarding the recognition-target object associated with the movement of the user.

18. The image processing method according to claim 17, further comprising:

displaying the auxiliary information regarding the recognition-target object associated with the movement of the user determined in the determining.

19. An image processing device that accesses a storage unit that stores a feature point of a recognition-target object, the device comprising:

an obtaining unit mounted with a user and configured to obtain image data in a direction of a field of view of the user;

a processor; and

a memory which stores a plurality of instructions, which when executed by the processor, cause the processor to,

recognize the recognition-target object included in the image data by extracting a feature point from the image data and associating the extracted feature point and the feature point of the recognition-target object stored in the storage unit with each other;

calculate a location change amount of the feature point corresponding to the recognition-target object recognized by the recognizing from a plurality of the image data obtained at different times and calculate a motion vector of the recognition-target object from the location change amount; and

determine a movement of the user based on the motion vector.