Image processing device, image processing method

• US 9,128,529 B2
• Filed: 12/17/2013
• Issued: 09/08/2015
• Est. Priority Date: 02/21/2013
• Status: Active Grant

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Hara, Nobuyuki
• Primary Examiner(s): Liang, Regina
• Application Number:
  US14/108,912
• Publication Number:
  US 20140232636A1
• Time in Patent Office: 630 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

An image processing device includes, a processor; and a memory which stores a plurality of instructions, which when executed by the processor, cause the processor to execute, acquiring an image including a subject recognized and a movement site of a user; recognizing the subject recognized and the movement site from the image; and controlling a display location, in the image, of an additional information image corresponding to the subject recognized, to a location other than the subject recognized and the movement site.

Claims

1. An image processing device comprising:

a processor; and

a memory which stores an instruction, which when executed by the processor, cause the processor to execute;

acquiring an image including a subject recognized and a movement site of a user;

recognizing the subject recognized and the movement site from the image;

controlling a display location, in the image, of an additional information image corresponding to the subject recognized, to a location other than the subject recognized and the movement site

calculating a movement vector of the movement site from a plurality of the images, and

wherein, in the controlling, the display location of the additional information image is controlled based on the location of the subject recognized and the movement vector.

2. The device according to claim 1,

wherein, in the recognizing, an access is made to a storage unit configured to store a first feature point that corresponds to the subject recognized and the first feature point has been extracted in advance, and

wherein a second feature point is extracted from the image, and, the recognizing of the subject is implemented by associating the first feature point and the second feature point.

3. The device according to claim 1,

wherein, in the controlling,

when the movement vector is equal to or greater than a predetermined first threshold value, the display location of the additional information image is controlled to the location other than the subject recognized and the movement site,

when the movement vector is less than the first threshold value, the display location of the additional information image is controlled to be superimposed on the subject recognized.

4. The device according to claim 3,

wherein, in the controlling, the display location of the additional information image is controlled in a direction of the movement vector, and in a direction in which an interior angle difference of directions normal to each side of an approximated rectangular region which is a rectangular approximation of the subject recognized is less than a predetermined angle.

5. The device according to claim 3,

wherein the movement vector has components for a movement amount and a movement direction of the movement site, and the first threshold value has a predetermined third threshold value for the movement amount, and a predetermined fourth threshold value for the movement direction.

6. The device according to claim 1,

wherein, in the controlling, when the movement vector is equal to or greater than a predetermined first threshold value, the display location of the additional information image is controlled

to a location where the subject recognized is between the movement vector and the additional information image, and

to a location where a distance between a reference location of the subject recognized and a reference location of the additional information image is a shortest distance.

7. The device according to claim 6,

wherein, in the controlling, if the shortest distance is equal to or greater than a predetermined second threshold value, a predetermined region is selected from the additional information image in such a way that the shortest distance is less than the second threshold value, and the predetermined region is displayed.

8. The device according to claim 7,

wherein the predetermined region includes a work location associated with the subject recognized.

9. The device according to claim 6, further comprising:

a display unit configured to display the additional information image,

wherein, in the controlling, when a part of the additional information image is outside of a display region of the display unit, a predetermined region that is within the display region of the display unit is selected from the additional information image, and the display region is displayed.

10. The device according to claim 1, further comprising capturing an image including the subject recognized and the movement site of the user, wherein, in the acquiring, the image is acquired by the capturing.

11. An image processing method comprising:

acquiring an image including a subject recognized and a movement site of a user;

recognizing, by a computer processor, the subject recognized and the movement site from the image;

controlling a display location, in the image, of an additional information image corresponding to the subject recognized, to a location other than the subject recognized and the movement site; and

calculating a movement vector of the movement site from a plurality of the images, and

wherein, in the controlling, the display location of the additional information image is controlled based on the location of the subject recognized and the movement vector.

12. The method according to claim 11,

wherein, in the recognizing, an access is made to a storage unit configured to store a first feature point that corresponds to the subject recognized and the first feature point has been extracted in advance, and

wherein a second feature point is extracted from the image, and, the recognizing of the subject is implemented by associating the first feature point and the second feature point.

13. The method according to claim 11,

wherein, in the controlling,

when the movement vector is equal to or greater than a predetermined first threshold value, the display location of the additional information image is controlled to the location other than the subject recognized and the movement site,

when the movement vector is less than the first threshold value, the display location of the additional information image is controlled to be superimposed on the subject recognized.

14. The method according to claim 11,

wherein, in the controlling, when the movement vector is equal to or greater than a predetermined first threshold value, the display location of the additional information image is controlled

to a location where the subject recognized is between the movement vector and the additional information image, and

to a location where a distance between a reference location of the subject recognized and a reference location of the additional information image is a shortest distance.

15. The method according to claim 14,

wherein, in the controlling, if the shortest distance is equal to or greater than a predetermined second threshold value, a predetermined region is selected from the additional information image in such a way that the shortest distance is less than the second threshold value, and the predetermined region is displayed.

16. The method according to claim 15,

wherein the predetermined region includes a work location associated with the subject recognized.

17. The method according to claim 14, further comprising:

displaying the additional information image,

wherein, in the controlling, when a part of the additional information image is outside of a display region of the displaying, a predetermined region that is within the display region of the displaying is selected from the additional information image, and the display region is displayed.

18. A non-transitory computer-readable storage medium storing an image processing program that causes a computer to execute a process comprising:

acquiring an image including a subject recognized and a movement site of a user;

recognizing the subject recognized and the movement site from the image;

controlling a display location, in the image, of an additional information image corresponding to the subject recognized, to a location other than the subject recognized and the movement site,

calculating a movement vector of the movement site from a plurality of the images, and

wherein, in the controlling, the display location of the additional information image is controlled based on the location of the subject recognized and the movement vector.