PICTURE DRAWING SUPPORT APPARATUS AND METHOD

• US 20140289632A1
• Filed: 03/04/2014
• Published: 09/25/2014
• Est. Priority Date: 03/21/2013
• Status: Abandoned Application

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Suzuki, Masaru, Okamoto, Masayuki, Cho, Kenta, Fume, Kosei
• Application Number:
  US14/196,435
• Publication Number:
  US 20140289632A1
• Time in Patent Office: Days
• US Class Current: 715/728
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/70 Labelling scene content, e.g. deriving syntactic or semantic representations

Abstract

According to an embodiment, a picture drawing support apparatus includes following components. The feature extractor extracts a feature amount from a picture drawn by a user. The speech recognition unit performs speech recognition on speech input by the user. The keyword extractor extracts at least one keyword from a result of the speech recognition. The image search unit retrieves one or more images corresponding to the at least one keyword from a plurality of images prepared in advance. The image selector selects an image which matches the picture, from the one or more images based on the feature amount. The image deformation unit deforms the image based on the feature amount to generate an output image. The presentation unit presents the output image.

Claims

1. A picture drawing support apparatus comprising:

a feature extractor configured to extract a feature amount from a picture drawn by a user;

a speech recognition unit configured to perform speech recognition on speech input by the user;

a keyword extractor configured to extract at least one keyword from a result of the speech recognition;

an image search unit configured to retrieve one or more images corresponding to the at least one keyword from a plurality of images prepared in advance;

an image selector configured to select an image which matches the picture, from the one or more images based on the feature amount;

an image deformation unit configured to deform the selected image based on the feature amount to generate an output image; and

a presentation unit configured to present the output image.

2. The apparatus according to claim 1, wherein the image selector calculates degrees of similarity between the picture and the one or more images based on the feature amount, and selects an image which matches the picture, based on comparisons between the degrees of similarity and a predetermined threshold.

3. The apparatus according to claim 2, wherein when the keyword extractor extracts a plurality of keywords and the image selector determines based on the comparisons that the one or more images do not include an image which matches the picture, the image search unit retrieves a plurality of images corresponding to the plurality of keywords, one or more images for each keyword, the image selector selects images which match parts of the picture from the plurality of images, and the image deformation unit combines the selected images.

4. The apparatus according to claim 2, wherein when the picture is a simple figure and the image selector determines based on the comparison that the one or more images do not include an image which matches the picture, the image selector selects an image having a highest degree of similarity from the one or more images, and the image deformation unit deforms the selected image based on a size and a position of the picture.

5. The apparatus according to claim 2, wherein the feature extractor extracts other feature amounts from the one or more images, and calculates the degrees of similarity based on the feature amount and the other feature amounts.

6. The apparatus according to claim 1, wherein when the keyword extractor extracts a plurality of keywords, the image deformation unit generates a plurality of deformed images by deforming a plurality of images which are selected respectively for the plurality of keywords, and generates an output image by combining the plurality of deformed images.

7. The apparatus according to claim 6, wherein the keyword extractor acquires relation information indicating a modification relation in the result of the speech recognition, and the image deformation unit controls a combination method of the plurality of deformed images in accordance with the relation information.

8. The apparatus according to claim 7, wherein the relation information includes a modification relation between the keyword and a modifying word, which modifies the keyword.

9. A picture drawing support method comprising:

extracting a feature amount from a picture drawn by a user;

performing speech recognition on speech input by the user;

extracting at least one keyword from a result of the speech recognition;

retrieving one or more images corresponding to the at least one keyword from a plurality of images prepared in advance;

selecting an image which matches the picture, from the one or more images based on the feature amount;

deforming the image based on the feature amount to generate an output image; and

presenting the output image.

10. The method according to claim 9, wherein the selecting comprises calculating degrees of similarity between the picture and the one or more images based on the feature amount, and selecting an image which matches the picture, based on comparisons between the degrees of similarity and a predetermined threshold.

11. The method according to claim 10, wherein when the at least one keyword includes a plurality of keywords and it is determined based on the comparisons that the one or more images do not include an image which matches the picture, the retrieving comprises retrieving a plurality of images corresponding to the plurality of keywords, one or more images for each keyword, the selecting comprises selecting images which match parts of the picture from the plurality of images, and the deforming comprising combining the selected images.

12. The method according to claim 10, wherein when the picture is a simple figure and it is determined based on the comparison that the one or more images do not include an image which matches the picture, the selecting comprises selecting an image having a highest degree of similarity from the one or more images, and the deforming comprises deforming the selected image based on a size and a position of the picture.

13. The method according to claim 10, further comprising extracting other feature amounts from the one or more images, wherein the calculating the degrees of similarity is based on the feature amount and the other feature amounts.

14. The method according to claim 9, wherein when the at least one keyword comprises a plurality of keywords, the deforming comprises generating a plurality of deformed images by deforming a plurality of images which are selected respectively for the plurality of keywords, and generating an output image by combining the plurality of deformed images.

15. The method according to claim 14, further comprising acquiring relation information indicating a modification relation in the result of the speech recognition, wherein the deforming comprises controlling a combination method of the plurality of deformed images in accordance with the relation information.

16. The method according to claim 15, wherein the relation information includes a modification relation between the keyword and a modifying word, which modifies the keyword.

17. A non-transitory computer readable medium including computer executable instructions, wherein the instructions, when executed by a processor, cause the processor to perform a method comprising:

extracting a feature amount from a picture drawn by a user;

performing speech recognition on speech input by the user;

extracting at least one keyword from a result of the speech recognition;

retrieving one or more images corresponding to the at least one keyword from a plurality of images prepared in advance;

selecting an image which matches the picture, from the one or more images based on the feature amount;

deforming the image based on the feature amount to generate an output image; and

presenting the output image.