Information processing apparatus and phrase output method for determining phrases based on an image

• US 9,361,316 B2
• Filed: 07/18/2012
• Issued: 06/07/2016
• Est. Priority Date: 07/26/2011
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Rekimoto, Junichi, Nashida, Tatsushi
• Primary Examiner(s): Lee, Michael
• Application Number:
  US13/551,908
• Publication Number:
  US 20130027535A1
• Time in Patent Office: 1,420 Days
• Field of Search: 348/589, 348/600, 348/333.02, 348/77, 707/621, 707/748, 707/749, 434/94, 434 98-100, 345/629, 345/636, 704/270, 704/272, 704/276, 715/230, 715/254, 463/31
• US Class Current: 1/1
• CPC Class Codes: G06F 16/2452 Query translation
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/638 Presentation of query results
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 40/274 Converting codes to words; Guess-ahead of partial word inputs
• CPC Class Codes: G06F 40/56 Natural language generation
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

In one aspect, an information processing system for determining complimentary phrases to present to a recipient is provided. The information processing system may receive an input image depicting a subject. A difference in appearance between the subject depicted in the input image and a model subject may be determined. One or more phrases (e.g., complimentary phrases) may be selected from a plurality of categorized phrase candidates based on the identified difference in appearance between the subject and the model subject. The difference in appearance may be determined by comparing selected features (e.g., skin, hair, eyes) of the subject with one or more predetermined features of the model subject. The model subject may be the same or different than the subject depicted in the input image.

Claims

1. An information processing system comprising:

one or more processors configured to;

receive an input image captured by an image sensor;

analyze the input image to identify a difference between one or more physical appearance features of a subject depicted in the input image and one or more corresponding physical appearance features of a model subject;

select one or more phrases from a plurality of phrase candidates based on the identified difference between the one or more physical appearance features of the subject and the one or more corresponding physical appearance features of the model subject;

detect a location of the information processing system; and

output, to an output device, at least one of the one or more selected phrases using a particular dialect associated with the detected location, wherein the particular dialect is in formal language of the particular dialect in an event the detected location corresponds to a public space in a reference map data.

2. The information processing system of claim 1, wherein the one or more processors are further configured to identify the difference between the subject depicted in the input image and the model subject by comparing the one or more physical appearance features of the subject extracted from the analysis of the input image with one or more predetermined physical appearance features of the model subject.

3. The information processing system of claim 1, wherein the one or more processors are further configured to select the one or more phrases from the plurality of phrase candidates based on a determined category of the identified difference between the one or more physical appearance features of the subject depicted in the input image and the one or more corresponding physical appearance features of the model subject.

4. The information processing system of claim 1, wherein the one or more processors are further configured to select the model subject based on a prior image or images of the subject depicted in the input image.

5. The information processing system of claim 1, wherein the one or more processors are further configured to select the model subject based on an image or images of one or more average subjects, the one or more average subjects being different from the subject depicted in the input image.

6. The information processing system of claim 1, wherein the one or more processors are further configured to:

display the input image depicting the subject to a user on a display device;

determine a physical appearance feature of the subject selected by the user; and

display, on the display device, one or more phrase candidates from the plurality of phrase candidates for further selection by the user, the one or more phrase candidates being displayed for further selection by the user based on the physical appearance feature of the subject selected by the user.

7. The information processing system of claim 1, wherein the one or more processors are further configured to exclude a given phrase from the one or more phrases selected from the plurality of phrase candidates, the given phrase being excluded based upon a determination by the one or more processors that the given phrase was previously or recently output to the output device.

8. The information processing system of claim 1, wherein the one or more processors are further configured to control a timing of the output of the at least one of the one or more selected phrases to the output device based on a condition of the subject determined from the analysis of the input image.

9. The information processing system of claim 8, wherein the condition of the subject determined from the analysis of the input image includes a specific expression, posture, or gesture of the subject depicted in the input image.

10. The information processing system of claim 1, wherein the one or more processors are further configured to determine a degree of the identified difference between the one or more physical appearance features of the subject and the one or more corresponding physical appearance features of the model subject.

11. The information processing system of claim 10, wherein the one or more processors are further configured to select the one or more phrases from the plurality of phrase candidates when the determined degree of the identified difference exceeds a predetermined threshold.

12. The information processing system of claim 1, wherein the model subject is selected from one of a plurality of selectable model subjects based on input received from a user.

13. The information processing system of claim 1, wherein the one or more processors are configured to output audio of the at least one of the one or more selected phrases to the output device.

16. The information processing system of claim 1, wherein the one or more physical appearance features comprise one or more facial features of the subject.

17. The information processing system of claim 16, wherein the one or more physical appearance features comprise one or more body parts of the subject or an expression of the subject.

18. The information processing system of claim 1, wherein the one or more physical appearance features comprise one or more of a hairstyle of the subject, clothing of the subject or accessories worn by the subject.

19. The information processing system of claim 1, wherein the one or more processors are configured to identify the difference between one or more demographic attributes of the subject depicted in the input image and one or more corresponding demographic attributes of the model subject.

20. The information processing system of claim 19, wherein the one or more attributes comprise gender of the subject or age group of the subject.

21. The information processing system of claim 19, wherein the one or more processors are configured to select the one or more phrases based on a difference between a combination of the one or more physical appearance features and the one or more demographic attributes of the subject and a combination of the one or more corresponding physical appearance features and the one or more corresponding demographic attributes of the model subject.

14. A method comprising:

in an information processing apparatus;

receiving an input image captured by an image sensor;

analyzing, with a processor, the input image to identify a difference between one or more physical appearance features of a subject depicted in the input image and one or more corresponding physical appearance features of a model subject;

selecting one or more phrases from a plurality of phrase candidates based on the identified difference between the one or more physical appearance features of the subject and the one or more corresponding physical appearance features of the model subject;

detecting a location of the information processing apparatus; and

outputting, to an output device, at least one of the one or more selected phrases using a particular dialect associated with the detected location, wherein the particular dialect is in formal language of the particular dialect in an event the detected location corresponds to a public space in a reference map data.

15. A non-transitory computer-readable storage medium having computer-executable instructions stored thereon that, when executed by one or more processors of an information processing apparatus, cause the one or more processors to:

receive an input image captured by an image sensor;

analyze the input image to identify a difference between one or more physical appearance features of a subject depicted in the input image and one or more corresponding physical appearance features of a model subject;

select one or more phrases from a plurality of phrase candidates based on the identified difference between the one or more physical appearance features of the subject and the one or more corresponding physical appearance features of the model subject;

detect a location of the information processing apparatus; and

output, to an output device, at least one of the one or more selected phrases using a particular dialect associated with the detected location, wherein the particular dialect is in formal language of the particular dialect in an event the detected location corresponds to a public space in a reference map data.

22. An information processing apparatus comprising:

one or more processors configured to;

receive an input image from another information processing apparatus;

analyze the input image to identify a difference between one or more physical appearance features of a subject depicted in the input image and one or more corresponding physical appearance features of a model subject;

select one or more phrases from a plurality of phrase candidates based on the identified difference between the one or more physical appearance features of the subject and the one or more corresponding physical appearance features of the model subject;

detect a location of the information processing apparatus;

transmit the one or more selected phrases to the other information processing apparatus using a particular dialect associated with the detected location, wherein the particular dialect is in formal language of the particular dialect in an event the detected location corresponds to a public space in a reference map data.