Multimedia presentation method and apparatus

• US 9,953,221 B2
• Filed: 06/29/2016
• Issued: 04/24/2018
• Est. Priority Date: 11/26/2015
• Status: Active Grant

• Current Assignee: Xiaomi Inc. (Xiaomi Corp.)
• Original Assignee: Xiaomi Inc. (Xiaomi Corp.)
• Inventors: Chen, Zhijun, Hou, Wendi, Long, Fei
• Primary Examiner(s): Chu, Randolph I
• Application Number:
  US15/196,766
• Publication Number:
  US 20170154220A1
• Time in Patent Office: 664 Days
• Field of Search: None
• CPC Class Codes: G06F 16/54 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 16/638 Presentation of query results
• CPC Class Codes: G06F 16/686 using information manually generated, e.g. tags, keywords, comments, title or artist information, time, location or usage information, user ratings
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 40/1365 Matching; Classification
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

The present disclosure is related to a photo album based music selection method, apparatus and device, and a storage medium. When a certain photo album is viewed on a device, the corresponding music can be automatically selected and played based on a character relationship determined according to the method between the viewer and a person being viewed in a picture of the photo album, which decreases the complexity of operations of the user, and thereby improves the user experience.

Claims

1. A photo album based music selection method, comprising:

acquiring a viewing instruction for viewing a target photo album stored on a device, the target photo album including at least one picture;

acquiring feature information of a user including at least acquiring fingerprint information of the user;

determining a character relationship between the user and a person represented by image data of a picture included in the target photo album by;

comparing the fingerprint information with preset fingerprint information of a first user;

when the fingerprint information is determined to match with the fingerprint information of the first user based on the comparison, determining the user is the first user;

assigning a first category tag to the target photo album based on the first user; and

determining the character relationship based on the first category tag assigned to the target photo album;

determining a target music type corresponding to the character relationship based on a preset correspondence relationship between available character relationships and available music types;

presenting the at least one picture included in the target photo album; and

playing music corresponding to the target music type in response to a corresponding picture being presented.

2. The method according to claim 1, wherein determining the character relationship further comprises:

when the fingerprint information is determined not to match with the fingerprint information of the first user based on the comparison, determining the user is a second user;

assigning a second category tag to the target photo album based on the second user; and

determining the character relationship based on the second category tag assigned to the target photo album.

3. The method according to claim 2, further comprising determining a character relationship between the second user and the person represented by the image data of the picture included in the target photo album by:

acquiring a face image of the second user;

comparing the face image of the second user with second face images corresponding to persons represented by image data of pictures included in corresponding photo albums stored on the device;

when the face image of the second user is determined to match the second face images based on the comparison, determining the character relationship between the second user and the person represented by the image data of the picture included in the target photo album based on a category tag of the corresponding photo album that includes the match second face image and the category tag of the target photo album.

4. The method according to claim 1, wherein, acquiring the feature information of the user comprises:

acquiring a face image of the user;

determining the character relationship between the user and the person represented by the image data of the picture included in the target photo album based on the feature information of the user by;

comparing the face image with a preset face image of a first user;

when the face image is determined to match with the face image of the first user based on the comparison, determining the user is the first user, and determining the character relationship between the first user and the person represented by the image data of the picture included in the target photo album based on a face category tag assigned to the target photo album; and

when the face image is determined not to match with the face image of the first user based on the comparison, determining the user is a second user.

5. The method according to claim 4, further comprising determining the character relationship between the second user and the person represented by the image data of the picture included in the target photo album by:

comparing the face image of the second user with the face image corresponding to persons represented by image data of pictures included in corresponding photo albums stored on the device;

when the face image of the second user is determined to match the second face images based on the comparison, determining the character relationship between the second user and the person represented by the image data of the picture included in the target photo album based on a category tag of the matched photo album and the category tag of the target photo album.

6. The method according to claim 3, wherein, determining the character relationship between the second user and the person represented by the image data of the picture included in the target photo album based on the category tag of the matched photo album and the category tag of the target photo album comprises:

determining the character relationship between the first user and the second user based on the category tag of the matched photo album; and

determining the character relationship between the second user and the person represented by the image data of the picture included in the target photo album based on the character relationship between the first user, and the person represented by the image data of the picture included in the target photo album, and the character relationship between the first user and the second user.

7. The method according to claim 5, wherein, determining the character relationship between the second user and the person represented by the image data of the picture included in the target photo album based on the category tag of the matched photo album and the category tag of the target photo album comprises:

determining the character relationship between the first user and the second user based on the category tag of the matched photo album; and

determining the character relationship between the second user and the person represented by the image data of the picture included in the target photo album based on the character relationship between the first user and the person represented by the image data of the picture included in the target photo album, and the character relationship between the first user and the second user.

8. The method according to claim 3, further comprising:

when it is determined that there is no photo album that matches with the face image of the second user, acquiring character feature information of the second user;

determining the character relationship between the second user and the person represented by the image data of the picture included in the target photo album based on the character feature information and the character relationship between the first user and the person represented by the image data of the picture included in the target photo album, the character feature information comprising age group feature information or gender feature information.

9. The method according to claim 5, further comprising:

when it is determined that there is no photo album that matches with the face image of the second user, acquiring character feature information of the second user;

determining the character relationship between the second user and the person represented by the image data of the picture included in the target photo album based on the character feature information and the character relationship between the first user and the person represented by the image data of the picture included in the target photo album, the character feature information comprising age group feature information or gender feature information.

10. The method according to claim 1, further comprising:

sorting the at least one picture in the target photo album in accordance with a chronological order of photographing or storage.

11. A device, comprising:

a memory configured to store processor executable instructions;

a processor in communication with the memory; and

wherein, the processor is configured to execute the executable instructions to;

acquire a viewing instruction for viewing a target photo album stored on a device, wherein the target photo album includes at least one picture;

acquire feature information of a user including at least acquiring fingerprint information of the user;

determine a character relationship between the user and a person represented by image data of a picture included in the target photo album by;

comparing the fingerprint information with preset fingerprint information of a first user;

when the fingerprint information is determined to match the fingerprint information of the first user based on the comparison, determining the user is the first user;

assigning a first category tag to the target photo album based on the first user; and

determining the character relationship based on the first category tag assigned to the target photo album;

determine a target music type corresponding to the character relationship based on a preset correspondence relationship between available character relationships and available music types;

present the at least one picture included in the target photo album; and

play music corresponding to the target music type in response to a corresponding picture being presented.

12. The device according to claim 11, wherein, the processor is configured to execute the executable instructions to determine the character relationship by further:

when the fingerprint information is determined not to match with the fingerprint information of the first user based on the comparison, determining the user is a second user;

assigning a second category tag to the target photo album based on the second user; and

determining the character relationship based on the second category tag assigned to the target photo album.

13. The device according to claim 12, wherein, the processor is also configured to execute the executable instructions to:

acquire a face image of the second user;

compare the face image of the second user with second face images corresponding to persons represented by image data of pictures included in corresponding photo albums stored on the device;

when the face image of the second user is determined to match the second face images based on the comparison, determine the character relationship between the second user and the person represented by the image data of the picture included in the target photo album based on a category tag of the corresponding photo album that includes the match second face image and the category tag of the target photo album.

14. The device according to claim 11, wherein, the processor is also configured to execute the executable instructions to:

acquire a face image of the user;

determine the character relationship between the user and the person represented by the image data of the picture included in the target photo album based on the feature information of the user by;

compare the face image with a preset face image of a first user;

when the face image is determined to match with the face image of the first user based on the comparison, determine the user is the first user, and determining the character relationship between the first user and the person represented by the image data of the picture included in the target photo album based on a face category tag assigned to the target photo album; and

when the face image is determined not to match with the face image of the first user based on the comparison, determine the user is a second user.

15. The device according to claim 14, wherein, the processor is also configured to execute the executable instructions to:

comparing the face image of the second user with the face image corresponding to persons represented by image data of pictures included in corresponding photo albums stored on the device;

when the face image of the second user is determined to match the second face images based on the comparison, determining the character relationship between the second user and the person represented by the image data of the picture included in the target photo album based on a category tag of the matched photo album and the category tag of the target photo album.

16. The device according to claim 13, wherein, the processor is also configured to execute the executable instructions to:

determine the character relationship between the first user and the second user based on the category tag of the matched photo album; and

determine the character relationship between the second user and the person represented by the image data of the picture included in the target photo album based on the character relationship between the first user, and the person represented by the image data of the picture included in the target photo album, and the character relationship between the first user and the second user.

17. The device according to claim 15, wherein, the processor is also configured to execute the executable instructions to:

determine the character relationship between the first user and the second user based on the category tag of the matched photo album; and

determine the character relationship between the second user and the person represented by the image data of the picture included in the target photo album based on the character relationship between the first user and the person represented by the image data of the picture included in the target photo album, and the character relationship between the first user and the second user.

18. The device according to claim 13, wherein, the processor is also configured to execute the executable instructions to:

when it is determined that there is no photo album that matches with the face image of the second user, acquire character feature information of the second user;

determine the character relationship between the second user and the person represented by the image data of the picture included in the target photo album based on the character feature information and the character relationship between the first user and the person represented by the image data of the picture included in the target photo album, the character feature information comprising age group feature information or gender feature information.

19. The device according to claim 15, wherein, the processor is also configured to execute the executable instructions to:

when it is determined that there is no photo album that matches with the face image of the second user, acquire character feature information of the second user;

determine the character relationship between the second user and the person represented by the image data of the picture included in the target photo album based on the character feature information and the character relationship between the first user and the person represented by the image data of the picture included in the target photo album, the character feature information comprising age group feature information or gender feature information.

20. A non-transitory computer-readable storage medium having stored therein instructions that, when executed by a processor of a mobile device, causes the mobile device to perform a photo album based music selection method, the method comprising:

acquiring a viewing instruction for viewing a target photo album stored on a device, the target photo album including at least one picture;

acquiring feature information of a user including at least acquiring fingerprint information of the user;

determining a character relationship between the user and a person represented by image data of a picture included in the target photo album by;

comparing the fingerprint information with preset fingerprint information of a first user;

when the fingerprint information is determined to match with the fingerprint information of the first user based on the comparison, determining the user is the first user;

assigning a first category tag to the target photo album based on the first user; and

determining the character relationship based on the first category tag assigned to the target photo album;

determining a target music type corresponding to the character relationship based on a preset correspondence relationship between available character relationships and available music types;

presenting the at least one picture included in the target photo album; and

playing music corresponding to the target music type in response to a corresponding picture being presented.