SHARING METHOD AND DEVICE FOR VIDEO AND AUDIO DATA PRESENTED IN INTERACTING FASHION

• US 20180286421A1
• Filed: 03/31/2017
• Published: 10/04/2018
• Est. Priority Date: 03/31/2017
• Status: Active Grant

• Current Assignee: Nanning Fugui Precision Industrial Co., Ltd.
• Original Assignee: Hong Fu Jin Precision Industry (Shenzhen) Co., Ltd. (Hon Hai Precision Industry Co., Ltd.), Hon Hai Precision Industry Co., Ltd.
• Inventors: YU, FEI-FAN
• Granted Patent: US 10,186,275 B2
• Time in Patent Office: Days
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/088 Word spotting
• CPC Class Codes: G10L 25/57 for processing of video signals
• CPC Class Codes: G11B 20/10527 Audio or video recording; Data buffering arrangements G11B20/12 - G11B20/18 take precedence
• CPC Class Codes: G11B 2020/10546 specifically adapted for audio data
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/8106 involving special audio data, e.g. different tracks for different languages
• CPC Class Codes: H04N 21/8405 represented by keywords

Abstract

A sharing method for video and audio data presented in an interacting manner applies to story-telling. The story from a book or other source includes a plurality of preset sentences, each having a plurality of keywords and which correspond to a preset video frame. The keywords represent roles and non-roles, and each of the roles corresponds to an image. Voice sources of the roles of the story are set according to a user instruction. Audio data of storytelling is matched with the keywords of the preset sentences, wherein the audio data is acquired from the voice sources. A matched sentence is acquired from the preset sentences. The preset video frame corresponding to the matched sentence is selected and displayed for vividness of presentation.

Claims

1. A computer-implemented method, executable by a processor of a sharing device for interactive video and audio data, the method comprising:

storing a story in a database, wherein the story comprises a plurality of preset sentences, each of the preset sentences comprises a plurality of keywords and corresponds to a preset video frame, the keywords represent roles and non-roles, and each of the roles is corresponding to an image;

setting voice sources of the roles of the story according to an operating instruction triggered by a user input;

matching audio data of storytelling with the keywords of the preset sentences, wherein the audio data is acquired from the voice sources;

acquiring a matched sentence from the preset sentences; and

selecting the preset video frame corresponding to the matched sentence from the database and displaying the preset video frame by a display;

determining one of the preset sentences as the matched sentence when an audio sentence of the audio data has more than half of the keywords of the preset sentence;

recording a group of similar words corresponding to each of the keywords, wherein the similar words in the group are sorted according to usage count of the similar words; and

adding a new word into the group when the usage count of that word accumulates to a preset count.

2. The method as claimed in claim 1, wherein the roles comprise user-defined roles, and the images of the user-defined roles are user-defined images uploaded by the user.

3. The method as claimed in claim 1, wherein the voice sources comprise a voice of the user, a voice of other user invited online, a preset voice of the role, and an audio file shared from social groups.

6. The method as claimed in claim 1, wherein the similar words of the group are determined to be as the same as the corresponding keyword in the preset sentences.

7. The method as claimed in claim 1, further comprising:

distinguishing a replacement role from the audio data when the user requests replacement of a role;

acquiring the image corresponding to the replacement role from the database;

updating foreground images of corresponding preset video frame according to the image; and

creating an updated video frame.

8. The method as claimed in claim 7, further comprising:

determining one of the preset sentences as the matched sentence when the audio sentence of the audio data has more than half of the keywords of the preset sentence, and determining that a word in the audio sentence of the audio data which differs from each of the keywords as the replacement role.

9. The method as claimed in claim 1, further comprising:

when the user requests recording the audio data and more than one audio sentence of the audio data is matched with a preset sentence, recording a first sentence which has a largest number of matched keywords among the audio sentences; and

sorting the first sentence according to a sequence of the corresponding matched preset sentence in the story.

4. (canceled)

5. (canceled)

10. A non-transitory storage medium, storing a set of instructions, the set of instructions being executed by a processor of a sharing device for interactive video and audio data, to perform a method comprising:

storing a story in a database, wherein the story comprises a plurality of preset sentences, each of the preset sentences comprises a plurality of keywords and corresponds to a preset video frame, the keywords represent roles and non-roles, and each of the roles is corresponding to an image;

setting voice sources of the roles of the story according to an operating instruction triggered by a user input;

matching audio data of storytelling with the keywords of the preset sentences, wherein the audio data is acquired from the voice sources;

acquiring a matched sentence from the preset sentences; and

selecting the preset video frame corresponding to the matched sentence from the database and displaying the preset video frame by a display;

determining one of the preset sentences as the matched sentence when an audio sentence of the audio data has more than half of the keywords of the preset sentence;

recording a group of similar words corresponding to each of the keywords, wherein the similar words in the group are sorted according to usage count of the similar words; and

adding a new word into the group when the usage count of that word accumulates to a preset count.

11. The non-transitory storage medium as claimed in claim 10, wherein the roles comprise user-defined roles, and the images of the user-defined roles are user-defined images uploaded by the user.

12. The non-transitory storage medium as claimed in claim 10, wherein the voice sources comprise a voice of the user, a voice of other user invited online, a preset voice of the role, and an audio file shared from social groups.

15. The non-transitory storage medium as claimed in claim 10, wherein the similar words of the group are determined to be as the same as the corresponding keyword in the preset sentences.

16. The non-transitory storage medium as claimed in claim 10, wherein the method further comprises:

distinguishing a replacement role from the audio data when the user requests replacement of a role;

acquiring the image corresponding to the replacement role from the database;

updating foreground images of corresponding preset video frame according to the image; and

creating an updated video frame.

17. The non-transitory storage medium as claimed in claim 16, wherein the method further comprises:

determining one of the preset sentences as the matched sentence when the audio sentence of the audio data has more than half of the keywords of the preset sentence, and determining that a word in the audio sentence of the audio data which differs from each of the keywords as the replacement role.

18. The non-transitory storage medium as claimed in claim 10, wherein the method further comprises:

when the user requests recording the audio data and more than one audio sentence of the audio data is matched with a preset sentence, recording a first sentence which has a largest number of matched keywords among the audio sentences; and

sorting the first sentence according to a sequence of the corresponding matched preset sentence in the story.

13. (canceled)

14. (canceled)

19. A sharing device for interactive video and audio data, the device comprising:

a processor;

memory storing instructions,

wherein the instructions are executed on the processor to cause the processor to;

storing a story in a database, wherein the story comprises a plurality of preset sentences, each of the preset sentences comprises a plurality of keywords and corresponds to a preset video frame, the keywords represent roles and non-roles, and each of the roles is corresponding to an image;

set voice sources of the roles of the story according to an operating instruction triggered by a user input;

match audio data of storytelling with the keywords of the preset sentences, wherein the audio data is acquired from the voice sources;

acquire a matched sentence from the preset sentences; and

select the preset video frame corresponding to the matched sentence from the database and displaying the preset video frame by a display;

determining one of the preset sentences as the matched sentence when an audio sentence of the audio data has more than half of the keywords of the preset sentence;

recording a group of similar words corresponding to each of the keywords, wherein the similar words in the group are sorted according to usage count of the similar words; and

adding a new word into the group when the usage count of that word accumulates to a preset count.

20. The device as claimed in claim 19, wherein the roles comprise user-defined roles, and the images of the user-defined roles are user-defined images uploaded by the user.

21. The device as claimed in claim 19, wherein the voice sources comprise a voice of the user, a voice of other user invited online, a preset voice of the role, and an audio file shared from social groups.

24. The device as claimed in claim 19, wherein the similar words of the group are determined to be as the same as the corresponding keyword in the preset sentences.

25. The device as claimed in claim 19, wherein the instructions executed on the processor further cause the processor to:

distinguish a replacement role from the audio data when the user requests replacement of a role;

acquire the image corresponding to the replacement role from the database;

update foreground images of corresponding preset video frame according to the image; and

create an updated video frame.

26. The device as claimed in claim 25, wherein the instructions executed on the processor further cause the processor to:

determine one of the preset sentences as the matched sentence when the audio sentence of the audio data has more than half of the keywords of the preset sentence, and determine that a word in the audio sentence of the audio data which differs from each of the keywords as the replacement role.

27. The device as claimed in claim 19, wherein the instructions executed on the processor further cause the processor to:

when the user requests recording the audio data and more than one audio sentence of the audio data is matched with a preset sentence, record a first sentence which has a largest number of matched keywords among the audio sentences; and

sort the first sentence according to a sequence of the corresponding matched preset sentence in the story.

22. (canceled)

23. (canceled)