Amusement system

• US 9,601,118 B2
• Filed: 10/06/2011
• Issued: 03/21/2017
• Est. Priority Date: 10/20/2010
• Status: Expired due to Fees

• Current Assignee: Megachips Corporation
• Original Assignee: Megachips Corporation
• Inventors: Tanaka, Motoyasu
• Primary Examiner(s): ADESANYA, OLUJIMI A
• Application Number:
  US13/877,029
• Publication Number:
  US 20130185069A1
• Time in Patent Office: 1,993 Days
• Field of Search: 704/236, 704/260, 704/270
• US Class Current: 1/1
• CPC Class Codes: A63F 2300/6063 for sound processing
• CPC Class Codes: G10H 1/0083 using wireless transmission, e.g. radio, light, infrared
• CPC Class Codes: G10H 1/368 displaying animated or moving pictures synchronized with the music or audio part
• CPC Class Codes: G10H 2210/091 for performance evaluation, i.e. judging, grading or scoring the musical qualities or faithfulness of a performance, e.g. with respect to pitch, tempo or other timings of a reference performance
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/28 Constructional details of speech recognition systems

Abstract

A technique for allowing a virtual experience of more realistic live performance. A main apparatus reproduces music data and audience video data recording a video image of audience. A user holds a microphone and makes a live performance for the audience displayed on a monitor. The microphone sends voice data and motion information of the microphone to the main apparatus. The main apparatus determines that the user makes a live performance when the user calls on the audience with a specific phrase and performs an action corresponding to the specific phrase. The main apparatus reproduces reaction data recording a video image and sound indicating a reaction of the audience to the live performance.

Claims

1. An amusement system, comprising:

a main apparatus;

a voice input device held by a user, for outputting voice inputted by the user as voice data; and

a voice recognition device for performing a voice recognition process on the voice data, to thereby generate phrase information indicating a phrase that the user says, wherein

the voice input device includes

a first motion information output part for outputting first motion information indicating a motion of the voice input device, and

the main apparatus includes

a reproduction part for reproducing music data selected by the user and audience condition data recording at least one of a video image and sound which indicate the condition of an audience;

a performance specifying part for specifying a performance of the user on the basis of at least one of the phrase information and the first motion information; and

a reaction instruction part for determining a reaction level of the audience for the specified performance on the basis of a time difference between a timing when the phrase information is detected and a timing when the first motion information is detected, selecting reproduced reaction data corresponding to the specified performance and the determined reaction level out of a plurality of pieces of reaction data each indicating a reaction of the audience and instructing the reproduction part to reproduce the reproduced reaction data.

2. The amusement system according to claim 1, wherein

the reaction instruction part determines a reproduction condition of the reproduced reaction data on the basis of the reaction level of the audience and instructs the reproduction part of the reproduction condition.

3. The amusement system according to claim 1, wherein

the reproduction part reproduces the audience condition data corresponding to an audience group set by the user, and

the reaction instruction part determines the reaction level of the audience on the basis of the audience group.

4. The amusement system according to claim 1, wherein

the performance specifying part determines that the user makes a performance for exciting the audience when the performance specifying part determines, on the basis of the phrase information, that the user calls on the audience with a specific phrase in a question form and determines, on the basis of the first motion information, that a direction of the voice input device is reversed, and

the reaction instruction part selects reaction data recording a video image and sound of the audience who answers the specific phrase in unison, as the reproduced reaction data.

5. The amusement system according to claim 1, wherein

the performance specifying part determines that the user makes a performance for requesting the audience to sing along when the performance specifying part determines, on the basis of the phrase information, that the user calls on the audience with a specific phrase requesting the audience to sing along and determines, on the basis of the first motion information, that a direction of the voice input device is reversed, and

the reaction instruction part selects reaction data recording a video image and sound of the audience who sings along, as the reproduced reaction data.

6. The amusement system according to claim 1, wherein

the performance specifying part determines that the user makes a performance for leading the audience to clap when the performance specifying part determines, on the basis of the motion information, that the user claps, and

the reaction instruction part selects reaction data recording a video image and sound of the audience who claps, as the reproduced reaction data.

7. The amusement system according to claim 1, wherein

the performance specifying part determines that the user makes a performance for requesting the audience to wave both arms when the performance specifying part determines, on the basis of the motion information, that the user waves both arms, and

the reaction instruction part selects reaction data recording a video image and sound of the audience who waves both arms, as the reproduced reaction data.

8. The amusement system according to claim 1, further comprising:

a controller held by the user in one hand, with the other hand holding the voice input device,

wherein the controller includes

a second motion information output part for outputting second motion information indicating a motion of the controller, and

the performance specifying part specifies a performance of the user on the basis of the second motion information.

9. The amusement system according to claim 1, wherein

the reaction instruction part scores each of performances made in a first part of the music data on the basis of the reaction level of the audience for the performance, and scores each of performances made in a second part of the music data on the basis of the reaction level of the audience for the performance.

10. A voice input device used in the amusement system as defined in claim 1.

11. An amusement system, comprising:

a main apparatus;

a voice input device held by a user, for outputting voice inputted by the user as voice data;

an image pickup device for picking up an image of the user to output video data; and

a voice recognition device for performing a voice recognition process on the voice data, to thereby generate phrase information indicating a phrase that the user says, wherein

the main apparatus includes

a reproduction part for reproducing music data selected by the user and audience condition data recording at least one of a video image and sound which indicate the condition of an audience;

a video analysis part for analyzing the video data to generate first motion information indicating a motion of the user;

a performance specifying part for specifying a performance of the user on the basis of at least one of the phrase information and the first motion information; and

a reaction instruction part for determining a reaction level of the audience for the specified performance on the basis of a time difference between a timing when the phrase information is detected and a timing when the first motion information is detected, selecting reproduced reaction data corresponding to the specified performance and the determined reaction level out of a plurality of pieces of reaction data each indicating a reaction of the audience and instructing the reproduction part to reproduce the reproduced reaction data.

12. The amusement system according to claim 11, wherein

the reaction instruction part determines a reproduction condition of the reproduced reaction data on the basis of the reaction level of the audience and instructs the reproduction part of the reproduction condition.

13. The amusement system according to claim 11, wherein

the reproduction part reproduces the audience condition data corresponding to an audience group set by the user, and

the reaction instruction part determines the reaction level of the audience on the basis of the audience group.

14. The amusement system according to claim 11, wherein

the performance specifying part determines that the user makes a performance for exciting the audience when the performance specifying part determines, on the basis of the phrase information, that the user calls on the audience with a specific phrase in a question form and determines, on the basis of the first motion information, that a direction of the voice input device is reversed, and

the reaction instruction part selects reaction data recording a video image and sound of the audience who answers the specific phrase in unison, as the reproduced reaction data.

15. The amusement system according to claim 11, wherein

the performance specifying part determines that the user makes a performance for requesting the audience to sing along when the performance specifying part determines, on the basis of the phrase information, that the user calls on the audience with a specific phrase requesting the audience to sing along and determines, on the basis of the first motion information, that a direction of the voice input device is reversed, and

the reaction instruction part selects reaction data recording a video image and sound of the audience who sings along, as the reproduced reaction data.

16. The amusement system according to claim 11, wherein

the performance specifying part determines that the user makes a performance for leading the audience to clap when the performance specifying part determines, on the basis of the motion information, that the user claps, and

the reaction instruction part selects reaction data recording a video image and sound of the audience who claps, as the reproduced reaction data.

17. The amusement system according to claim 11, wherein

the performance specifying part determines that the user makes a performance for requesting the audience to wave both arms when the performance specifying part determines, on the basis of the motion information, that the user waves both arms, and

the reaction instruction part selects reaction data recording a video image and sound of the audience who waves both arms, as the reproduced reaction data.

18. The amusement system according to claim 11, wherein

the reaction instruction part scores each of performances made in a first part of the music data on the basis of the reaction level of the audience for the performance, and scores each of performances made in a second part of the music data on the basis of the reaction level of the audience for the performance.

19. A voice input device used in the amusement system as defined in claim 11.

20. A non-transitory computer readable medium having a program used in an amusement system comprising:

a main apparatus;

a voice input device held by a user, for outputting voice inputted by the user as voice data and outputting first motion information indicating a motion of the voice input device; and

a voice recognition device for performing a voice recognition process on the voice data, to thereby generate phrase information indicating a phrase that the user says, the program executed by a computer incorporated in the main apparatus to cause the computer to perform a method, the method comprising;

reproducing music data selected by the user and audience condition data recording at least one of a video image and sound which indicate the condition of an audience;

specifying a performance of the user on the basis of at least one of the phrase information and the first motion information;

determining a reaction level of the audience for the specified performance on the basis of a time difference between a timing when the phrase information is detected and a timing when the first motion information is detected; and

selecting reproduced reaction data corresponding to the specified performance and the determined reaction level out of a plurality of pieces of reaction data each indicating a reaction of the audience and instructing reproduction of the reproduced reaction data.

21. A non-transitory computer readable medium having a program used in an amusement system comprising:

a main apparatus;

a voice input device held by a user, for outputting voice inputted by the user as voice data;

an image pickup device for picking up an image of the user to output video data; and

a voice recognition device for performing a voice recognition process on the voice data, to thereby generate phrase information indicating a phrase that the user says, the program executed by a computer incorporated in the main apparatus to cause the computer to perform a method, the method comprising;

reproducing music data selected by the user and audience condition data recording at least one of a video image and sound which indicate the condition of an audience;

analyzing the video data to generate first motion information indicating a motion of the user;

specifying a performance of the user on the basis of at least one of the phrase information and the first motion information;

determining a reaction level of the audience for the specified performance on the basis of a time difference between a timing when the phrase information is detected and a timing when the first motion information is detected; and

selecting reproduced reaction data corresponding to the specified performance and the determined reaction level out of a plurality of pieces of reaction data each indicating a reaction of the audience and instructing reproduction of the reproduced reaction data.