AMUSEMENT SYSTEM

• US 20130185069A1
• Filed: 10/06/2011
• Published: 07/18/2013
• Est. Priority Date: 10/20/2010
• Status: Active Grant

• Current Assignee: Megachips Corporation
• Original Assignee: Megachips Corporation
• Inventors: Tanaka, Motoyasu
• Granted Patent: US 9,601,118 B2
• Time in Patent Office: Days
• US Class Current: 704/235
• CPC Class Codes: A63F 2300/6063 for sound processing
• CPC Class Codes: G10H 1/0083 using wireless transmission, e.g. radio, light, infrared
• CPC Class Codes: G10H 1/368 displaying animated or moving pictures synchronized with the music or audio part
• CPC Class Codes: G10H 2210/091 for performance evaluation, i.e. judging, grading or scoring the musical qualities or faithfulness of a performance, e.g. with respect to pitch, tempo or other timings of a reference performance
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/28 Constructional details of speech recognition systems

Abstract

A technique for allowing a virtual experience of more realistic live performance. A main apparatus reproduces music data and audience video data recording a video image of audience. A user holds a microphone and makes a live performance for the audience displayed on a monitor. The microphone sends voice data and motion information of the microphone to the main apparatus. The main apparatus determines that the user makes a live performance when the user calls on the audience with a specific phrase and performs an action corresponding to the specific phrase. The main apparatus reproduces reaction data recording a video image and sound indicating a reaction of the audience to the live performance.

Claims

1. An amusement system, comprising:

a main apparatus;

a voice input device held by a user, for outputting voice inputted by said user as voice data; and

a voice recognition device for performing a voice recognition process on said voice data, to thereby generate phrase information indicating a phrase that said user says,

wherein said voice input device includes

a first motion information output part for outputting first motion information indicating a motion of said voice input device, and

said main apparatus includes

a reproduction part for reproducing music data selected by said user and audience condition data recording at least one of a video image and sound which indicate the condition of audience;

a performance specifying part for specifying a performance of said user on the basis of at least one of said phrase information and said first motion information; and

a reaction instruction part for selecting reproduced reaction data corresponding to said specified performance out of a plurality of pieces of reaction data each indicating a reaction of said audience and instructing said reproduction part to reproduce said reproduced reaction data.

3. The amusement system according to claim 1, wherein

said reaction instruction part determines a reaction level of said audience for said specified performance on the basis of at least one of said phrase information and said first motion information and selects said reproduced reaction data on the basis of said reaction level of said audience.

4. The amusement system according to claim 3, wherein

said reaction instruction part determines a reproduction condition of said reproduced reaction data on the basis of said reaction level of said audience and instructs said reproduction part of said reproduction condition.

5. The amusement system according to claim 3, wherein

said reproduction part reproduces said audience condition data corresponding to an audience group set by said user, and

said reaction instruction part determines said reaction level of said audience on the basis of said audience group.

6. The amusement system according to claim 1, wherein

said performance specifying part determines that said user makes a performance for exciting said audience when said performance specifying part determines, on the basis of said phrase information, that said user calls on said audience with a specific phrase in a question form and determines, on the basis of said first motion information, that the direction of said voice input device is reversed, and

said reaction instruction part selects reaction data recording a video image and sound of said audience who answers said specific phrase in unison, as said reproduced reaction data.

7. The amusement system according to claim 1, wherein

said performance specifying part determines that said user makes a performance for requesting said audience to sing along when said performance specifying part determines, on the basis of said phrase information, that said user calls on said audience with a specific phrase requesting said audience to sing along and determines, on the basis of said first motion information, that the direction of said voice input device is reversed, and

said reaction instruction part selects reaction data recording a video image and sound of said audience who sings along, as said reproduced reaction data.

8. The amusement system according to claim 1, wherein

said performance specifying part determines that said user makes a performance for leading said audience to clap when said performance specifying part determines, on the basis of said motion information, that said user claps, and

said reaction instruction part selects reaction data recording a video image and sound of said audience who claps, as said reproduced reaction data.

9. The amusement system according to claim 1, wherein

said performance specifying part determines that said user makes a performance for requesting said audience to wave both arms when said performance specifying part determines, on the basis of said motion information, that said user waves both arms, and

said reaction instruction part selects reaction data recording a video image and sound of said audience who waves both arms, as said reproduced reaction data.

10. The amusement system according to claim 1, further comprising:

a controller held by said user in one hand, with the other hand holding said voice input device;

wherein said controller includes

a second motion information output part for outputting second motion information indicating a motion of said controller, and

said performance specifying part specifies a performance of said user on the basis of said second motion information.

11. The amusement system according to claim 3, wherein

said reaction instruction part scores each of performances made in a first part of said music data on the basis of said reaction level of said audience for said performance, and scores each of performances made in a second part of said music data on the basis of said reaction level of said audience for said performance.

15. A voice input device used in said amusement system as defined in claim 1.

2. An amusement system, comprising:

a main apparatus;

a voice input device held by a user, for outputting voice inputted by said user as voice data;

an image pickup device for picking up an image of said user to output video data; and

a voice recognition device for performing a voice recognition process on said voice data, to thereby generate phrase information indicating a phrase that said user says,

wherein said main apparatus includes

a reproduction part for reproducing music data selected by said user and audience condition data recording at least one of a video image and sound which indicate the condition of audience;

a video analysis part for analyzing said video data to generate first motion information indicating a motion of said user;

a performance specifying part for specifying a performance of said user on the basis of at least one of said phrase information and said first motion information; and

a reaction instruction part for selecting reproduced reaction data corresponding to said specified performance out of a plurality of pieces of reaction data each indicating a reaction of said audience and instructing said reproduction part to reproduce said reproduced reaction data.

29. The amusement system according to claim 2, wherein

said reaction instruction part determines a reaction level of said audience for said specified performance on the basis of at least one of said phrase information and said first motion information and selects said reproduced reaction data on the basis of said reaction level of said audience.

30. The amusement system according to claim 29, wherein

said reaction instruction part determines a reproduction condition of said reproduced reaction data on the basis of said reaction level of said audience and instructs said reproduction part of said reproduction condition.

31. The amusement system according to claim 29, wherein

said reproduction part reproduces said audience condition data corresponding to an audience group set by said user, and

said reaction instruction part determines said reaction level of said audience on the basis of said audience group.

32. The amusement system according to claim 2, wherein

said performance specifying part determines that said user makes a performance for exciting said audience when said performance specifying part determines, on the basis of said phrase information, that said user calls on said audience with a specific phrase in a question form and determines, on the basis of said first motion information, that the direction of said voice input device is reversed, and

said reaction instruction part selects reaction data recording a video image and sound of said audience who answers said specific phrase in unison, as said reproduced reaction data.

33. The amusement system according to claim 2, wherein

said performance specifying part determines that said user makes a performance for requesting said audience to sing along when said performance specifying part determines, on the basis of said phrase information, that said user calls on said audience with a specific phrase requesting said audience to sing along and determines, on the basis of said first motion information, that the direction of said voice input device is reversed, and

said reaction instruction part selects reaction data recording a video image and sound of said audience who sings along, as said reproduced reaction data.

34. The amusement system according to claim 2, wherein

said performance specifying part determines that said user makes a performance for leading said audience to clap when said performance specifying part determines, on the basis of said motion information, that said user claps, and

said reaction instruction part selects reaction data recording a video image and sound of said audience who claps, as said reproduced reaction data.

35. The amusement system according to claim 2, wherein

said performance specifying part determines that said user makes a performance for requesting said audience to wave both arms when said performance specifying part determines, on the basis of said motion information, that said user waves both arms, and

said reaction instruction part selects reaction data recording a video image and sound of said audience who waves both arms, as said reproduced reaction data.

36. The amusement system according to claim 29, wherein

said reaction instruction part scores each of performances made in a first part of said music data on the basis of said reaction level of said audience for said performance, and scores each of performances made in a second part of said music data on the basis of said reaction level of said audience for said performance.

37. A voice input device used in said amusement system as defined in claim 2.

12. An amusement system, comprising:

a main apparatus; and

a voice input device held by a user,

wherein said voice input device includes

a wireless communication part for performing wireless communication with said main apparatus, and

said main apparatus includes

a reproduction part for reproducing music data selected by said user and audience condition data recording at least one of a video image and sound which indicate the condition of audience;

a performance specifying part for specifying a performance of said user on the basis of whether or not there is a wireless signal transmitted from said voice input device; and

a reaction instruction part for selecting reproduced reaction data corresponding to said specified performance out of a plurality of pieces of reaction data each recording sound or a video image indicating a reaction of said audience and instructing said reproduction part to reproduce said reproduced reaction data.

13. The amusement system according to claim 12, wherein

said performance specifying part determines that said user enters a virtual live place when said performance specifying part detects said wireless signal after reproduction of said music data is started.

14. The amusement system according to claim 12, wherein

said performance specifying part determines that said user leaves a virtual live place when said performance specifying part cannot detect said wireless signal after reproduction of said music data is started.

38. A voice input device used in said amusement system as defined in claim 12.

16. A program used in an amusement system comprising:

a main apparatus;

a voice input device held by a user, for outputting voice inputted by said user as voice data and outputting first motion information indicating a motion of said voice input device; and

a voice recognition device for performing a voice recognition process on said voice data, to thereby generate phrase information indicating a phrase that said user says,

said program executed by a computer incorporated in said main apparatus to cause said computer to perform the steps of;

reproducing music data selected by said user and audience condition data recording at least one of a video image and sound which indicate the condition of audience;

specifying a performance of said user on the basis of at least one of said phrase information and said first motion information; and

selecting reproduced reaction data corresponding to said specified performance out of a plurality of pieces of reaction data each indicating a reaction of said audience and instructing reproduction of said reproduced reaction data.

17. A program used in an amusement system comprising:

a main apparatus;

a voice input device held by a user, for outputting voice inputted by said user as voice data;

an image pickup device for picking up an image of said user to output video data; and

a voice recognition device for performing a voice recognition process on said voice data, to thereby generate phrase information indicating a phrase that said user says,

said program executed by a computer incorporated in said main apparatus to cause said computer to perform the steps of;

reproducing music data selected by said user and audience condition data recording at least one of a video image and sound which indicate the condition of audience;

analyzing said video data to generate first motion information indicating a motion of said user;

specifying a performance of said user on the basis of at least one of said phrase information and said first motion information; and

selecting reproduced reaction data corresponding to said specified performance out of a plurality of pieces of reaction data each indicating a reaction of said audience and instructing reproduction of said reproduced reaction data.

18. A program executed by a computer incorporated in a main apparatus capable of performing wireless communication with a voice input device held by a user, to cause said computer to perform the steps of:

reproducing music data selected by said user and audience condition data recording at least one of a video image and sound which indicate the condition of audience;

specifying a performance of said user on the basis of whether or not there is a wireless signal transmitted from said voice input device; and

selecting reproduced reaction data corresponding to said specified performance out of a plurality of pieces of reaction data each indicating a reaction of said audience and instructing reproduction of said reproduced reaction data.

19. An amusement system, comprising:

a main apparatus for reproducing selected music data;

a performance information generation device for generating performance information in accordance with a performance of a first user; and

a staging instruction device for generating staging instruction information used for instructing staging of said performance of said first user in accordance with a manipulation of a second user,

wherein said main apparatus includes

a performance condition data generation part for generating performance condition data indicating the condition of said performance of said first user on the basis of said music data and said performance information and changing a generation condition of said performance condition data in accordance with said staging instruction information;

a staging evaluation part for evaluating a staging content of said performance of said first user with reference to said staging instruction information and instructing said performance condition data generation part to generate reaction data indicating a reaction of audience who views said performance of said first user on the basis of an evaluation result; and

a reproduction part for reproducing said performance condition data and said reaction data.

20. The amusement system according to claim 19, wherein

said staging evaluation part evaluates said staging of said performance of said first user on the basis of all pieces of said staging instruction information generated in a period while said performance condition data is reproduced, to thereby generate total evaluation information, and instructs generation of said reaction data corresponding to said total evaluation information.

21. The amusement system according to claim 19, wherein

said staging evaluation part evaluates said staging of said performance of said first user every time when said staging instruction information is inputted to said main apparatus, to thereby generate individual evaluation information, and instructs generation of said reaction data corresponding to said individual evaluation information.

22. The amusement system according to claim 19, wherein

said staging instruction device comprises

a voice input part for inputting voice of said second user to generate voice data;

and

a staging content instruction part for displaying a content that said second user should read aloud during reproduction of said performance condition data and outputting said voice data as said staging instruction information to said main apparatus,

said performance condition data generation part generates performance condition data including said voice data, and

said staging evaluation part evaluates said staging of said performance on the basis of a timing at which input of said voice data is ended.

23. The amusement system according to claim 22, wherein

said staging evaluation part performs a voice recognition process on said voice data to specify a voice phrase inputted to said voice input part by said second user and compares said voice phrase with said content, to thereby evaluate said staging of said performance.

24. The amusement system according to claim 19, wherein

said staging instruction device comprises

a staging content instruction part for generating firework setting-off information designating the type of firework selected by said second user and instructing setting off of said selected firework, to output said firework setting-off information as said staging instruction information to said main apparatus,

said performance condition data generation part generates performance condition data indicating how said firework which colors said performance of said first user is set off, on the basis of said firework setting-off information, and

said staging evaluation part evaluates said staging of said performance on the basis of said firework setting-off information.

25. The amusement system according to claim 19, wherein

said music data is composed of a plurality of pieces of part data corresponding to performance parts of music, respectively,

said staging instruction device comprises

a staging content instruction part for generating reproduction condition information indicating a reproduction condition of each part music data in accordance with said manipulation of said second user, to output said reproduction condition information as said staging instruction information to said main apparatus,

said performance condition data generation part generates performance condition data in which a condition for reproduction of said part music data is changed on the basis of said reproduction condition information, and

said staging evaluation part evaluates said staging of said performance on the basis of said reproduction condition information.

26. The amusement system according to claim 19, wherein

said staging instruction device comprises

a staging content instruction part for generating lighting object information indicating an irradiation object of lighting which colors said performance of said first user in accordance with said manipulation of said second user, to output said lighting object information as said staging instruction information to said main apparatus,

said performance condition data generation part generates performance condition data indicating how said lighting is performed on the basis of said lighting object information, and

said staging evaluation part evaluates said staging of said performance on the basis of said lighting object information.

27. The amusement system according to claim 19, wherein

said staging instruction device comprises

a display part for displaying video data transmitted from said main apparatus;

a motion detection part for detecting a motion of said staging instruction device, to thereby generate motion information; and

a staging content instruction part for outputting said motion information as said staging instruction information to said main apparatus,

said main apparatus further comprises

a lighting video data generation part for specifying an irradiation area of lighting which colors said performance of said first user on the basis of said motion information, to thereby generate lighting video data indicating how a light is shined on said irradiation area; and

an output part for outputting said lighting video data to said staging instruction device.

28. A program used in an amusement system comprising:

a main apparatus for reproducing selected music;

a performance information generation device for generating performance information in accordance with a performance of a first user; and

a staging instruction device for generating staging instruction information used for instructing staging of said performance of said first user in accordance with a manipulation of a second user,

said program executed by a computer incorporated in said main apparatus to cause said computer to perform the steps of;

generating performance condition data indicating the condition of said performance of said first user on the basis of said music data and said performance information and changing a generation condition of said performance condition data in accordance with said staging instruction information;

evaluating a staging content of said performance of said first user with reference to said staging instruction information and instructing said performance condition data generation part to generate reaction data indicating a reaction of audience who views said performance of said first user on the basis of an evaluation result; and

reproducing said performance condition data and said reaction data.