Image recognition apparatus and image recognition program

• US 20050254685A1
• Filed: 07/31/2002
• Published: 11/17/2005
• Est. Priority Date: 07/31/2002
• Status: Active Grant

• Current Assignee: National Institute of Information and Communications Technology
• Original Assignee: National Institute of Information and Communications Technology
• Inventors: Miyamori, Hisashi
• Granted Patent: US 7,436,983 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44

Abstract

An image recognition apparatus is configured to recognize movements of players in a sport match or game from contents recording a sport match or game wherein the players match against each other between domains partitioned with such an obstacle as net, the image recognition apparatus including: a picture information obtaining section configured to obtain picture information containing an image of a movement of at least one of the players from the contents; a sound information obtaining section 103 configured to obtain sound information generated in synchronism with the picture information from the contents, the sound information including information on a hitting sound generated upon hitting of such an instrument as a ball moving between the domains; a hitting time information specifying section 105 configured to specify a hitting time at which the instrument is hit based on the sound information; a rule information storage section 102 configured to store rule information for carrying out the sport match or game; and an image substance recognizing section 106 configured to recognize a substance of an image containing the image of the movement of the player provided by the picture information based on the picture information, a position of the instrument at the specified hitting time and the rule information.

Claims

1. An image recognition apparatus for recognizing movements of players matched against each other between domains partitioned with such an obstacle as net in a sport match or game from contents including a television program being telecasted to show the sport match or game, an image material in an uncompleted state for broadcasting and contents recorded in such a recording medium as a VTR, the image recognition apparatus comprising:

a picture information obtaining section configured to obtain picture information containing an image of a movement of at least one of the players playing in the sport match or game from the contents;

a sound information obtaining section configured to obtain sound information generated in synchronism with the picture information from the contents, the sound information including information on a hitting sound generated upon hitting of such an instrument as a ball moving between the domains to serve as an object of score count in the sport match or game;

a hitting time information specifying section configured to specify a hitting time at which the instrument is hit based on the sound information obtained by the sound information obtaining section;

a rule information storage section configured to store rule information for carrying out the sport match or game; and

an image substance recognizing section configured to recognize a substance of an image containing the image of the movement of the player provided by the picture information based on the picture information obtained by the picture information obtaining section, a position of the instrument at the hitting time specified by the hitting time specifying section and the rule information stored in the rule information storage section.

2. The image recognition apparatus according to claim 1, wherein when the sound information assumes a value higher than a predetermined level, the hitting time information specifying section specifies as the hitting time a point in time at which the higher value is assumed.

3. The image recognition apparatus according to claim 1, wherein:

the sound information obtaining section is provided with a filter portion configured to permit sound within a predetermined frequency band to pass therethrough; and

the sound information is information on the sound having passed through the filter portion.

4. The image recognition apparatus according to claim 3, wherein the filter portion comprises a band-pass filter.

5. The image recognition apparatus according to claim 1, wherein the hitting time information specifying section is configured to specify the hitting time based on hitting sound prospect data including data on a predetermined time period within which the hitting sound extracted from the sound information is generated.

6. The image recognition apparatus according to claim 1, wherein the hitting time information specifying section is configured to extract plural hitting sound prospect data items from the sound information in such a manner that a hitting sound prospect data item generated at one point in time and a subsequent hitting sound prospect data item generated at a succeeding point in time share data on a same time and then specify the hitting time based on the plural hitting sound prospect data items.

7. The image recognition apparatus according to claim 6, wherein the plural hitting sound prospect data items have equal data length, while the hitting time information specifying section is configured to extract the plural hitting sound prospect data items from the sound information at constant time intervals.

8. The image recognition apparatus according to claim 1, further comprising a hitting sound pattern information storage section configured to store hitting sound pattern information including information on patterns of sound changes that occur depending on how the instrument is hit by such an instrument as a racket constantly held and used by each of the players, wherein the hitting time information specifying section is configured to specify the hitting time based on the hitting sound pattern information stored in the hitting sound pattern information storage section and the sound information.

9. The image recognition apparatus according to claim 1, wherein the picture information obtaining section includes a domain element extracting section configured to extract from the picture information facility information including information on the obstacle, information on the domains and information on boundary lines between each of the domains and an area outside the domain, player'"'"'s position information indicative of a player'"'"'s position, and instrument information on the instrument moving between the domains to serve as an object of score count in the sport match or game.

10. The image recognition apparatus according to claim 9, wherein the player'"'"'s position information is position information on a domain containing each of the players and the instrument constantly held and used by the player.

11. The image recognition apparatus according to claim 10, wherein the domain element extracting section is configured to extract the player'"'"'s position information from the picture information based on the facility information extracted by the domain element extracting section.

12. The image recognition apparatus according to claim 9, wherein the domain element extracting section is configured to extract the instrument information from the picture information based on the facility information and the player'"'"'s position information extracted by the domain element extracting section.

13. The image recognition apparatus according to claim 9, wherein the facility information, the player'"'"'s position information, the instrument information and the rule information are based on knowledge about a sport as a subject for image extraction.

14. An image recognition program cooperative with a computer for causing an image recognition apparatus to operate to recognize movements of players matched against each other between domains partitioned with such an obstacle as net in a sport match or game from contents including a television program being telecasted to show the sport match or game, an image material in an uncompleted state for broadcasting and contents recorded in such a recording medium as a VTR, the image recognition program being configured to cause the image recognition apparatus to function as:

a picture information obtaining section configured to obtain picture information containing an image of a movement of at least one of the players playing in the sport match or game from the contents;

a sound information obtaining section configured to obtain sound information generated in synchronism with the picture information from the contents, the sound information including information on a hitting sound generated upon hitting of such an instrument as a ball moving between the domains to serve as an object of score count in the sport match or game;

a hitting time information specifying section configured to specify a hitting time at which the instrument is hit based on the sound information obtained by the sound information obtaining section;

a rule information storage section configured to store rule information for carrying out the sport match or game; and

an image substance recognizing section configured to recognize a substance of an image containing the image of the movement of the player provided by the picture information based on the picture information obtained by the picture information obtaining section, a position of the instrument at the hitting time specified by the hitting time specifying section and the rule information stored in the rule information storage section.