IMAGE-BASED DETECTION OF OFFSIDE IN GAMEPLAY

• US 20200134319A1
• Filed: 10/30/2018
• Published: 04/30/2020
• Est. Priority Date: 10/30/2018
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: RANJAN, SAKET, KUMAR, SHREAY
• Granted Patent: US 10,853,658 B2
• Time in Patent Office: Days
• CPC Class Codes: A63B 71/06 Indicating or scoring devices for games or players , or for other sports activities
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/30224 Ball; Puck
• CPC Class Codes: G06T 2207/30228 Playing field
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 20/42 of sport video content

Abstract

An electronic device is provided for image-based detection of offside in gameplay. The electronic device estimates positions of each player-object of a first team and a second team in a current image and further estimates displacement and velocity values of a soccer-object. The electronic device detects a pass state of the soccer-object based on the displacement and the velocity values. The electronic device determines a set of passive offside positions of a set of player-objects of the first team based on the estimated positions of the each player-object of the first team. The electronic device further detects an active offside state of at least one player-object in the set of player-objects, based on a first distance between the soccer-object and each of the set of player-objects, and transmits a notification to a referee of the gameplay, in real-time or near real time, based on the detected active offside state.

Claims

1. An electronic device, comprising:

a memory configured to store a sequence of image frames of a match field; and

circuitry configured to;

estimate, from a current image frame of the sequence of image frames, a position of each player-object of a first plurality of player-objects of a first team and each player-object of a second plurality of player-objects of a second team, wherein the second team is in play with the first team;

estimate a set of displacement values and a set of velocity values of a soccer-object based on a position parameter of a soccer-object in a previous image frame, the current image frame, and a successive image frame of the sequence of image frames;

detect a pass state of the soccer-object based on at least one of the estimated set of displacement values and the set of velocity values,

wherein the detected pass state corresponds to a release of the soccer-object from a first player-object of the first plurality of player-objects towards at least one player-object of the first plurality of player-objects;

determine, from the current image frame, a set of passive offside positions of a set of player-objects from the first plurality of player-objects, based on the estimated position of each player-object of the first plurality of player-objects;

compute a first distance value between the soccer-object and a second player-object in the set of player-objects, for each of the current image frame and a set of successive image frames of the sequence of image frames;

compare the computed first distance value for each of the current image frame and the set of successive image frames with a threshold distance value; and

detect an active offside state of the second player-object, based on the comparison of the first distance value with the threshold distance value.

2. The electronic device according to claim 1, further comprising a plurality of image sensors,

wherein the plurality of image sensors is configured to capture the sequence of image frames from a field-of-view (FOV) of the plurality of image sensors,

wherein the FOV corresponds to a top view of at least a first half, a second half, or a full size of the match field, and

wherein at least one of the first half and the second half of the match field correspond to a territory of the second team.

3. The electronic device according to claim 2, wherein at least one image sensor of the plurality of image sensors is present in at least a drone or a cable-suspended camera system for the match field.

4. The electronic device according to claim 2, wherein the circuitry is further configured to control motion of the plurality of image sensors to capture a view of the match field from different image sensor positions or different elevations with respect to the match field.

5. The electronic device according to claim 1, wherein the circuitry is further configured to detect the first plurality of player-objects and the second plurality of player-objects based on at least one of facial characteristics, a color, or a pattern associated with each player-object of the first plurality of player-objects and the second plurality of player-objects.

6. The electronic device according to claim 1, wherein the circuitry is further configured to estimate a first identifier for the first team, a second identifier for the second team, and a third identifier for a referee in the match field.

7. The electronic device according to claim 6, wherein the first identifier, the second identifier, and the third identifier are estimated based on a deep neural network (DNN) model.

8. The electronic device according to claim 7,

wherein the DNN model is based on a faster-regional convolutional neural network (RCNN).

9. The electronic device according to claim 7,

wherein the DNN model is pre-trained based on a match dataset that comprises a plurality of image frames of historical gameplays of a plurality of soccer teams.

10. The electronic device according to claim 1, wherein the current image frame and the previous image frame are consecutive image frames of the sequence of image frames.

11. The electronic device according to claim 1, wherein the circuitry is further configured to:

compute a ratio of a first net displacement value and a second net displacement value based on the estimated set of displacement values,

wherein the first net displacement value corresponds to a displacement of the soccer-object from the previous image frame to the current image frame, and

wherein the second net displacement value corresponds to a displacement of the soccer-object from the current image frame to the successive image frame;

compare the computed ratio with a threshold difference value; and

detect the pass state further based on the comparison of the ratio with the threshold difference value.

12. The electronic device according to claim 1, wherein the circuitry is further configured to:

determine a first direction value and a second direction value based on a first displacement value and a second displacement value of the set of displacement values,

wherein the first displacement value and the second displacement value correspond to the current image frame and the previous image frame;

compare a ratio of the first direction value and the second direction value to a threshold direction value; and

detect the pass state further based on the comparison of the ratio with the threshold direction value.

13. The electronic device according to claim 1,

wherein the first player-object of the first plurality of player-objects is in possession of the soccer-object in a territory of the second team prior to the detection of the pass state, and

wherein the current image frame and the previous image frame correspond to the territory of the second team in the match field.

14. The electronic device according to claim 1, wherein the circuitry is further configured to:

generate a first list that comprises the estimated position of each player-object of the first plurality of player-objects in an ascending order; and

generate a second list that comprises the estimated position of each player-object of the second plurality of player-objects in the ascending order.

15. The electronic device according to claim 14,

compare the estimated position of each of the first plurality of player-objects with the estimated position of a reference player-object of the second team,

wherein the reference player-object corresponds to a second last position in the generated second list;

and

determine the set of passive offside positions further based on the comparison of the estimated position of each of the first plurality of player-objects with the estimated position of the reference player-object.

16. The electronic device according to claim 1, wherein a distance of each of the set of player-objects from a goalpost of the second team is less than the estimated position of a reference player-object from the goalpost.

17. The electronic device according to claim 1, wherein the active offside state corresponds to an offside offense committed by the at least one player-object of the first team within a duration of a gameplay.

18. The electronic device according to claim 17, wherein circuitry is further configured to generate an offside status image based on the detected active offside state,

wherein the offside status image comprises an offside identifier, and

wherein the offside identifier comprises a line that corresponds to the estimated position of a reference player-object of the second team.

19. The electronic device according to claim 18, wherein the circuitry is further configured to control display of offside information on a display screen, wherein the offside information corresponds to the detected active offside state and the offside status image.

20. The electronic device according to claim 19, wherein the circuitry is further configured to:

transmit a notification on a wearable device based on the detected active offside state,

wherein the wearable device is worn by a referee in the match field, and

wherein the notification comprises the offside information that corresponds to the active offside state and the offside status image.

21. The electronic device according to claim 20, wherein the notification is transmitted in real time or near real time to the wearable device.

22. A method, comprising:

in an electronic device;

estimating, from a current image frame of a sequence of image frames, a position of each player-object of a first plurality of player-objects of a first team and each player-object of a second plurality of player-objects of a second team, wherein the second team is in play with the first team;

estimating a set of displacement values and a set of velocity values of a soccer-object based on a position parameter of a soccer-object in a previous image frame, the current image frame, and a successive image frame of the sequence of image frames;

detecting a pass state of the soccer-object based on at least one of the estimated set of displacement values and the set of velocity values,

wherein the detected pass state corresponds to a release of the soccer-object from a first player-object of the first plurality of player-objects towards at least one player-object of the first plurality of player-objects;

determining, from the current image frame, a set of passive offside positions of a set of player-objects of the first plurality of player-objects, based on the estimated position of each player-object of the first plurality of player-objects; and

computing a first distance value between the soccer-object and a second player-object in the set of player-objects, for each of the current image frame and a set of successive image frames of the sequence of image frames;

comparing the computed first distance value for each of the current image frame and the set of successive image frames with a threshold distance value; and

detecting an active offside state of the second player-object, based on the comparison of the first distance value with the threshold distance value.