Head rotation tracking device for video highlights identification

• US 10,097,745 B2
• Filed: 04/27/2016
• Issued: 10/09/2018
• Est. Priority Date: 04/27/2016
• Status: Active Grant

• Current Assignee: Beijing Shunyuan Kaihua Technology Limited
• Original Assignee: Zepp Labs, Inc. (Zepp Health Corporation)
• Inventors: Han, Zheng, Dai, Xiaowei, Ye, Yusheng
• Primary Examiner(s): Czekaj, Dave
• Assistant Examiner(s): BRUMFIELD, SHANIKA M
• Application Number:
  US15/139,683
• Publication Number:
  US 20170318214A1
• Time in Patent Office: 895 Days
• Field of Search: 348157
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06V 20/42 of sport video content
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/66 Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects

Abstract

A solution is disclosed for conveniently capturing video highlights in a sports event by tracking head rotation movement of a user and synchronizing rotations of a video capturing device (e.g., the user'"'"'s mobile phone) on a rotary station with the head rotation movement of the user. The user wears a rotation tracking device, which tracks the head rotation movement of the user in connection with watching an event. The head rotation data from the user'"'"'s head rotation movement is filtered to generate filtered head rotation data, which excludes noise motions in the head rotation data. Corresponding rotary control data is generated based on the filtered head rotation data. The rotary control data is used to instruct a rotary system to rotate synchronously with head rotations of the user when a video capturing device mounted on the rotary system captures videos of the sports event.

Claims

1. A computer-implemented method, the method comprising:

receiving head rotation data of a user in connection with watching an event from a rotation tracking device worn by the user;

filtering the head rotation data to generate filtered head rotation data by;

storing the head rotation data in a sliding window buffer of a predetermined length,

determining at least one varying pattern of the head rotation data at each sampling point, and

generating a filtered head rotation data for each sampling point;

generating corresponding rotary control data based on the filtered head rotation data; and

instructing a rotary system to rotate synchronously with head rotations of the user based on the rotary control data.

2. The method of claim 1, further comprising:

calibrating the rotary system with the rotation tracking device worn by the user, wherein the rotary system and the rotation tracking device are in two different coordinate systems.

3. The method of claim 2, wherein the calibrating comprises:

determining a plurality of boundaries of a workspace of the rotary system, a boundary of the workspace of the rotary system describing a limit of rotation which can be performed by the rotary system; and

adjusting the determined boundaries of the workspace of the rotary system based on a calibrating point defined by the rotation tracking device.

4. The method of claim 1, wherein generating the filtered head rotation data for each sampling point comprises:

responsive to the determined varying pattern at a sampling point being linear, applying a linear regression process to the head rotation data; and

generating the filtered head rotation data for the sampling point based on the linear regression.

5. The method of claim 1, wherein generating the filtered head rotation data for each sampling point further comprises:

responsive to the determined varying pattern at a sampling point being nonlinear, calculating a median of the head rotation data stored in the sliding window buffer; and

generating the filtered head rotation data for the sampling point based on the median of the head rotation data.

6. The method of claim 1, wherein generating corresponding rotary control data comprises:

calculating amount of rotation and direction of the rotation for each rotation point based on the head rotation data corresponding to each rotation point; and

generating a plurality of rotary rotation commands for each rotation point, a rotary rotation command including information of amount of rotation and direction of the rotation to be performed by the rotary system.

7. The method of claim 1, further comprising:

receiving rotary rotation feedback data from the rotary system for each rotation point, the rotary rotation feedback data for each rotation point comprising information describing actual amount of rotation and direction of rotation performed by the rotary system; and

generating rotary control data for a next rotation point based on the received rotary rotation feedback data for a current rotation point.

8. The method of claim 1, wherein instructing the rotary system to rotate synchronously with the head rotations of the user comprises:

instructing the rotary system to follow the head rotation of the user at each corresponding rotation point in a same direction and by a same amount of rotation within boundaries of a workspace of the rotary system.

9. The method of claim 1, further comprising:

receiving instructions from the user to activate a video capturing device mounted unto the rotary system for capturing a video clip of the event; and

instructing the rotary system to follow the head rotations of the user when capturing the video clip.

10. A non-transitory computer-readable storage medium storing executable computer program instruction, the instructions when executed by a computer processor cause the computer processor to:

receive head rotation data of a user in connection with watching an event from a rotation tracking device worn by the user;

filter the head rotation data to generate filtered head rotation data by;

storing the head rotation data in a sliding window buffer of a predetermined length,

determine at least one varying pattern of the head rotation data at each sampling point, and

generate a filtered head rotation data for each sampling point;

generate corresponding rotary control data based on the filtered head rotation data; and

instruct a rotary system to rotate synchronously with head rotations of the user based on the rotary control data.

11. The non-transitory computer-readable storage medium of claim 10, further comprising computer programs instructions, when executed by the computer processor, that cause the computer processor to:

calibrate the rotary system with the rotation tracking device worn by the user, wherein the rotary system and the rotation tracking device are in two different coordinate systems.

12. The non-transitory computer-readable storage medium of claim 11, wherein the instructions for calibrating the rotary system with the rotation tracking device comprise instructions, when executed by the computer processor, that cause the computer processor to:

determine a plurality of boundaries of a workspace of the rotary system, a boundary of the workspace of the rotary system describing a limit of rotation which can be performed by the rotary system; and

adjust the determined boundaries of the workspace of the rotary system based on a calibrating point defined by the rotation tracking device.

13. The non-transitory computer-readable storage medium of claim 10, wherein the instructions for generating the filtered head rotation data for each sampling point comprise instructions, when executed by the processor, that cause the computer processor to:

responsive to the determined varying pattern at a sampling point being linear, apply a linear regression process to the head rotation data; and

generate the filtered head rotation data for the sampling point based on the linear regression.

14. The non-transitory computer-readable storage medium of claim 10, wherein the instructions for generating the filtered head rotation data for each sampling point further comprise instructions, when executed by the processor, that cause the computer processor to:

responsive to the determined varying pattern at a sampling point being nonlinear, calculate a median of the head rotation data stored in the sliding window buffer; and

generate the filtered head rotation data for the sampling point based on the median of the head rotation data.

15. The non-transitory computer-readable storage medium of claim 10, wherein the instructions for generating corresponding rotary control data comprise instructions, when executed by the computer processor, that cause the computer processor to:

calculate amount of rotation and direction of the rotation for each rotation point based on the head rotation data corresponding to each rotation point; and

generate a plurality of rotary rotation commands for each rotation point, a rotary rotation command including information of amount of rotation and direction of the rotation to be performed by the rotary system.

16. The non-transitory computer-readable storage medium of claim 10, further comprising computer program instructions, when executed by the computer processor, that cause the computer processor to:

receive rotary rotation feedback data from the rotary system for each rotation point, the rotary rotation feedback data for each rotation point comprising information describing actual amount of rotation and direction of rotation performed by the rotary system; and

generate rotary control data for a next rotation point based on the received rotary rotation feedback data for a current rotation point.

17. The non-transitory computer-readable storage medium of claim 10, wherein the instructions for instructing the rotary system to rotate synchronously with the head rotations of the user comprise instructions, when executed by the computer processor, that cause the computer processor to:

instruct the rotary system to follow the head rotation of the user at each corresponding rotation point in a same direction and by a same amount of rotation within boundaries of a workspace of the rotary system.

18. The non-transitory computer-readable storage medium of claim 10, further comprising computer program instructions, when executed by the computer processor, that cause the computer processor to:

receive instructions from the user to activate a video capturing device mounted unto the rotary system for capturing a video clip of the event; and

instruct the rotary system to follow the head rotations of the user when capturing the video clip.

19. A computer system, comprising:

a rotation tracking device configured to track head rotation movement of a user in connection with the user watching an event, the rotation tracking device worn by the user on a part of the user'"'"'s head; and

a rotary station configured to communicatively couple with the rotation tracking device, the rotary station comprising;

a control module configured to analyze the tracked head rotation movement of the user and to generate rotary control data based on the analyzed head rotation movement of the user by;

storing head rotation data representing the head rotation movement of the user in a sliding window buffer of a predetermined length,

determining at least one varying pattern of the head rotation data at each sampling point, and

generating a filtered head rotation data for each sampling point; and

a rotary system configured to rotate a video capturing device synchronously with head rotations of the user based on the rotary control data.

20. The computer system of claim 19, wherein the control module is further configured to:

calibrate the rotary system with the rotation tracking device worn by the user, wherein the rotary system and the rotation tracking device are in two different coordinate systems.

21. The computer system of claim 20, wherein calibrating the rotary system comprises:

determining a plurality of boundaries of a workspace of the rotary system, a boundary of the workspace of the rotary system describing a limit of rotation which can be performed by the rotary system; and

adjusting the determined boundaries of the workspace of the rotary system based on a calibrating point defined by the rotation tracking device.

22. The computer system of claim 19, wherein generating rotary control data comprises:

calculating amount of rotation and direction of the rotation for each rotation point based on the head rotation data corresponding to each rotation point; and

generating a plurality of rotary rotation commands for each rotation point, a rotary rotation command including information of amount of rotation and direction of the rotation to be performed by the rotary system.

23. The computer system of claim 19, wherein the control module is further configured to:

receive rotary rotation feedback data from the rotary system for each rotation point, the rotary rotation feedback data for each rotation point comprising information describing actual amount of rotation and direction of rotation performed by the rotary system; and

generate rotary control data for a next rotation point based on the received rotary rotation feedback data for a current rotation point.

24. The computer system of claim 19, further comprising:

a remote controller used by the user to control the video capturing device connected to the rotary system for capturing one or more videos of the event.