Indicating method and device for correcting failure of motion-sensing interaction tracking

• US 9,990,031 B2
• Filed: 10/21/2016
• Issued: 06/05/2018
• Est. Priority Date: 01/18/2016
• Status: Active Grant

• Current Assignee: BOE Technology Group Co., Ltd.
• Original Assignee: BOE Technology Group Co., Ltd.
• Inventors: Li, Yingjie
• Primary Examiner(s): SAINI, AMANDEEP SINGH
• Application Number:
  US15/542,353
• Publication Number:
  US 20180067544A1
• Time in Patent Office: 592 Days
• Field of Search: 382103
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training

Abstract

An indicating method and device for correcting failure of motion-sensing interaction tracking. The method includes: acquiring physical space coordinates of a border of an actual motion-sensing interaction range of a human body and physical space coordinates of one or more key-part points in the human body; if a failure of motion-sensing interaction tracking is determined, obtaining, a current position of the human body relative to the border of the actual motion-sensing interaction range based on the physical space coordinates of the border of the actual motion-sensing interaction range and a physical space coordinate of at least one key-part point from the one or more key-part points acquired in a latest time; and scaling down the border of the actual motion-sensing interaction range and a distance of the current position relative to the border of the actual motion-sensing interaction range, and displaying at a display interface through an auxiliary image.

Claims

1. An indicating method for correcting failure of motion-sensing interaction tracking, comprising:

acquiring physical space coordinates of a border of an actual motion-sensing interaction range of a human body on a ground and physical space coordinates of one or more key-part points in the human body;

if a failure of motion-sensing interaction tracking is determined, obtaining, on the ground or a plane parallel to the ground, a current position of the human body relative to the border of the actual motion-sensing interaction range based on the physical space coordinates of the border of the actual motion-sensing interaction range and a physical space coordinate of at least one key-part point from the one or more key-part points acquired in a latest time; and

scaling down the border of the actual motion-sensing interaction range and a distance of the current position relative to the border of the actual motion-sensing interaction range, and displaying at a preset position of a display interface through an auxiliary image.

2. The method according to claim 1, wherein before performing motion sensing interaction, the method further comprises:

reading a parameter of the auxiliary image, the parameter of the auxiliary image including a position and a size of the auxiliary image, whether the auxiliary image is always displayed, or an automatic disappearing delay time;

displaying the parameter through the display interface so that the parameter is set by a user; and

if resetting data of the parameter is received, configuring the parameter of the auxiliary image to be stored as a latest configured parameter;

otherwise, exiting setting of the parameter of the auxiliary image.

3. The method according to claim 2, wherein if the parameter of the auxiliary image is set to be always displayed, the auxiliary image is displayed at the preset position of the display interface all the time;

or if the parameter of the auxiliary image is set to be not always displayed and the automatic appearing delay time is not zero, the auxiliary image is displayed at the preset position of the display interface within the automatic disappearing delay time; and

in a normal process of motion-sensing interaction tracking, the current position of the human body relative to the border of the actual motion-sensing interaction range on the ground or the plane parallel to the ground is obtained based on the acquired physical space coordinates of the border of the actual motion-sensing interaction range and a current physical space coordinate of the at least one key-part point.

4. The method according to claim 1, wherein the step of if a failure of motion-sensing interaction tracking is determined, obtaining, on the ground or the plane parallel to the ground, the current position of the human body relative to the border of the actual motion-sensing interaction range based on the physical space coordinates of the border of the actual motion-sensing interaction range and the physical space coordinate of at least one key-part point from the one or more key-part points acquired in the latest time, includes:

if it is determined that the human body is currently located within the border of the actual motion-sensing interaction range, taking a position of the human body relative to the actual motion-sensing interaction range on the ground or the plane parallel to the ground obtained in the latest time as the current position, based on the physical space coordinates of the border of the actual motion-sensing interaction range and the physical space coordinate of the at least one key-part point from the one or more key-part points acquired in the latest time; and

if it is determined that the human body is currently located outside the actual motion-sensing interaction range, calculating a current physical space coordinate of the at least one key-part point in the human body based on the physical space coordinate of the at least one key-part point acquired in the latest time and a moving direction of the human body, and then obtaining the current position of the human body relative to the border of the actual motion-sensing interaction range on the ground or the plane parallel to the ground.

5. The method according to claim 4, wherein the step of if it is determined that the human body is currently located within the border of the actual motion-sensing interaction range, taking a position of the human body relative to the actual motion-sensing interaction range obtained in the latest time on the ground or the plane parallel to the ground as the current position, based on the physical space coordinates of the border of the actual motion-sensing interaction range and the physical space coordinate of the at least one key-part point from the one or more key-part points acquired in the latest time, includes:

obtaining a circular range by taking an average physical space coordinate of the at least one key-part point as a center and taking a maximum moving distance of a common human body within a time period of acquiring each image frame as a radius, based on the physical space coordinate of the at least one key-part point acquired in the latest time; and

based on the physical space coordinates of the border of the actual motion-sensing interaction range, if the circular range is located within the border of the actual motion-sensing interaction range, determining that the human body is currently located within the border of the actual motion-sensing interaction range, and taking a position of the human body relative to the border of the actual motion-sensing interaction range on the ground or the plane parallel to the ground obtained in the latest time as the current position.

6. The method according to claim 5, wherein if the circular range is not completely located within the border of the actual motion-sensing interaction range, the method further comprises:

processing an acquired current image line by line, counting a number of points within a maximum moving distance range of the common human body within the time period of acquiring each image frame by taking a preset position in the current image as a center, determining that the human body is currently located within the border of the actual motion-sensing interaction range if a counting result is greater than a preset threshold, otherwise, determining that the human body is currently located outside the border of the actual motion-sensing interaction range,

wherein the preset position is a position corresponding to a point on the trunk in an image acquired in the latest time.

7. The method according to claim 6, wherein the step of if it is determined that the human body is currently located outside the actual motion-sensing interaction range, calculating a current physical space coordinate of the at least one key-part point in the human body based on the physical space coordinate of the at least one key-part point acquired in the latest time and a moving direction of the human body, and then obtaining the current position of the human body relative to the border of the actual motion-sensing interaction range on the ground or the plane parallel to the ground, includes:

obtaining a current physical space coordinate of the at least one key-part point in the human body based on the physical space coordinate of the at least one key-part point acquired in the latest time, the moving direction of the human body, and the maximum moving distance of the common human body within the time period of acquiring each image frame; and

obtaining the current position of the human body relative to the border of the actual motion-sensing interaction range on the ground or the plane parallel to the ground based on the physical space coordinate of the at least one key-part point in the human body,

wherein the moving direction is obtained by comparing the physical space coordinate of the at least one key-part point acquired in the latest time before failure of motion-sensing interaction tracking and a physical space coordinate of the at least one key-part point acquired in a time prior to the latest time.

8. The method according to claim 1, wherein in a case of failure of motion-sensing interaction tracking and after the display of the auxiliary image in the display interface, the method further comprises:

sending a message to prompt a user to move.

9. The method according to claim 1, wherein the one or more key-part points of the human body are one or more joint points of the human body.

10. An indicating device for correcting failure of motion-sensing interaction tracking, comprising:

an acquisition module, a processing module connected with the acquisition module, and a control module connected with both the acquisition module and the processing module, wherein;

the acquisition module is configured to acquire physical space coordinates of a border of an actual motion-sensing interaction range of the human body on a ground and physical space coordinates of one or more key-part points in the human body;

the processing module is configured to, if failure of motion-sensing interaction tracking is determined, obtain a current position of the human body relative to the border of the actual motion-sensing interaction range on the ground or a plane parallel to the ground based on the physical space coordinates of the border of the actual motion-sensing interaction range acquired by the acquisition module and a physical space coordinate of at least one key-part point from the one or more key-part points acquired in the latest time; and

the control module is configured to scale down the border of the actual motion-sensing interaction range acquired by the acquisition module and a distance of the current position relative to the border of the actual motion-sensing interaction range obtained by the processing module, and to display at a preset position of a display interface through an auxiliary image.

11. The device according to claim 10, further comprising a read module connected with the control module, wherein the read module is configured to read a parameter of the auxiliary image before motion sensing interaction, and the parameter of the auxiliary image includes:

a position and a size of the auxiliary image, whether the auxiliary image is always displayed, and an automatic disappearing delay time; and

the control module is also configured to;

display the parameter through the display interface so that the parameter is set by a user; and

if resetting data of the parameter is received, configure the parameter of the auxiliary image to be stored as a latest configured parameter, otherwise, exit setting of the parameter of the auxiliary image.

12. The device according to claim 11, wherein if the parameter of the auxiliary image is set to be always displayed, the auxiliary image is displayed at the preset position of the display interface all the time;

or if the parameter of the auxiliary image is set to be not always displayed and the automatic appearing delay time is not zero, the auxiliary image is displayed at the preset position of the display interface within the automatic disappearing delay time; and

in a normal process of motion-sensing interaction tracking, the processing module is further configured to obtain the current position of the human body relative to the border of the actual motion-sensing interaction range on the ground or the plane parallel to the ground based on the acquired physical space coordinates of the border of the actual motion-sensing interaction range and a current physical space coordinate of the at least one key-part point.

13. The device according to claim 10, wherein the processing module is configured to:

when failure of motion-sensing interaction tracking is determined, if it is determined that the human body is currently located within the border of the actual motion-sensing interaction range, take a position of the human body relative to the actual motion-sensing interaction range on the ground or the plane parallel to the ground obtained in the latest time as the current position, based on the physical space coordinates of the border of the actual motion-sensing interaction range and the physical space coordinate of the at least one key-part point from the one or more key-part points acquired in the latest time; and

if it is determined that the human body is currently located outside the actual motion-sensing interaction range, calculate a current physical space coordinate of the at least one key-part point in the human body based on the physical space coordinate of the at least one key-part point acquired in the latest time and a moving direction of the human body, and then obtain the current position of the human body relative to the border of the actual motion-sensing interaction range on the ground or the plane parallel to the ground.

14. The device according to claim 13, wherein the processing module is configured to:

obtain a circular range by taking an average physical space coordinate of the at least one key-part point as a center and taking a maximum moving distance of a common human body within a time period of acquiring each image frame as a radius, based on the physical space coordinate of the at least one key-part point acquired in the latest time; and

based on the physical space coordinates of the border of the actual motion-sensing interaction range, if the circular range is located within the border of the actual motion-sensing interaction range, determine that the human body is currently located within the border of the actual motion-sensing interaction range, and take a position of the human body relative to the border of the actual motion-sensing interaction range on the ground or the plane parallel to the ground obtained in the latest time as the current position.

15. The device according to claim 14, wherein if the circular range is not completely located within the border of the actual motion-sensing interaction range, the processing module is further configured to:

process an acquired current image line by line, count a number of points within a maximum moving distance range of the common human body within the time period of acquiring each image frame by taking a preset position in the current image as a center; and

determine that the human body is currently located within the border of the actual motion-sensing interaction range if a counting result is greater than a preset threshold, otherwise, determine that the human body is currently located outside the border of the actual motion-sensing interaction range,

wherein the preset position is a position corresponding to a point on the trunk in an image acquired in the latest time.

16. The device according to claim 15, wherein if the human body is currently located outside the border of the actual motion-sensing interaction range, the processing module is configured to:

obtain a current physical space coordinate of the at least one key-part point in the human body based on the physical space coordinate of the at least one key-part point acquired in the latest time, the moving direction of the human body, and the maximum moving distance of the common human body within the time period of acquiring each image frame; and

obtain the current position of the human body relative to the border of the actual motion-sensing interaction range on the ground or the plane parallel to the ground based on the physical space coordinate of the at least one key-part point in the human body,

wherein the moving direction is obtained by comparing the physical space coordinate of the at least one key-part point acquired in the latest time before failure of motion-sensing interaction tracking and a physical space coordinate of the at least one key-part point acquired in a time prior to the latest time.

17. The device according to claim 10, wherein in a case of failure of motion-sensing interaction tracking and after the display of the auxiliary image in the display interface, the control module is configured to send a message to prompt a user to move.