Tracking specific gestures relative to user movement

• US 10,265,621 B2
• Filed: 01/20/2015
• Issued: 04/23/2019
• Est. Priority Date: 01/20/2015
• Status: Active Grant

• Current Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Original Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Inventors: Goslin, Michael P., Olson, Joseph L., Haseltine, Eric C.
• Primary Examiner(s): Garner, Werner G
• Application Number:
  US14/601,105
• Publication Number:
  US 20160206957A1
• Time in Patent Office: 1,554 Days
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/245 specially adapted to a particular type of game, e.g. steering wheels
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/833 Hand-to-hand fighting, e.g. martial arts competition A63F13/837 takes precedence
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Abstract

Techniques are disclosed for distinguishing user movements in an immersive play environment. A first tracking device is provided that measures first tracking data. A second tracking device is also provided to measure second tracking data. A controller device is also provided to receive tracking data from the second tracking device. The second tracking device measures movement of a user by receiving the first tracking data from the first tracking device and determining third tracking data movement based on a combination of the first tracking data and the second tracking data.

Claims

1. A platform providing an immersive play environment, the platform comprising:

a controller device configured to transmit instructions to an interactive toy device in the immersive play environment;

a plurality of tracking devices of a user during a play experience in a physical space, the plurality of tracking devices including a handheld device configured to be held by the user and a wearable device configured to be worn by the user, wherein the handheld device and the wearable device include at least one of a gyroscope, an accelerometer, and a magnetometer, wherein each of the handheld device and the wearable device is configured to capture movement of the user in the physical space over time as first tracking data and second tracking data, respectively, wherein a virtual object is extrapolated from the handheld device in a three-dimensional vector space corresponding to the physical space, wherein the virtual object is substantially linear, wherein the wearable device is configured to receive the first tracking data from the handheld device, generate combined tracking data based at least in part on the first tracking data and the second tracking data, and transmit the combined tracking data to the controller device, whereupon the controller device;

determines that the combined tracking data reflects a movement sequence that triggers a hit event, based on determining;

(i) that the movement sequence is within a predefined range of the interactive toy device and (ii) that the movement sequence defines an arc in the three-dimensional vector space with a rotational velocity exceeding a specified threshold velocity; and

upon determining that the movement sequence triggers the hit event, causes the interactive toy device to perform, in the physical space, an action simulating collision between the virtual object and the interactive toy device absent a physical collision therebetween, the action comprising an effect selected from an audio effect, a visual effect, and an audiovisual effect.

2. The platform of claim 1, wherein the handheld device includes the gyroscope, wherein the wearable device includes the accelerometer and the magnetometer.

3. The platform of claim 1, wherein the handheld device simulates a sword object.

4. The platform of claim 3, wherein the hit event comprises a sword hit event, wherein the controller device is further configured to:

register the sword hit event during the play experience; and

notify the user of the sword hit event.

5. The platform of claim 3, wherein the controller device is further configured to:

upon determining that either the rotational velocity does not exceed the specified threshold velocity or that the movement sequence is not within the predefined range of the interactive toy device, register a miss event during the play experience; and

notify the user of the miss event.

6. The platform of claim 1, wherein the controller device is further configured to instruct the user of a next movement sequence.

7. A computer-implemented method for monitoring movement of a user during a play experience in a physical space, the computer-implemented method comprising:

receiving combined tracking data associated with a plurality of tracking devices of the user, the plurality of tracking devices including a handheld device configured to be held by the user and a wearable device configured to be worn by the user, wherein the handheld device and the wearable device include at least one of a gyroscope, an accelerometer, and a magnetometer, wherein each of the handheld device and the wearable device is configured to capture the movement of the user in the physical space over time as first tracking data and second tracking data, respectively, wherein a virtual object is extrapolated from the handheld device in a three-dimensional vector space corresponding to the physical space, wherein the virtual object is substantially linear, wherein the wearable device is configured to receive the first tracking data from the handheld device and generate combined tracking data based at least in part on the first tracking data and the second tracking data, wherein the combined tracking data is received from the wearable device;

determining, by a controller device, that the combined tracking data reflects a movement sequence that triggers a hit event, based on determining;

(i) that the movement sequence is within a predefined range of an interactive toy device and (ii) that the movement sequence defines an arc in the three-dimensional vector space with a rotational velocity exceeding a specified threshold velocity; and

upon determining that the movement sequence triggers the hit event, causing the interactive toy device to perform, in the physical space, an action simulating collision between the virtual object and the interactive toy device absent a physical collision therebetween, the action comprising an effect selected from an audio effect, a visual effect, and an audiovisual effect.

8. The computer-implemented method of claim 7, wherein the handheld device simulates a sword object.

9. The computer-implemented method of claim 8, wherein the hit event comprises a sword hit event, wherein the computer-implemented method further comprises:

registering the sword hit event during the play experience; and

notifying the user of the sword hit event.

10. The computer-implemented method of claim 8, further comprising:

upon determining that either the rotational velocity does not exceed the specified threshold velocity or that the movement sequence is not within the predefined range of the interactive toy device, registering a miss event during the play experience; and

notifying the user of the miss event.

11. The computer-implemented method of claim 7, further comprising;

instructing the user of a next movement sequence.

12. The computer-implemented method of claim 7, wherein that the movement sequence triggers the hit event is determined with a higher measure of precision than in an absence of one of the handheld tracking device and the wearable tracking device.

13. The computer-implemented method of claim 12, wherein the controller device is configured to transmit instructions to the interactive toy device in the physical space, wherein the handheld device simulates a sword object, wherein the hit event comprises a sword hit event, wherein the computer-implemented method further comprises:

upon determining that the movement sequence triggers the hit event, registering the sword hit event during the play experience; and

notifying the user of the sword hit event.

14. The computer-implemented method of claim 13, further comprising:

upon determining that the rotational velocity does not exceed the specified threshold velocity, registering a first miss event during the play experience, whereafter the user is notified of the first miss event; and

upon determining that the movement sequence is not within the predefined range of the interactive toy device, registering a second miss event during the play experience, whereafter the user is notified of the second miss event.

15. The computer-implemented method of claim 14, further comprising instructing the user of a next movement sequence different than the movement sequence.

16. A non-transitory computer-readable medium storing instructions executable to perform an operation for monitoring movement of a user during a play experience in a physical space, the operation comprising:

receiving combined tracking data associated with a plurality of tracking devices of the user, the plurality of tracking devices including a handheld device configured to be held by the user and a wearable device configured to be worn by the user, wherein the handheld device and the wearable device include at least one of a gyroscope, an accelerometer, and a magnetometer, wherein each of the handheld device and the wearable device is configured to capture the movement of the user in the physical space over time as first tracking data and second tracking data, respectively, wherein a virtual object is extrapolated from the handheld device in a three-dimensional vector space corresponding to the physical space, wherein the virtual object is substantially linear, wherein the wearable device is configured to receive the first tracking data from the handheld device and generate combined tracking data based at least in part on the first tracking data and the second tracking data, wherein the combined tracking data is received from the wearable device;

determining that the combined tracking data reflects a movement sequence that triggers a hit event, based on determining;

(i) that the movement sequence is within a predefined range of an interactive toy device and (ii) that the movement sequence defines an arc in the three-dimensional vector space with a rotational velocity exceeding a specified threshold velocity; and

upon determining that the movement sequence triggers the hit event, causing the interactive toy device to perform, in the physical space, an action simulating collision between the virtual object and the interactive toy device absent a physical collision therebetween, the action comprising an effect selected from an audio effect, a visual effect, and an audiovisual effect.

17. The non-transitory computer-readable medium of claim 16, wherein the handheld device simulates a sword object.

18. The non-transitory computer-readable medium of claim 17, wherein the hit event comprises a sword hit event, wherein the operation further comprises:

registering the sword hit event during the play experience; and

notifying the user of the sword hit event.

19. The non-transitory computer-readable medium of claim 16, wherein the handheld device simulates a sword object, wherein the operation further comprises:

upon determining that either the rotational velocity does not exceed the specified threshold velocity or that the movement sequence is not within the predefined range of the interactive toy device, registering a miss event during the play experience; and

notifying the user of the miss event.

20. The non-transitory computer-readable medium of claim 16, wherein the operation further comprises instructing the user of a next movement sequence.