Processing tracking and recognition data in gestural recognition systems

• US 10,642,364 B2
• Filed: 10/28/2013
• Issued: 05/05/2020
• Est. Priority Date: 04/02/2009
• Status: Active Grant

• Current Assignee: Oblong Industries, Inc. (Oblong, Inc.)
• Original Assignee: Oblong Industries, Inc. (Oblong, Inc.)
• Inventors: Minnen, David
• Primary Examiner(s): Anya, Charles E
• Application Number:
  US14/064,736
• Publication Number:
  US 20140240231A1
• Time in Patent Office: 2,381 Days
• Field of Search: None
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/107 Static hand or arm
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

Systems and methods are described for detecting an event of a source device, and generating at least one data sequence comprising device event data specifying the event and state information of the event. The device event data and state information are type-specific data having a type corresponding to an application of the source device. A data capsule is formed to include the at least one data sequence. The data capsule has a data structure comprising an application-independent representation of the at least one data sequence. The systems and methods detect poses and motion of an object, translate the poses and motion into a control signal using a gesture notation, and control a computer application using the control signal. The systems and methods automatically detect a gesture of a body, translate the gesture to a gesture signal, and control a component coupled to a computer in response to the gesture signal.

Claims

1. A method of decoupling plurality of applications in a processing environment of a plurality of devices, the method comprising:

detecting an event of a source device;

generating at least one data sequence comprising device event data specifying the event am state information of the event, wherein the device event data and state information are type-specific data having a type corresponding to an application of the source device;

forming a data capsule to include the at least one data sequence, the data capsule having a data structure comprising an application-independent representation of the at least one data sequence;

selecting a repository of a plurality of repositories, according to at least one of a type of the device event data and a destination application;

transferring the data capsule into the selected repository;

maintaining intact the at least one data sequence of the data capsule during the transferring;

detecting a second event of a second source device;

searching the repository for data capsules corresponding to the second event identifying a correspondence between the data capsule and the second event, wherein the searching includes skimming backwards and skimming forwards to determine whether the data capsules corresponding to the second event has appeared in conjunction with certain others;

extracting the data capsule from the repository in response to the identifying; and

executing on behalf of the second source device a processing operation corresponding to the second event on behalf of the second source device in response to contents of the data capsule, wherein the source device corresponds to an application of a first type and the second source device corresponds to a second application of a second type.

2. The method of claim 1, wherein the generating of the at least one data sequence comprises:

generating a first respective data set that includes first respective device event data;

generating a second respective data set that includes second respective state information; and

forming a first data sequence to include the first respective data set and the second respective data set.

3. The method of claim 2, wherein the generating of the first respective data set includes forming the first respective data set to include identification data of the source device, the identification data including data identifying the source device.

4. The method of claim 2, wherein the generating of the at least one data sequence comprises:

generating a first respective data set that includes first respective device event data;

generating a second respective data set that includes second respective state information; and

forming a second data sequence to include the first respective data set and the second respective data set.

5. The method of claim 4, wherein the generating of the first respective data set includes generating a first respective data set offset, wherein the first respective data set offset points to the first respective data set of the second data sequence.

6. The method of claim 4, wherein the generating of the second respective data set includes generating a second respective data set offset, wherein the second respective data set offset points to the second respective data set of the second data sequence.

7. The method of claim 2, wherein the first respective data set is a description list, the description list including a description of the data.

8. The method of claim 1, wherein the device event data is a tagged byte-sequence representing typed data.

9. The method of claim 8, wherein the device event data includes a type header and a type-specific data layout.

10. The method of claim 1, wherein the state information is a tagged byte-sequence representing typed data.

11. The method of claim 10, wherein the state information includes a type header and a type-specific data layout.

12. The method of claim 1, comprising:

generating at least one offset; and

forming the data capsule to include the at least one offset.

13. The method of claim 12, comprising:

generating a first offset having a first variable length,

wherein the first offset points to the device event data of a first data sequence of the at least one data sequence.

14. The method of claim 12, comprising:

generating a second offset having a second variable length,

wherein the second offset points to the state information of a first data sequence of the at least one data sequence.

15. The method of claim 12, comprising:

forming a first code path through the data capsule using a first offset of the at least one offset; and

forming a second code path through the data capsule using a second offset of the at least one offset,

wherein the first code path and the second code path are different paths.

16. The method of claim 12, wherein at least one of the first offset and the second offset include metadata, the metadata comprising context-specific metadata corresponding to a context of the application.

17. The method of claim 1, comprising:

generating a header that includes a length of the data capsule; and

forming the data capsule to include the header.

18. The method of claim 1, wherein the repository is coupled to the plurality of applications, wherein the repository includes a plurality of data capsules corresponding to the plurality of applications, the repository providing access to the plurality of data capsules by the plurality of applications, and wherein at least two applications of the plurality of applications are different applications.

19. The method of claim 1, wherein the repository provides state caching of a plurality of data capsules.

20. The method of claim 1, wherein the repository provides linear sequencing of a plurality of data capsules.

21. The method of claim 1, wherein the plurality of repositories includes an input pool.

22. The method of claim 21, wherein the type of the device event data comprises tracking data.

23. The method of claim 21, wherein the destination application is a gesture recognition application.

24. The method of claim 1, wherein the plurality of repositories includes an output pool.

25. The method of claim 24, wherein the type of the device event data comprises gestural event data.

26. The method of claim 24, wherein the destination application is a gesture-aware application.

27. The method of claim 1, wherein the data structure is untyped.

28. The method of claim 1, wherein the data structure of the data capsule provides a platform-independent representation of the device event data and the state information.

29. The method of claim 1, wherein the data structure of the data capsule provides platform-independent access to the device event data and the state information.

30. The method of claim 1, comprising:

transferring the data capsule from a first application having a first application type to at least one second application having at least one second application type, wherein the first application type is different than the second application type, and wherein the generating of the at least one data sequence was executed by the first application.

31. The method of claim 30, comprising using the at least one data sequence during operations of the second application.

32. The method of claim 1, wherein the event comprises a user interface event.

33. The method of claim 1, wherein the event comprises a graphics event.

34. The method of claim 1, wherein the event comprises depositing of state information.