Adaptive tracking system for spatial input devices

• US 9,984,285 B2
• Filed: 07/07/2017
• Issued: 05/29/2018
• Est. Priority Date: 04/24/2008
• Status: Expired due to Fees

• Current Assignee: Oblong Industries, Inc. (Oblong, Inc.)
• Original Assignee: Oblong Industries, Inc. (Oblong, Inc.)
• Inventors: Csaszar, Ambrus, Kogan, Dima, Yarin, Paul
• Primary Examiner(s): Shankar, Vijay
• Application Number:
  US15/644,092
• Publication Number:
  US 20170308743A1
• Time in Patent Office: 326 Days
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: G16C 10/00 Computational theoretical chemistry, i.e. ICT specially adapted for theoretical aspects of quantum chemistry, molecular mechanics, molecular dynamics or the like
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

An adaptive tracking system for spatial input devices provides real-time tracking of spatial input devices for human-computer interaction in a Spatial Operating Environment (SOE). The components of an SOE include gestural input/output; network-based data representation, transit, and interchange; and spatially conformed display mesh. The SOE comprises a workspace occupied by one or more users, a set of screens which provide the users with visual feedback, and a gestural control system which translates user motions into command inputs. Users perform gestures with body parts and/or physical pointing devices, and the system translates those gestures into actions such as pointing, dragging, selecting, or other direct manipulations. The tracking system provides the requisite data for creating an immersive environment by maintaining a model of the spatial relationships between users, screens, pointing devices, and other physical objects within the workspace.

Claims

1. A method comprising:

a tracking component of a gestural control system receiving feature data of a first object of a plurality of objects included in a spatial operating environment (SOE) from at least one sensor of a plurality of sensors of the SOE, wherein locations of each of the plurality of sensors define the SOE; and

the tracking component generating from the received feature data of the first object a coherent model of spatial relationships that includes a spatial relationship between the first object and a second object of the plurality of objects.

2. The method of claim 1, wherein each sensor is a camera.

3. The method of claim 1, wherein at least one sensor is a camera.

4. The method of claim 1, wherein the tracking component receives the feature data of the first object from one sensor of the plurality of sensors, and wherein the one sensor is a camera.

5. The method of claim 1, wherein the plurality of sensors includes two or more cameras, and wherein the tracking component receiving the feature data of the first object comprises the tracking component receiving the feature data of the first object from two or more cameras of the plurality of sensors.

6. The method of claim 1 wherein the first object is a wand.

7. The method of claim 1 wherein the first object is an input device.

8. The method of claim 1 wherein the first object is display device.

9. The method of claim 1 wherein the first object is mobile screen.

10. The method of claim 1, further comprising the tracking component receiving feature data of the second object of the plurality of objects included in the SOE from at least one sensor of the plurality of sensors of the SOE.

11. The method of claim 1, wherein the first object is a wand and the second object is a wand.

12. The method of claim 1, wherein the first object is an input device and the second object is an input device.

13. The method of claim 1, wherein the first object is a wand and the second object is a display device.

14. The method of claim 1, wherein the first object is an input device and the second object is a display device.

15. The method of claim 1, wherein the first object is a wand and the second object is a mobile screen.

16. The method of claim 1, wherein the feature data of the first object is absolute three-space location data of an instantaneous state of the first object at a point in time and space.

17. The method of claim 1, wherein the tracking component receives the feature data of the first object from at least a first sensor and a second sensor of the plurality of sensors.

18. The method of claim 17, wherein the coherent model of spatial relationships includes a spatial relationship between the first sensor and the second sensor.

19. The method of claim 18, wherein the feature data of the first object is absolute three-space location data of an instantaneous state of the first object at a point in time and space.

20. The method of claim 1, wherein the gestural control system includes the plurality of sensors.