Adaptive tracking system for spatial input devices

• US 10,255,489 B2
• Filed: 03/20/2018
• Issued: 04/09/2019
• Est. Priority Date: 04/24/2008
• Status: Active Grant

• Current Assignee: Oblong Industries, Inc. (Oblong, Inc.)
• Original Assignee: Oblong Industries, Inc. (Oblong, Inc.)
• Inventors: Csaszar, Ambrus, Kogan, Dima, Yarin, Paul
• Primary Examiner(s): Shankar, Vijay
• Application Number:
  US15/926,280
• Publication Number:
  US 20180218205A1
• Time in Patent Office: 385 Days
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: G16C 10/00 Computational theoretical chemistry, i.e. ICT specially adapted for theoretical aspects of quantum chemistry, molecular mechanics, molecular dynamics or the like
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

An adaptive tracking system for spatial input devices provides real-time tracking of spatial input devices for human-computer interaction in a Spatial Operating Environment (SOE). The components of an SOE include gestural input/output; network-based data representation, transit, and interchange; and spatially conformed display mesh. The SOE comprises a workspace occupied by one or more users, a set of screens which provide the users with visual feedback, and a gestural control system which translates user motions into command inputs. Users perform gestures with body parts and/or physical pointing devices, and the system translates those gestures into actions such as pointing, dragging, selecting, or other direct manipulations. The tracking system provides the requisite data for creating an immersive environment by maintaining a model of the spatial relationships between users, screens, pointing devices, and other physical objects within the workspace.

Claims

1. A system comprising:

computing hardware of a spatial operating system;

at least one sensor; and

a plurality of display devices,

wherein the computing hardware is constructed to model a 6-degree-of-freedom pose and physical size of each of the plurality of display devices by using the at least one sensor,

wherein the computing hardware is constructed to generate a coherent model of spatial relationships based on the modeled 6-degree-of-freedom poses and physical sizes of the display devices, and

wherein the coherent model of spatial relationships includes a spatial relationship between a first display device of the plurality of display devices and a second display device of the plurality of display devices.

2. The system of claim 1, wherein the at least one sensor includes at least one camera.

3. The system of claim 1, wherein the computing hardware is a pre-processor.

4. The system of claim 1, wherein the computing hardware is a tracking component.

5. The system of claim 1, wherein the at least one sensor is included in a plurality of sensors, and wherein locations of the plurality of sensors define a spatial operating environment (SOE).

6. The system of claim 1, wherein the computing hardware is constructed to provide at least one application running on an external device with access to the coherent model of spatial relationships.

7. The system of claim 1, wherein each of the plurality of display devices is communicatively coupled to the computing hardware and controlled by the computing hardware.

8. The system of claim 1, wherein at least one of the plurality of display devices is communicatively coupled to the computing hardware and controlled by the computing hardware, and at least one of the plurality of display devices is controlled by a computer that is independent from the computing hardware.

9. The system of claim 1, wherein the computing hardware is constructed to update the coherent model of spatial relationships responsive to repositioning of at least one display device of the plurality of display devices.

10. A method comprising:

computing hardware of a spatial operating system modeling a 6-degree-of-freedom pose and physical size of each of a plurality of display devices by using at least one sensor; and

the computing hardware generating a coherent model of spatial relationships based on the modeled 6-degree-of-freedom poses and physical sizes of the display devices,

wherein the coherent model of spatial relationships includes a spatial relationship between a first display device of the plurality of display devices and a second display device of the plurality of display devices.

11. The method of claim 10, wherein the at least one sensor includes at least one camera.

12. The method of claim 10, wherein the computing hardware is a pre-processor.

13. The method of claim 10, wherein the computing hardware is a tracking component.

14. The method of claim 10, wherein the at least one sensor is included in a plurality of sensors, and wherein locations of the plurality of sensors define a spatial operating environment (SOE).

15. The method of claim 10, further comprising, the computing hardware providing at least one application running on an external device with access to the coherent model of spatial relationships.

16. The method of claim 10, wherein each of the plurality of display devices is communicatively coupled to the computing hardware and controlled by the computing hardware.

17. The method of claim 10, wherein at least one of the plurality of display devices is communicatively coupled to the computing hardware and controlled by the computing hardware, and at least one of the plurality of display devices is controlled by a computer that is independent from the computing hardware.

18. The method of claim 10, further comprising, the computing hardware updating the coherent model of spatial relationships responsive to repositioning of at least one display device of the plurality of display devices.

19. The method of claim 10, further comprising, the computing hardware updating the coherent model of spatial relationships responsive to repositioning of at least one display device of the plurality of display devices as determined by using the at least one sensor.

20. The method of claim 10, wherein the computing hardware is constructed to update the coherent model of spatial relationships responsive to repositioning of at least one display device of the plurality of display devices as determined by using the at least one sensor.