Collaboration with 3D data visualizations

• US 10,359,905 B2
• Filed: 12/19/2014
• Issued: 07/23/2019
• Est. Priority Date: 12/19/2014
• Status: Expired due to Fees

• Current Assignee: Micro Focus LLC (Open Text Corporation)
• Original Assignee: entIT Software LLC (Open Text Corporation)
• Inventors: Cook, Gregory W., Hsiao, Chih-Pin, Wei, Jishang, Vankipuram, Mithra
• Primary Examiner(s): Balaoing, Ariel A
• Application Number:
  US15/534,258
• Publication Number:
  US 20170344220A1
• Time in Patent Office: 1,677 Days
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 2219/024 Multi-user, collaborative environment
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

An example collaboration system is provided in according with one implementation of the present disclosure. The system includes a 3D display a 3D data visualization, at least two hand avatars of two different users, and a view field avatar. The system also includes a plurality of auxiliary computing devices and a behavior analysis engine to perform a behavior analysis of a user. The behavior analysis engine is to: determine an attention engagement level of the user, and determine a pose of the user in relation to the auxiliary computing device. The system further includes an intention analysis engine to determine an intention of the user in relation to the 3D visualization based on the user'"'"'s attention engagement level and the user'"'"'s pose, and a collaboration engine to implement an action with the 3D data visualization by using a hand avatar based on the user'"'"'s intention and an identified gesture.

Claims

1. A collaboration system comprising:

a three-dimensional (3D) display displaying a 3D data visualization, at least two hand avatars of two different users, and a view field avatar, wherein the two hand avatars are of a first user and a collaborator, and wherein the view field avatar is of the collaborator;

a plurality of auxiliary computing devices;

a behavior analysis engine to perform a behavior analysis of a user, the behavior analysis engine to;

determine an attention engagement level of the user, and

determine a pose of the user in relation to an auxiliary computing device;

an intention analysis engine to determine an intention of the user in relation to the at least one 3D data visualization based on the user'"'"'s attention engagement level and the user'"'"'s pose;

an interaction mode engine to;

select, based on the determined user intention, an interaction mode of a plurality of interaction modes of the collaboration system, wherein each interaction mode is associated with a unique set of commands for interaction with the 3D data visualization, and

automatically adjust the collaboration system to the selected interaction mode; and

a collaboration engine to implement an action with the 3D data visualization based on the selected interaction mode and an identified gesture,

wherein the action is to couple a first user'"'"'s view field of the 3D data visualization with a collaborator'"'"'s view field of the 3D data visualization, when the system is in a navigation interaction mode and the collaboration engine is to detect a change of view gesture performed by the first user, and

wherein the change of view gesture by the first user is selecting the collaborator'"'"'s view field avatar with the first user'"'"'s hand avatar.

2. The collaboration system of claim 1, wherein the identified gesture indicates the implemented action while the collaboration system is in the selected interaction mode.

3. The collaboration system of claim 2, wherein another action is to select a portion of the 3D data visualization that is collaboratively shown to the first user and the collaborator, when the system is in a selection interaction mode and the collaboration engine is to detect a selection gesture with a hand avatar.

4. The collaboration system of claim 2, wherein another action is to save a collaborative view of the 3D data visualization, when the system is in a save interaction mode and the collaboration engine is to detect a save view gesture with a hand avatar.

11. The collaboration system of claim 1, wherein the plurality of interaction modes comprises a navigation interaction mode and a selection interaction mode.

5. A method comprising, by at least one processor:

identifying an intention of a user of a system in relation to a 3D virtual data object, the system including a plurality of computing devices and a 3D display displaying a 3D virtual data object at least two hand avatars of two different users, and a view field avatar;

selecting, based on the identified intention of the user, an interaction mode of a plurality of interaction modes of the system, wherein each interaction mode is associated with a unique set of commands for interaction with the 3D virtual data object;

transitioning the system to the selected interaction mode; and

executing an action with the 3D virtual data object by using a hand avatar based on the selected interaction mode and an identified gesture, comprising;

detecting a change of view gesture performed by a first user as the identified gesture, wherein the change of view gesture is a selection of a collaborator'"'"'s view field avatar with the first user'"'"'s hand avatar, and

coupling the first user'"'"'s view field of the 3D virtual data object with a collaborator'"'"'s view field of the 3D virtual data object when the system is in a navigation interaction mode.

6. The method of claim 5, wherein identifying the intention of a user in relation to the 3D virtual data object includes:

identifying an attention engagement level of the user;

identifying a pose of the user in relation to a computing device; and

determining the intention of the user in relation to the 3D virtual data object based on the user'"'"'s attention engagement level and the user'"'"'s pose.

7. The method of claim 5, further comprising:

detecting a save view gesture with a hand avatar, and saving a collaborative view of the 3D virtual data object when the system is in a save interaction mode.

12. The method of claim 5, wherein the identified gesture indicates the implemented action while the collaboration system is in the selected interaction mode.

13. The method of claim 5, wherein the plurality of interaction modes comprises a navigation interaction mode and a selection interaction mode.

8. A non-transitory machine-readable storage medium encoded with instructions executable by, at least one processor, the machine-readable storage medium comprising instructions to:

perform a behavior analysis of a user of a system including a plurality of computing devices and a 3D display displaying a 3D virtual data object, at least two hand avatars of two different users, and a view field avatar, the behavior analysis to;

identify an attention engagement level of the user, and

identify a pose of the user in relation to a computing device;

identify an intention of the user in relation to the virtual data object based on the user'"'"'s attention engagement level and the user'"'"'s pose;

select, based on the identified intention of the user, an interaction mode of a plurality of interaction modes of the system, wherein each interaction mode is associated with a unique set of commands for interaction with the 3D virtual data object;

transition the system to the selected interaction mode; and

implement an action with the 3D virtual data object by using a hand avatar based on the selected interaction mode and an identified gesture, comprising;

detect a change of view gesture performed by a first user as the identified gesture, wherein the change of view gesture includes a selection of a collaborator'"'"'s view field avatar with the first user'"'"'s hand avatar, and

couple the first user'"'"'s view field of the 3D virtual data object with a collaborator'"'"'s view field of the 3D virtual data object when the system is in a navigation interaction mode.

9. The non-transitory machine-readable storage medium of claim 8, wherein the identified gesture indicates the implemented action while the collaboration system is in the selected interaction mode.

10. The non-transitory machine-readable storage medium of claim 8, wherein the plurality of interaction modes comprises a navigation interaction mode and a selection interaction mode.