Hands-On Laboratory and Demonstration Equipment with a Hybrid Virtual/Augmented Environment, Along With Their Methods of Use

• US 20190318659A1
• Filed: 04/15/2019
• Published: 10/17/2019
• Est. Priority Date: 04/14/2018
• Status: Active Grant

• Current Assignee: The California State University - San Marcos
• Original Assignee: The California State University - San Marcos
• Inventors: Hamadani, Kambiz, Ahmadinia, Ali, Ye, Xin, Jiang, Yuanyuan
• Granted Patent: US 11,694,575 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09B 19/00 Teaching not covered by other main groups of this subclass teaching or practice apparatus for gun-aiming or gun-laying F41G3/26
• CPC Class Codes: G09B 23/28 for medicine

Abstract

Systems and methods for utilizing laboratory equipment in a hybrid reality environment or augmented virtual reality environment, are contemplated herein and include: at least one piece of laboratory equipment having at least one feature, wherein the at least one piece of laboratory equipment is tracked; a tracking module for tracking a position and an orientation of the at least one piece of tracked laboratory equipment; a virtual model, stored in a memory, comprising at least one 3-D virtual representation of the at least one feature of the at least one piece of laboratory equipment; and an experimentation module. A piece of laboratory equipment for use in a hybrid reality environment or augmented virtual reality environment is also included that comprises at least one piece of laboratory equipment having at least one feature, wherein the at least one piece of laboratory equipment is tracked; at least one marker that is coupled with the at least one piece of laboratory equipment; and a tracking module for tracking a position and an orientation of the at least one piece of tracked laboratory equipment, wherein the tracking module accesses the at least one marker.

Claims

1. A system for utilizing laboratory equipment in a hybrid reality environment, comprising:

at least one piece of laboratory equipment having at least one feature, wherein the at least one piece of laboratory equipment is tracked;

a tracking module for tracking a position and an orientation of the at least one piece of tracked laboratory equipment;

a virtual model, stored in a memory, comprising at least one 3-D virtual representation of the at least one feature of the at least one piece of laboratory equipment;

an experimentation module that;

accesses the virtual model stored in the memory;

receives input from the tracking module regarding the position and the orientation of the at least one piece of tracked laboratory equipment;

determines, based on the input from the tracking module, a corresponding interaction between the at least one piece of laboratory equipment and at least one additional piece of laboratory equipment, at least a portion or part of a user, or a combination thereof;

determines a consequence of the corresponding interaction; and

renders, to a display, a hybrid representation comprising a particular set of the at least one 3-D virtual representation, at least a part or portion of the user, the corresponding interaction, the consequence of the corresponding interaction, or a combination thereof.

2. The system of claim 1, further comprising at least one piece of laboratory equipment, laboratory solid material, laboratory liquid material, laboratory gaseous material, or a combination thereof, wherein the at least one piece of laboratory equipment, laboratory solid material, laboratory liquid material, laboratory gaseous material, or a combination thereof is a physical actual model representation, a virtual model representation or a combination thereof.

3. The system of claim 1, further comprising at least one piece of additional laboratory equipment, laboratory solid material, laboratory liquid material, laboratory gaseous material, or a combination thereof, wherein the at least one piece of additional laboratory equipment, laboratory solid material, laboratory liquid material, laboratory gaseous material, or a combination thereof is a physical actual model representation, a virtual model representation or a combination thereof.

4. The system of claim 2, wherein the at least one piece of laboratory equipment, laboratory solid material, laboratory liquid material, laboratory gaseous material, or a combination thereof is a virtual model representation.

5. The system of claim 3, wherein the at least one piece of additional laboratory equipment, laboratory solid material, laboratory liquid material, laboratory gaseous material, or a combination thereof is a virtual model representation.

6. The system of claim 1, wherein the hybrid reality environment includes an augmented virtual reality environment.

7. The system of claim 1, further comprising at least one marker that is coupled with the at least one piece of laboratory equipment;

and

wherein the tracking module accesses the at least one marker.

8. A system for utilizing laboratory equipment in a hybrid reality environment, comprising:

at least one first piece of laboratory equipment having at least one feature, wherein the at least one piece of laboratory equipment is tracked;

a tracking module for tracking a position and an orientation of the at least one piece of tracked laboratory equipment;

a virtual model, stored in a memory, comprising at least one 3-D virtual representation of at least one feature of an additional piece of laboratory equipment, at least a portion of a part of a user, or a combination thereof;

an experimentation module that;

accesses the virtual model stored in the memory;

receives input from the tracking module regarding the position and the orientation of the at least one first piece of tracked laboratory equipment;

determines, based on the input from the tracking system, a corresponding interaction between the at least one piece of laboratory equipment and at least one additional piece of laboratory equipment, at least a portion or part of a user, or a combination thereof;

determines a consequence of the corresponding interaction; and

renders, to a display, a hybrid representation comprising the physical visual representation of the at least one piece of tracked laboratory equipment, at least one 3-D virtual representation, at least a part or portion of the user, the corresponding interaction, the consequence of the corresponding interaction, or a combination thereof.

9. The system of claim 8, further comprising at least one piece of laboratory equipment, laboratory solid material, laboratory liquid material, laboratory gaseous material, or a combination thereof, wherein the at least one piece of laboratory equipment, laboratory solid material, laboratory liquid material, laboratory gaseous material, or a combination thereof is a physical actual model representation, a virtual model representation or a combination thereof.

10. The system of claim 8, further comprising at least one piece of additional laboratory equipment, laboratory solid material, laboratory liquid material, laboratory gaseous material, or a combination thereof, wherein the at least one piece of additional laboratory equipment, laboratory solid material, laboratory liquid material, laboratory gaseous material, or a combination thereof is a physical actual model representation, a virtual model representation or a combination thereof.

11. The system of claim 9, wherein the at least one piece of laboratory equipment, laboratory solid material, laboratory liquid material, laboratory gaseous material, or a combination thereof is a virtual model representation.

12. The system of claim 10, wherein the at least one piece of additional laboratory equipment, laboratory solid material, laboratory liquid material, laboratory gaseous material, or a combination thereof is a virtual model representation.

13. The system of claim 8, wherein the hybrid reality environment includes an augmented virtual reality environment.

14. The system of claim 8, further comprising at least one marker that is coupled with the at least one piece of laboratory equipment;

and

wherein the tracking module accesses the at least one marker.

15. A piece of laboratory equipment for use in a hybrid reality environment, comprising:

at least one piece of laboratory equipment having at least one feature, wherein the at least one piece of laboratory equipment is tracked;

at least one marker that is coupled with the at least one piece of laboratory equipment; and

a tracking module for tracking a position and an orientation of the at least one piece of tracked laboratory equipment, wherein the tracking module accesses the at least one marker.

16. The piece of laboratory equipment for use in a hybrid reality environment of claim 15, wherein the piece of laboratory equipment comprises at least one laboratory solid material, at least one laboratory liquid material, at least one laboratory gaseous material, or a combination thereof, wherein the at least one piece of laboratory equipment, laboratory solid material, laboratory liquid material, laboratory gaseous material, or a combination thereof is a virtual model representation.

17. The piece of laboratory equipment for use in a hybrid reality environment of claim 15, wherein the piece of laboratory equipment comprises a pipetteman, a beaker, a sample tube, a graduated cylinder, a scale, or a combination thereof.