Methods and systems for multiple access to a single hardware data stream

• US 10,623,721 B2
• Filed: 06/12/2019
• Issued: 04/14/2020
• Est. Priority Date: 10/02/2015
• Status: Active Grant

• Current Assignee: West Texas Technology Partners LLC
• Original Assignee: Atheer, Inc.
• Inventors: Fateh, Sina, Abercrombie, Nathan, Itani, Sleiman
• Primary Examiner(s): Dang, Philip P.
• Application Number:
  US16/438,770
• Publication Number:
  US 20190297318A1
• Time in Patent Office: 307 Days
• CPC Class Codes: G02B 2027/0134 of stereoscopic type
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/327 Calibration thereof
• CPC Class Codes: H04N 13/332 Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays
• CPC Class Codes: H04N 13/383 for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
• CPC Class Codes: H04N 2213/002 Eyestrain reduction by processing stereoscopic signals or controlling stereoscopic devices

Abstract

A target is outputted to an ideal position in 3D space. A viewer indicates the apparent position of the target, and the indication is sensed. An offset between the ideal and apparent positions is determined, and an adjustment determined from the offset such that the apparent position of the ideal position with the adjustment matches the ideal position without the adjustment. The adjustment is made to the first entity and/or a second entity, such that the entities appear to the viewer in the ideal position. The indication may be monocular with a separate indication for each eye, or binocular with a single viewer indication for both eyes. The indication also may serve as communication, such as a PIN input, so that calibration is transparent to the viewer. The method may be continuous, intermittent, or otherwise ongoing over time.

Claims

1. A method, comprising:

defining, by a processing device, a first coordinate in a three-dimensional (3D) space to display a virtual object to a viewer;

displaying, by a display, the virtual object at the first coordinate in the 3D space;

determining, by a sensor, that an end-effector interacting with the virtual object at a first point in time is pointing to a second coordinate in the 3D space that is different than the first coordinate where the virtual object is displayed;

determining, by the processing device, an offset value between the first coordinate where the virtual object is located and the second coordinate where the end-effector is pointing to, wherein the offset value indicates a difference in the first coordinate and the second coordinate;

determining, by the processing device, a third coordinate in the 3D space, wherein the third coordinate is the first coordinate of the first coordinate adjusted by the offset value so that the viewer perceives the virtual object as being located at the first coordinate in the 3D space;

determining, by the sensor, that the end-effector interacting with the virtual object at a second point in time is pointing to a fourth coordinate in the 3D space that is different than the third coordinate where the virtual object is displayed; and

in response to the end-effector pointing to the fourth coordinate in the 3D space that is not the third coordinate, iteratively adjusting, by the processing device, the fourth coordinate for the virtual object until the end-effector points to the first coordinate.

2. The method of claim 1, further comprises displaying the virtual object to a first display and a second display of a stereo display pair.

3. The method of claim 1, further comprises:

displaying the virtual object at a first location at a first display of a stereo display pair; and

displaying the virtual object in a second location at a second display of the stereo display pair.

4. The method of claim 2, wherein the first coordinate is a first two-dimensional coordinate and the second coordinate is a second two-dimensional coordinate.

5. The method of claim 1, further comprises:

displaying the virtual object at a first location to a first eye of the viewer but not a second eye of the viewer; and

displaying the virtual object at a second location to the second eye of the viewer but not the first eye of the viewer.

6. The method of claim 5, further comprises displaying the virtual object simultaneously or at different points in time to the first eye of the viewer and the second eye of the viewer.

7. The method of claim 5, further comprises displaying the virtual object sequentially to the first eye of the viewer and the second eye of the viewer.

8. The method of claim 1, further comprises:

displaying nothing to a second eye of the viewer while displaying the virtual object to the first eye of the viewer; and

displaying nothing to the first eye of the viewer while displaying the virtual object to the second eye of the viewer.

9. The method of claim 1, further comprises:

obstructing or partially obstructing a second eye of the viewer while displaying the virtual object to the first eye of the viewer; and

obstructing or partially obstructing the first eye of the viewer while displaying the virtual object to the second eye of the viewer.

10. The method of claim 1, further comprises sensing at least one of a posture or a gesture of the viewer.

11. A method, comprising:

defining, by a processing device, a first coordinate in an augmented reality environment or a virtual reality environment;

displaying, by a display, a first virtual object to a viewer at the first coordinate;

determining, by the processing device, that an end-effector interacting with the first virtual object is pointing to a second coordinate in the augmented reality environment or the virtual reality environment, wherein the second coordinate is different than the first coordinate;

determining, by the processing device, a difference between the first coordinate where the first virtual object is located and the second coordinate where the end-effector is pointing to;

determining, by the processing device, a third coordinate in the augmented reality environment or the virtual reality environment, wherein the third coordinate is the first coordinate of the first coordinate adjusted by the determined difference;

generating a second virtual object;

defining, by the processing device, a fourth coordinate to display the second virtual object to the viewer in the augmented reality environment or the virtual reality environment, wherein the fourth coordinate is based on the determined difference; and

displaying, by the display, the first virtual object at the third coordinate and the second virtual object at the fourth coordinate.

12. The method of claim 11, further comprising sequentially displaying the first virtual object and the second virtual object.

13. The method of claim 11, further comprising:

displaying the first virtual object by a first display while outputting nothing by a second display; and

displaying the second virtual object by the second display while outputting nothing by the first display.

14. The method of claim 11, further comprising:

displaying the first virtual object by a first display while obstructing or partially obstructing a second display; and

displaying the second virtual object by the second display while obstructing or partially obstructing the first display.

15. The method of claim 11, further comprising sensing a viewer first indication of the first virtual object and a viewer second indication of the second virtual object.

16. The method of claim 11, further comprising displaying the first virtual object by a first display and simultaneously or at different points in time displaying the second virtual object by a second display.

17. An apparatus comprising:

a first three-dimensional (3D) display operable to output a virtual object at a first coordinate in a 3D space to a first eye of a viewer;

a second 3D display operable to output the virtual object at a second coordinate in the 3D space to a second eye of the viewer, wherein the first coordinate is different than the second coordinate;

a first sensor configured to measure a difference between a location of an end-effector in the 3D space interacting with the virtual object at the first coordinate relative to the first eye of the viewer;

a second sensor configured to measure a difference between the location of the end-effector in the 3D space interacting with the virtual object at the second coordinate relative to the second eye of the viewer; and

a processing device coupled to the first 3D display, the second 3D display, the first sensor, and the second sensor, wherein the processing device is operable to;

determine that the end-effector is pointing to a third coordinate on the first 3D display that is different than the first coordinate where the virtual object is displayed at a first point in time;

determine that the end-effector is pointing to a fourth coordinate on the second 3D display that is different than the second coordinate where the virtual object is displayed at the first point in time;

determine a first difference between the first coordinate where the virtual object is located and the third coordinate where the end-effector is pointing to on a first display;

determine a second difference between the second coordinate where the virtual object is located and the fourth coordinate where the end-effector is pointing to on a second display;

determine a fifth coordinate in the 3D space that is the first coordinate adjusted by the first difference so that the viewer perceives the virtual object as being located at the first coordinate on the first 3D display, wherein the first 3D display is to display the virtual object at the fifth coordinate on the first 3D display at a second point in time; and

determine a sixth coordinate in the 3D space that is the second coordinate adjusted by the second difference so that the viewer perceives the virtual object as being located at the second coordinate on the second 3D display, wherein the second 3D display is to display the virtual object at the sixth coordinate on the second 3D display at the second point in time.

18. The apparatus of claim 17, wherein the first 3D display and the second 3D display are a stereo display.

19. The apparatus of claim 17, wherein the first 3D display or the second 3D display comprises a see-through display.

20. The apparatus of claim 17, wherein the first sensor or the second sensor comprises an imager.