System and method for role-switching in multi-reality environments

• US 9,710,968 B2
• Filed: 12/26/2012
• Issued: 07/18/2017
• Est. Priority Date: 12/26/2012
• Status: Active Grant

• Current Assignee: Help Lightning, Inc.
• Original Assignee: Help Lightning, Inc.
• Inventors: Dillavou, Marcus W., Deaton, Drew Steven, May, Matthew Benton
• Primary Examiner(s): Ge, Jin
• Application Number:
  US13/727,202
• Publication Number:
  US 20140176533A1
• Time in Patent Office: 1,665 Days
• Field of Search: 345633
• CPC Class Codes: A61B 2017/00216 with eye tracking or head position tracking control
• CPC Class Codes: A61B 2034/258 providing specific settings for specific users
• CPC Class Codes: A61B 2090/502 Headgear, e.g. helmet, spectacles
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2219/024 Multi-user, collaborative environment

Abstract

Provided herein are methods and systems for role designation with multiple sources. A method for role designation can comprise rendering a common field of interest that reflects a presence of a plurality of elements, wherein at least one of the elements is a remote element located remotely from another of the elements, receiving a role designation, and updating the common field of interest based upon the role designation.

Claims

1. A method for role designation comprising:

rendering a common field of interest via a local device comprising a display that reflects a presence of a plurality of elements, wherein at least one of the elements is a remote element located remotely from another of the elements;

rendering a graphical user interface comprising a user engageable element indicating a role designation associated with one or more pre-defined, stored parameters;

receiving a selection of the role designation via the user engageable element of the graphical user interface, wherein the selection of the role designation initiates transmission of a role change request and causes transmission of an indication of a role change to a remote device rendering the common field of interest;

upon determining the selected role designation associated with the one or more pre-defined, stored parameters, retrieving the stored parameters associated with the selected role designation; and

updating the common field of interest based upon at least the retrieved stored parameters;

outputting the updated common field of interest to the display via the graphical user interface, such that merged video presented via the local device has a display characteristic that is distinct from a display characteristic of the merged video presented via the remote device, wherein the display characteristic of the merged video presented via the remote device is dependent upon the selected role designation; and

wherein the selected role designation from the local device comprises one of a helper role and an operator role, and wherein the selected role designation from the remote device comprises one of the helper role and the operator role not currently selected by the local device.

2. The method of claim 1, further comprising the step of rendering interaction between the elements in the common field of interest, wherein the rendering of the interactions are controlled based at least upon the retrieved stored parameters.

3. The method of claim 1, wherein each of the elements in the common field of interest is rendered to a remote viewer and a local viewer with a substantially similar alignment and orientation.

4. The method of claim 1, wherein the retrieved stored parameters comprise at least an opacity parameter and further comprising rendering one or more graphical controls configured to control the visual appearance of the common field of interest.

5. The method of claim 1, wherein when the operator role is selected, one or more images captured by the local camera and processed by the local processor are rendered either with a degree of transparency or with substantially full opacity.

6. The method of claim 5, wherein when the operator role is selected, one or more images captured by the remote camera and processed by the remote processor are rendered with a degree of transparency.

7. The method of claim 1, wherein a heads up role designation may be selected via the user engageable element of the graphical user interface, where one or more local images may be rendered in a display screen of the local device, with one or more remote images displayed in a smaller window, or one or more remote images may be rendered in a majority of the display screen of the local viewer, with one or more local images displayed in a smaller window.

8. The method of claim 1, wherein the display characteristic of the merged video presented via the local device is dependent upon the selected role designation.

9. A method for role designation comprising:

generating a first image representing a first element;

generating a second image representing a second element disposed remotely from the first element;

rendering an interface element to one or more of a local user and a remote user relating to a plurality of stored role designations, each of the stored role designations having a set of pre-defined display parameters associated therewith, wherein the interface element is rendered to a local user via a local device comprising a display that reflects a presence of a plurality of elements and at least one of the elements is a remote element located remotely from another of the elements, and

receiving a selection of the interface element representing a selection of a role designation from the plurality of stored role designations, wherein the selection is received from the one or more of the local user and the remote user, wherein the selection of the role designation initiates transmission of a role change request and causes transmission of an indication of the role change request to a remote device rendering a common field of interest;

upon determining the selected role designation having a set of pre-defined display parameters, retrieving the parameters associated with the selected role designation;

updating the interface element based upon at least the retrieved pre-defined display parameters;

generating, in response to the role change request, a composite image including the first image and the second image;

outputting the composite image to the display,

wherein a display characteristic of at least one of the first image and the second image relative to the other of the first image and the second image is modified based upon the set of pre-defined display parameters associated with the selected role designation,

wherein the composite image presented via the local device has a display characteristic that is distinct from a display characteristic of the composite image presented via the remote device, wherein the display characteristic of the composite image presented via the remote device is dependent upon the selected role designation; and

wherein the selected role designation from the local user comprises one of a helper role and an operator role, and wherein the selected role designation from the remote user comprises one of the helper role and the operator role not currently selected by the local user.

10. The method according to claim 9, wherein the display characteristic is opacity.

11. The method according to claim 9, wherein the display characteristic is an orientation or arrangement of the one or more of the first element and the second element.

12. The method according to claim 9, wherein the method further comprises outputting the composite image to a remote display,

wherein the remote display reflects a presence of a plurality of elements and at least one of the elements is a remote element located remotely from another of the elements,

wherein the display characteristic of the composite image presented via the remote device is dependent upon the selected role designation, and

wherein one or more of the first element and the second element is a representation of the physical presence of one or more of the remote user and the local user.

13. The method of claim 9, wherein at least one of the first image and the second image is part of a video stream.

14. The method of claim 9, wherein at least one of the first image and the second image is part of a live video stream.

15. The method of claim 9, wherein a heads up role designation may be selected via the interface element, where one or more local images may be rendered in a display screen of the local user, with one or more remote images displayed in a smaller window, or one or more remote images may be rendered in a majority of the display screen of the local user, with one or more local images displayed in a smaller window.

16. A system for role designation comprising:

a display configured for displaying a common field of interest;

a sensor configured for obtaining image data;

a processor in signal communication with the display and the sensor,

wherein the processor is configured to perform steps comprising,

rendering the common field of interest via a local device that reflects a presence of a plurality of elements, wherein at least one of the elements is a remote element located remotely from another of the elements;

rendering a graphical user interface comprising a user engageable element indicating a role designation associated with one or more pre-defined, stored parameters;

receiving a selection of the role designation via the user engageable element of the graphical user interface,

wherein the selection of the role designation initiates transmission of a role change request and causes transmission of an indication of a role change to a remote device rendering the common field of interest;

upon determining the selected role designation associated with one or more pre-defined, stored parameters, retrieving stored parameters associated with the selected role designation;

updating the common field of interest based upon at least the retrieved stored parameters such that merged video presented via the local device has a display characteristic that is distinct from a display characteristic of merged video presented via the remote device, wherein the display characteristic of the merged video presented via the remote device is dependent upon the selected role designation; and

outputting the updated common field of interest to the display via the graphical user interface,

wherein the selected role designation from the local device comprises one of a helper role and an operator role, and wherein the selected role designation from the remote device comprises one of the helper role and the operator role not currently selected by the local device.

17. The system of claim 16, wherein the sensor is at least one of a camera, an infrared sensor, a light sensor, a RADAR device, a SONAR device, a computed tomography device, a magnetic resonance imaging device, and a depth scan sensor.

18. The system of claim 16, wherein the processor is further configured to continuously update the common field of interest such that the presence of the remote element is rendered in real time to a local viewer.

19. The system of claim 16, wherein the processor is further configured to continuously update the common field of interest such that the presence of a local one of the elements is rendered in real time to a remote viewer.

20. The method of claim 16, wherein a heads up role designation may be selected via the user engageable button of the graphical user interface, where one or more local images may be rendered in a display screen of the local device, with one or more remote images displayed in a smaller window, or one or more remote images may be rendered in a majority of the display screen of the local device, with one or more local images displayed in a smaller window.