Augmented reality tools for PV solar and/or storage system site survey, installation, commissioning, and operation and maintenance supported by on-site data collection

• US 9,858,727 B2
• Filed: 10/06/2016
• Issued: 01/02/2018
• Est. Priority Date: 01/22/2016
• Status: Active Grant

• Current Assignee: Locus Energy, LLC (Also Energy Incorporated)
• Original Assignee: Locus Energy, LLC (Also Energy Incorporated)
• Inventors: Ramachandran, Anil, Herzig, Michael, Kerrigan, Shawn
• Primary Examiner(s): CRADDOCK, ROBERT J
• Application Number:
  US15/287,001
• Publication Number:
  US 20170213390A1
• Time in Patent Office: 453 Days
• Field of Search: 345633
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G09G 2370/022 Centralised management of display operation, e.g. in a server instead of locally
• CPC Class Codes: G09G 2380/00 Specific applications

Abstract

A system and method in which information relevant to a Photovoltaic (PV) solar-site specific activity environment is displayed on the display screen of an Augmented Reality (AR) device as part of the surrounding environment, instead of on a separate computing device or an installation manual. The activity environment includes PV solar site-specific survey and feasibility analysis, installation and commissioning, Operation and Maintenance (O&M), and site overhaul/removal activities. Thus, relevant information is available to an installer/technician at their fingertips so that the installer/technician can continue the task they are involved in without losing focus. The AR device may record relevant data during site survey and note important points, accelerate installation and commissioning, make O&M more efficient, and record the complete process for ongoing improvement/management of the site. Moreover, company-specific best practices may be loaded on the AR device as guidelines to make the process uniform across the entire fleet of installers.

Claims

1. An Augmented Reality (AR) device operable to perform the following:

monitor and analyze visual content associated with a visible field of view of the AR device to self-determine that the AR device is present at a Photovoltaic (PV) solar site, wherein the field of view includes a visible portion of a site-specific activity environment at the PV solar site, and wherein the site-specific activity environment includes one of the following PV site-specific activities;

PV site survey and feasibility analysis activity,

PV site installation and commissioning activity,

PV site Operations and Maintenance (O&

M) activity, and

PV site overhaul activity;

based on an analysis of the visual content, determine the site-specific activity environment surrounding the AR device; and

display information related to the site-specific activity environment on a display screen of the AR device.

2. The AR device of claim 1, wherein the visual content is at least one of the following:

still images; and

video clips.

3. The AR device of claim 1, wherein the AR device is operable to

have at least some of the visual content processed and analyzed by an external source.

4. The AR device of claim 1, wherein the AR device is operable to store the following in a memory of the AR device:

one or more scenes visible through the field of view of the AR device; and

an indication identifying the PV solar site-specific activity related information being displayed on the display screen of the AR device.

5. The AR device of claim 1, wherein the information related to the site-specific activity environment includes at least one of the following:

first activity-specific data received from an external source for an activity in the site-specific activity environment;

second activity-specific data observed through the visible field of view of the AR device by a user of the AR device for the activity in the site-specific activity environment; and

third activity-specific data recorded by the AR device for the activity in the site-specific activity environment.

6. The AR device of claim 1, wherein the AR device is operable to further perform one of the following:

display at least a first portion of the information related to the site-specific activity environment overlaid on the display screen on top of a field of view-based scene of the PV solar site; and

display at least a second portion of the information related to the site-specific activity environment on the display screen without displaying other content on the display screen.

7. The AR device of claim 1, wherein the AR device is operable to further perform the following:

receive an input from a user of the AR device; and

display a user input-specific portion of the information related to the site-specific activity environment on the display screen of the AR device.

8. A non-transitory data storage medium operable with an Augmented Reality (AR) device and containing program instructions, which, when executed by the AR device, cause the AR device to perform the following:

capture and analyze visual content within a field of view of the AR device to self-determine that the AR device is present at a Photovoltaic (PV) solar site, wherein the field of view relates to a visible portion of a site-specific activity environment at the PV solar site, and wherein the site-specific activity environment includes one of the following PV site-specific activities;

PV site survey and feasibility analysis activity,

PV site installation and commissioning activity,

PV site Operations and Maintenance (O&

M) activity, and

PV site overhaul activity;

based on an analysis of the visual content, determine the site-specific activity environment surrounding the AR device; and

display information related to the site-specific activity environment on a display screen of the AR device.

9. The non-transitory data storage medium of claim 8, wherein the visual content is at least one of still images and video clips.

10. The non-transitory data storage medium of claim 9, wherein the program instructions, when executed by the AR device, cause the AR device to

have at least some of the captured visual content processed and analyzed by an external source.

11. The non-transitory data storage medium of claim 8, wherein the information related to the site-specific activity environment includes at least one of the following:

first activity-specific data received from an external source for an activity in the site-specific activity environment;

second activity-specific data obtained through the captured visual content for the activity in the site-specific activity environment; and

third activity-specific data recorded by the AR device for the activity in the site-specific activity environment.

12. The non-transitory data storage medium of claim 8, wherein the program instructions, when executed by the AR device, cause the AR device to further perform one of the following:

display at least a first portion of the information related to the site-specific activity environment overlaid on top of the visual content of the PV solar site; and

display at least a second portion of the information related to the site-specific activity environment on the display screen without the visual content of the PV solar site.

13. The non-transitory data storage medium of claim 8, wherein the program instructions, when executed by the AR device, cause the AR device to further perform the following:

receive an input from a user of the AR device; and

display a user input-specific portion of the information related to the site-specific activity environment on the display screen of the AR device.

14. The non-transitory data storage medium of claim 8, wherein the program instructions, when executed by the AR device, cause the AR device to store the following:

the visual content captured within the field of view of the AR device; and

an indication identifying the PV solar site-specific activity related information being displayed on the display screen of the AR device.

15. A method of facilitating management of a Photovoltaic (PV) solar site using an Augmented Reality (AR) device, wherein the method comprises:

the AR device monitoring a visible field of view of the AR device;

the AR device capturing and analyzing visual content within the field of view of the AR device;

based on an analysis of the visual content, the AR device self-determining that the AR device is present at the PV solar site;

the AR device identifying a site-specific activity environment at the PV solar site based on the analysis of the captured visual content, wherein the site-specific activity environment includes one of the following PV site-specific activities;

PV site survey and feasibility analysis activity,

PV site installation and commissioning activity,

PV site Operations and Maintenance (O&

M) activity, and

PV site overhaul activity; and

the AR device displaying information related to the site-specific activity environment on a display screen of the AR device.

16. The method of claim 15, further comprising:

the AR device processing and analyzing a first portion of the captured visual content to determine first activity-specific visual information for an activity in the site-specific activity environment;

the AR device sending a second portion of the captured visual content to an external source for processing and analysis;

the AR device receiving second activity-specific visual information from the external source for the activity in the site-specific activity environment; and

the AR device displaying the first and the second activity-specific visual information on the display screen of the AR device.

17. The method of claim 15, wherein the AR device displaying information related to the site-specific activity environment includes at least one of the following:

the AR device displaying at least a first portion of the information related to the site-specific activity environment overlaid on top of the field of view-based visual content of the PV solar site;

the AR device displaying at least a second portion of the information related to the site-specific activity environment on the display screen without the field of view-based visual content of the PV solar site; and

upon receiving an input from a user of the AR device, the AR device displaying a user input-specific portion of the information related to the site-specific activity environment on the display screen.