SYSTEMS AND METHODS FOR INTERACTING WITH TARGETS IN A BUILDING

• US 20180218540A1
• Filed: 01/16/2018
• Published: 08/02/2018
• Est. Priority Date: 01/30/2017
• Status: Abandoned Application

• Current Assignee: Johnson Controls Technology Company (Johnson Controls International plc)
• Original Assignee: Johnson Controls Technology Company (Johnson Controls International plc)
• Inventors: Sridharan, Ashok, S, Jeeva, Patil, Jayesh, Bhattacharya, Subrata, M, Shyam Sunder, Thareja, Ankur
• Application Number:
  US15/872,653
• Publication Number:
  US 20180218540A1
• Time in Patent Office: Days
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30108 Industrial image inspection
• CPC Class Codes: G06T 2219/2016 Rotation, translation, scaling
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: H04M 2250/52 including functional features of a camera
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources
• CPC Class Codes: H04W 4/02 Services making use of location information
• CPC Class Codes: H04W 88/02 Terminal devices

Abstract

A system for locating a target in a building includes a mobile application and a remote system. The mobile application is for implementation on a mobile device including a camera configured to be utilized by the mobile application to selectively obtain a first image data of a first environment. The remote system is configured to selectively receive the first image data from the mobile application. The remote system is also configured to compare, in response to receiving the first image data from the mobile application, the first image data to a database of targets. The remote system is also configured to determine if a portion of the first image data is indicative of a target in the database of targets.

Claims

1. A system for locating a target in a building, the system comprising:

a mobile application for implementation on a mobile device comprising a camera configured to be utilized by the mobile application to selectively obtain a first image data of a first environment; and

a remote system configured to;

selectively receive the first image data from the mobile application;

compare, in response to receiving the first image data from the mobile application, the first image data to a database of targets;

determine if a portion of the first image data is indicative of a target in the database of targets; and

transmit, in response to determining that a portion of the first image data is indicative of a determined target, a target indication to the mobile application, the target indication comprising a 3D model associated with the determined target;

wherein the mobile application is further configured to selectively provide, in response to receiving the target indication from the remote system, the 3D model on a display of a mobile device.

2. The system of claim 1, further comprising a building management system (BMS) communicable with a network and configured to:

receive the target indication from the network;

determine, in response to receiving the target indication, an operating parameter associated with the determined target; and

provide the operating parameter to the mobile application;

wherein the mobile application is further configured to selectively provide, in response to receiving the operating parameter from the BMS, the operating parameter on a display of a mobile device.

3. The system of claim 2, wherein the mobile application is further configured to provide the operating parameter on a display of a mobile device in response to receiving a selection made by a user in the mobile application.

4. The system of claim 2, wherein the mobile application is further configured to:

facilitate interaction with the determined target by a user, the interaction causing a change in the operating parameter; and

display the change in the operating parameter on the 3D model on a display of a mobile device.

5. The system of claim 1, wherein the mobile application is further configured to:

provide, in response to obtaining the first image data from the camera, the first image data on a display of a mobile device; and

overlay, in response to receiving the target indication from the remote system, the 3D model on the first image data on a display of a mobile device.

6. The system of claim 5, wherein:

the camera is further configured to be utilized by the mobile application to selectively obtain a second image data of a second environment; and

the mobile application is further configured to;

obtain the second image data from the camera;

provide, in response to obtaining the second image data from the camera, the second image data on a display of a mobile device; and

overlay the 3D model on the second image data on a display of a mobile device.

7. The system of claim 5, wherein the mobile application is further configured to:

receive a selection made by a user in the mobile application; and

at least one of;

rotate, in response to receiving the selection, the 3D model relative to the first image data; and

explode, in response to receiving the selection, the 3D model.

8. A system for locating a target in a building, the system comprising:

a mobile application for implementation on a mobile device and configured to communicate via a network, the mobile device comprising;

an imaging device configured to be utilized by the mobile application to selectively obtain image data of an environment;

a display configured to;

selectively provide the image data to a user; and

receive a first command from the user; and

a communications device configured to transmit, in response to receiving the first command from the user, the image data via the network; and

a remote system configured to communicate via the network and configured to;

selectively receive the image data from the mobile device via the network;

compare, in response to receiving the image data from the mobile device, the image data to a database of targets;

determine if a portion of the image data is indicative of a target in the database of targets; and

transmit, in response to determining that a portion of the image data is indicative of a determined target in the database of targets, a target indication to at least one of the mobile device or a building management system (BMS) via the network, the target indication comprising a 3D model associated with the target.

9. The system of claim 8, wherein the mobile application is configured to display, in response to receiving the target indication via the network, the 3D model on the display of the mobile device.

10. The system of claim 9, wherein the BMS is configured to:

determine, in response to receiving the target indication, an operating parameter associated with the determined target; and

provide the operating parameter to the mobile application via the network.

11. The system of claim 10, wherein the mobile application is configured to display, in response to receiving the operating parameter via the network, the operating parameter on the display of the mobile device while the 3D model is displayed on the display of the mobile device.

12. The system of claim 11, wherein:

the mobile application is further configured to provide a selectable button on the display, the selectable button corresponding with the operating parameter;

the display is configured to receive a second command from the user in response to a selection of the selectable button by the user, the second command different from the first command;

the communications device is further configured to transmit, in response to receiving the second command from the user, the second command to the BMS via the network; and

the BMS is configured to interact, in response to receiving the second command via the network, with the determined target to change the operating parameter according to the second command.

13. The system of claim 8, wherein:

the database of targets comprises at least one of edge detection data, shape recognition data, color detection data, and object recognition data for each target in the database of targets;

the remote system is further configured to obtain, in response to receiving the image data from the network, at least one of edge detection data, shape recognition data, color detection data, and object recognition data for the image data; and

the comparison performed by the remote system is a comparison of the at least one of edge detection data, shape recognition data, color detection data, and object recognition data for the image data with the at least one of edge detection data, shape recognition data, color detection data, and object recognition data for each target in the database of targets.

14. The system of claim 8, wherein the remote system is configured to determine if a portion of the image data is indicative of a target in the database of targets independent of any data provided by a marker present in the image data.

15. A system for locating a target in a building, the system comprising:

a mobile application for implementation on a mobile device and configured to;

obtain a first image data of a first environment;

provide the first image data to a display of the mobile device; and

transmit the first image data; and

a remote system communicable with the mobile application and configured to;

receive the first image data from the mobile application

compare the first image data to a database of targets;

determine if a portion of the first image data is indicative of a target in the database of targets; and

transmit, in response to determining that a portion of the first image data is indicative of a determined target in the database of targets, a target indication to the mobile application;

wherein the mobile application is configured to provide the target indication on the display of the mobile device.

16. The system of claim 15, wherein the remote system is configured to determine that a portion of the first image data is indicative of the determined target in the database of targets independent of any data provided by a marker present in the first image data.

17. The system of claim 15, wherein the mobile application is further configured to:

provide the first image data on the display of the mobile device; and

overlay the target indication on the first image data on the display of the mobile device.

18. The system of claim 17, wherein the mobile application is further configured to:

obtain a second image data of a second environment;

provide the second image data on the display of the mobile device in place of the first image data; and

overlay the target indication on the second image data on the display of the mobile device.

19. The system of claim 15, further comprising a building management system (BMS) communicable with the mobile application and configured to:

receive the target indication from the mobile application;

determine an operating parameter associated with the determined target; and

provide the operating parameter to the mobile application;

wherein the mobile application is further configured to selectively provide, in response to receiving the operating parameter from the BMS, the operating parameter on the display of the mobile device.

20. The system of claim 19, wherein the mobile application is further configured to:

facilitate interaction with the determined target by a user, the interaction causing a change in the operating parameter; and

display the change in the operating parameter on the 3D model on the display of the mobile device.