Remote sensor access and queuing

• US 9,578,399 B2
• Filed: 08/25/2014
• Issued: 02/21/2017
• Est. Priority Date: 08/25/2014
• Status: Active Grant

• Current Assignee: META Platforms Technologies LLC (Meta Platforms, Inc. (f/k/a Facebook, Inc.))
• Original Assignee: DAQRI, LLC
• Inventors: Mullins, Brian, Kammerait, Matthew, Broaddus, Christopher
• Primary Examiner(s): Zimmerman, Brian
• Assistant Examiner(s): MCCORMACK, THOMAS S
• Application Number:
  US14/467,700
• Publication Number:
  US 20160057511A1
• Time in Patent Office: 911 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0141 characterised by the informative content of the display
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G08C 17/00 Arrangements for transmitting signals characterised by the use of a wireless electrical link
• CPC Class Codes: H04Q 2209/47 using RFID associated with sensors
• CPC Class Codes: H04Q 2209/75 by polling or interrogating the sub-stations
• CPC Class Codes: H04Q 2209/84 Measuring functions
• CPC Class Codes: H04Q 9/00 Arrangements in telecontrol or telemetry systems for selectively calling a substation from a main station, in which substation desired apparatus is selected for applying a control signal thereto or for obtaining measured values therefrom

Abstract

An application generates instructions to a wearable device to remotely activate a sensor in the wearable device and to receive sensor data from the sensor. A query related to a physical object is received. Instructions to wearable devices are generated to remotely activate at least one sensor of the wearable devices in response to the query. Sensor data is received from at least one of the wearable devices in response to that wearable device being within a range of the physical object.

Claims

1. A server comprising:

a hardware processor comprising a remote sensor application, the remote sensor application comprising;

a queuing module configured to receive a query to identify a physical object, a location of the physical object, a three-dimensional model of the physical object, and a sensor reading from a sensor of the physical object, to generate instructions to a plurality of wearable devices to remotely activate at least one sensor of the plurality of wearable devices in response to the query, and to receive sensor data from at least one of the plurality of wearable devices in response to the at least one of the plurality of wearable devices being within a range of the physical object;

an object recognition module configured to build the three-dimensional model of the physical object based on the sensor data; and

a data extraction module configured to optically extract the sensor reading of the sensor of the physical object in response to the query.

2. The server of claim 1, wherein the at least one wearable device comprises a helmet comprising at least one camera, at least one depth sensor, at least one inertial sensor, and a visor comprising a transparent display that displays augmented reality content based on the sensor data.

3. The server of claim 2, wherein the remote access module is further configured to remotely activate at least one of the camera, depth sensor, and inertial sensor of the at least one wearable device.

4. The server of claim 1, wherein the sensor data comprise a video feed from a corresponding wearable device, depth sensor data from the corresponding wearable device, an orientation of the corresponding wearable device, and a location of the corresponding wearable device.

5. The server of claim 1, wherein the queuing module is configured to determine whether at least one of the wearable devices is within the range of the physical object based on at least one of the identification of the physical object and the location of the physical object.

6. The server of claim 5, wherein the range corresponds to a threshold sensor range within which a three-dimensional model of the physical object is built based on the sensor data.

7. The server of claim 1, wherein the at least one sensor is configured to generate sensor data at a rate that is based on at least the relative distance between the plurality of wearable devices or the distances between each of the wearable devices and the physical object.

8. The server of claim 1, wherein the hardware processor further comprises:

a remote access module configured to generate instructions to remotely activate at least one sensor of at least one wearable device and to receive sensor data from the at least one sensor of the at least one wearable device.

9. A method comprising:

receiving a query to identify a physical object, a location of the physical object, a three-dimensional model of the physical object, and a sensor reading from a sensor of the physical object;

generating, implemented by a hardware processor, instructions to the plurality of wearable devices to remotely activate at least one sensor of the plurality of wearable devices in response to the query;

receiving sensor data from at least one of the plurality of wearable devices in response to the at least one of the plurality of the wearable devices being within a range of the physical object;

building the three-dimensional model of the physical object based on the sensor data; and

optically extracting the sensor reading of the sensor of the physical object in response to the query.

10. The method of claim 9, wherein the at least one wearable device comprises a helmet comprising at least one camera, at least one depth sensor, at least one inertial sensor, and a visor comprising a transparent display that displays augmented reality content based on the sensor data.

11. The method of claim 10, further comprising:

remotely activating at least one of the camera, depth sensor, and inertial sensor of the at least one wearable device.

12. The method of claim 9, wherein the sensor data comprise a video feed from a corresponding wearable device, depth sensor data from the corresponding wearable device, an orientation of the corresponding wearable device, and a location of the corresponding wearable device.

13. The method of claim 9, further comprising:

determining whether at least one of the wearable devices is within the range of the physical object based on at least one of the identification of the physical object and the location of the physical object.

14. The method of claim 13, wherein the range corresponds to a threshold sensor range within which a three-dimensional model of the physical object is built based on the sensor data.

15. The method of claim 9,

wherein the at least one sensor is configured to generate sensor data at a rate that is based on at least the relative distance between the plurality of wearable devices or the distances between each of the wearable devices and the physical object.

16. A non-transitory machine-readable medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:

receiving a query to identify a physical object, a location of the physical object, a three-dimensional model of the physical object, and a sensor reading from a sensor of the physical object;

generating instructions to the plurality of wearable devices to remotely activate at least one sensor of the plurality of wearable devices in response to the query;

receiving sensor data from at least one of the plurality of wearable devices in response to the at least one of the plurality of the wearable devices being within a range of the physical object;

building the three-dimensional model of the physical object based on the sensor data; and

optically extracting the sensor reading of the sensor of the physical object in response to the query.