Using sensor-based observations of agents in an environment to estimate the pose of an object in the environment and to estimate an uncertainty measure for the pose
First Claim
1. A computer-implemented method, comprising:
- receiving a plurality of observations from a plurality of agents in an environment, each of the observations generated based on sensor data from one or more sensors of a corresponding agent of the agents and including;
an observation time, a source identifier of the corresponding agent, an observed object identifier of a corresponding observed object of multiple objects of the environment, a measured object pose for the corresponding observed object, and an observation uncertainty measure for the measured object pose;
identifying a group of the observations for a target time based on the observation times of the observations of the group satisfying a threshold relative to the target time;
generating, based on the group of observations, a multigraph having a plurality of nodes and edges connecting the nodes, the generating comprising defining a reference frame node of the nodes, defining object nodes of the nodes that are based on the observed object identifiers of the group, and generating the edges connecting the nodes based on the observed object identifiers, the source identifiers, and the measured object poses and observation uncertainty measures of the group,wherein generating each of the edges comprises;
connecting the edge with a respective pair of nodes,determining an edge measured object pose based on at least one of one or more of the measured object poses corresponding to the respective pair of nodes, anddetermining an edge object uncertainty measure based on at least one of one or more of the object uncertainty measures corresponding to the respective pair of the nodes, andwherein generating the multigraph comprises;
generating a first set of multiple of the edges, wherein the first set of the edges define a first simple path that connects the reference frame node and a given object node of the object nodes, the first set of the edges comprising a first set first edge that connects the given object node to a first object node, and a first set second edge that connects the first object node to a second object node or to the reference frame node, andgenerating a second set of multiple of the edges that define a second simple path that connects the reference frame node and the given object node, the second set of edges comprising a second set first edge that is in addition to the edges of the first set of the edges that define the first simple path;
generating a composite pose and composite uncertainty measure for each of a plurality of simple paths along the edges that connect the reference frame node and the given object node of the object nodes, including the first simple path and the second simple path, wherein the composite pose and the composite uncertainty measure for each of the simple paths is based on the edge measured object pose and the edge observation uncertainty measure for each of the edges of the simple path;
generating, based on the composite poses and the composite uncertainty measures, a pose and an uncertainty measure for the object identifier on which the given object node is based;
assigning the pose and the uncertainty measure to the object identifier on which the given object node is based; and
providing the object identifier, the pose, and the uncertainty measure to at least one robotic agent, of the agents in the environment, to influence control commands generated by the robotic agent.
4 Assignments
0 Petitions
Accused Products
Abstract
Methods, apparatus, systems, and computer-readable media are provided for using sensor-based observations from multiple agents (e.g., mobile robots and/or fixed sensors) in an environment to estimate the pose of an object in the environment at a target time and to estimate an uncertainty measure for that pose. Various implementations generate a multigraph based on a group of observations from multiple agents, where the multigraph includes a reference frame node, object nodes, and a plurality edges connecting the nodes. In some implementations, a composite pose and composite uncertainty measure are generated for each of a plurality of simple paths along the edges of the multigraph that connect the reference frame node to a given object node—and a pose and uncertainty measure for an object identifier associated with the given object node is generated based on the composite poses and the composite uncertainty measures.
15 Citations
20 Claims
-
1. A computer-implemented method, comprising:
-
receiving a plurality of observations from a plurality of agents in an environment, each of the observations generated based on sensor data from one or more sensors of a corresponding agent of the agents and including; an observation time, a source identifier of the corresponding agent, an observed object identifier of a corresponding observed object of multiple objects of the environment, a measured object pose for the corresponding observed object, and an observation uncertainty measure for the measured object pose; identifying a group of the observations for a target time based on the observation times of the observations of the group satisfying a threshold relative to the target time; generating, based on the group of observations, a multigraph having a plurality of nodes and edges connecting the nodes, the generating comprising defining a reference frame node of the nodes, defining object nodes of the nodes that are based on the observed object identifiers of the group, and generating the edges connecting the nodes based on the observed object identifiers, the source identifiers, and the measured object poses and observation uncertainty measures of the group, wherein generating each of the edges comprises; connecting the edge with a respective pair of nodes, determining an edge measured object pose based on at least one of one or more of the measured object poses corresponding to the respective pair of nodes, and determining an edge object uncertainty measure based on at least one of one or more of the object uncertainty measures corresponding to the respective pair of the nodes, and wherein generating the multigraph comprises; generating a first set of multiple of the edges, wherein the first set of the edges define a first simple path that connects the reference frame node and a given object node of the object nodes, the first set of the edges comprising a first set first edge that connects the given object node to a first object node, and a first set second edge that connects the first object node to a second object node or to the reference frame node, and generating a second set of multiple of the edges that define a second simple path that connects the reference frame node and the given object node, the second set of edges comprising a second set first edge that is in addition to the edges of the first set of the edges that define the first simple path; generating a composite pose and composite uncertainty measure for each of a plurality of simple paths along the edges that connect the reference frame node and the given object node of the object nodes, including the first simple path and the second simple path, wherein the composite pose and the composite uncertainty measure for each of the simple paths is based on the edge measured object pose and the edge observation uncertainty measure for each of the edges of the simple path; generating, based on the composite poses and the composite uncertainty measures, a pose and an uncertainty measure for the object identifier on which the given object node is based; assigning the pose and the uncertainty measure to the object identifier on which the given object node is based; and providing the object identifier, the pose, and the uncertainty measure to at least one robotic agent, of the agents in the environment, to influence control commands generated by the robotic agent. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. A system comprising:
-
a pose and uncertainty system comprising one or more computing systems; a plurality of agents in an environment providing sensor-based observations to the pose and uncertainty system over a period of time, wherein each of the observations is generated based on sensor data from one or more sensors of a corresponding agent of the agents and includes; an observation time, a source identifier of the corresponding agent, an observed object identifier of a corresponding observed object of multiple objects of the environment, a measured object pose for the corresponding observed object, and an observation uncertainty measure for the measured object pose; wherein the pose and uncertainty system comprises instructions that, when executed by the one or more computing systems, cause the computing systems to; identify a group of the observations for a target time based on the observation times of the observations of the group satisfying a threshold relative to the target time; generate, based on the group of observations, a multigraph having a plurality of nodes and edges connecting the nodes, the generating comprising defining a reference frame node of the nodes, defining object nodes of the nodes that are based on the observed object identifiers of the group, and generating the edges connecting the nodes based on the observed object identifiers, the source identifiers, and the measured object poses and observation uncertainty measures of the group, wherein in generating each of the edges the instructions cause one or more of the computing systems to; connect the edge with a respective pair of nodes, determine an edge measured object pose based on at least one of one or more of the measured object poses corresponding to the respective pair of nodes, and determine an edge object uncertainty measure based on at least one of one or more of the object uncertainty measures corresponding to the respective pair of the nodes, and wherein in generating the multigraph the instructions cause one or more of the computing systems to; generate a first set of multiple of the edges, wherein the first set of the edges define a first simple path that connects the reference frame node and a given object node of the object nodes, the first set of the edges comprising a first set first edge that connects the given object node to a first object node, and a first set second edge that connects the first object node to a second object node or to the reference frame node, and generate a second set of multiple of the edges that define a second simple path that connects the reference frame node and the given object node, the second set of edges comprising a second set first edge that is in addition to the edges of the first set of the edges that define the first simple path; generate a composite pose and composite uncertainty measure for each of a plurality of simple paths along the edges that connect the reference frame node and the given object node of the object nodes, including the first simple path and the second simple path, wherein the composite pose and the composite uncertainty measure for each of the simple paths is based on the edge measured object pose and the edge observation uncertainty measure for each of the edges of the simple path; generate, based on the composite poses and the composite uncertainty measures, a pose and an uncertainty measure for the object identifier on which the given object node is based; assign the pose and the uncertainty measure to the object identifier on which the given object node is based; and provide the object identifier and the pose to at least one robotic agent, of the agents in the environment, to influence control commands generated by the robotic agent. - View Dependent Claims (12, 13, 14, 15, 16, 17, 18, 19)
-
-
20. At least one non-transitory computer-readable medium comprising instructions that, in response to execution of the instructions by one or more processors, cause the one or more processors to perform the following operations:
-
receiving a plurality of observations from a plurality of agents in an environment, each of the observations generated based on sensor data from one or more sensors of a corresponding agent of the agents and including; an observation time, a source identifier of the corresponding agent, an observed object identifier of a corresponding observed object of multiple objects of the environment, a measured object pose for the corresponding observed object, and an observation uncertainty measure for the measured object pose; identifying a group of the observations for a target time based on the observation times of the observations of the group satisfying a threshold relative to the target time; generating, based on the group of observations, a multigraph having a plurality of nodes and edges connecting the nodes, the generating comprising defining a reference frame node of the nodes, defining object nodes of the nodes that are based on the observed object identifiers of the group, and generating the edges connecting the nodes based on the observed object identifiers, the source identifiers, and the measured object poses and observation uncertainty measures of the group, wherein generating each of the edges comprises; connecting the edge with a respective pair of nodes, determining an edge measured object pose based on at least one of one or more of the measured object poses corresponding to the respective pair of nodes, and determining an edge object uncertainty measure based on at least one of one or more of the object uncertainty measures corresponding to the respective pair of the nodes, and wherein generating the multigraph comprises; generating a first set of multiple of the edges, wherein the first set of the edges define a first simple path that connects the reference frame node and a given object node of the object nodes, the first set of the edges comprising a first set first edge that connects the given object node to a first object node, and a first set second edge that connects the first object node to a second object node or to the reference frame node, and generating a second set of multiple of the edges that define a second simple path that connects the reference frame node and the given object node, the second set of edges comprising a second set first edge that is in addition to the edges of the first set of the edges that define the first simple path; generating a composite pose and composite uncertainty measure for each of a plurality of simple paths along the edges that connect the reference frame node and the given object node, including the first simple path and the second simple path, wherein the composite pose and the composite uncertainty measure for each of the simple paths is based on the edge measured object pose and the edge observation uncertainty measure for each of the edges of the simple path; generating, based on the composite poses and the composite uncertainty measures, a pose and an uncertainty measure for the object identifier on which the given object node is based; assigning the pose and the uncertainty measure to the object identifier on which the given object node is based; and providing the object identifier and the pose to at least one robotic agent, of the agents in the environment, to influence control commands generated by the robotic agent.
-
Specification