METHODS AND APPARATUS FOR AUTONOMOUS ROBOTIC CONTROL

• US 20150269439A1
• Filed: 03/19/2015
• Published: 09/24/2015
• Est. Priority Date: 03/19/2014
• Status: Active Grant

• Current Assignee: Neurala, LLC, Trustees of Boston University
• Original Assignee: Neurala, LLC, Trustees of Boston University
• Inventors: Versace, Massimiliano, Gorshechnikov, Anatoly, Livitz, Gennady, Palma, Jesse
• Granted Patent: US 9,626,566 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 2219/39082 Collision, real time collision avoidance
• CPC Class Codes: G06N 3/008 based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
• CPC Class Codes: G06N 3/0409 Adaptive resonance theory [ART] networks
• CPC Class Codes: G06N 3/049 Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
• CPC Class Codes: G06N 3/088 Non-supervised learning, e.g. competitive learning
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/187 involving region growing; involving region merging; involving connected component labelling
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59

Abstract

Sensory processing of visual, auditory, and other sensor information (e.g., visual imagery, LIDAR, RADAR) is conventionally based on “stovepiped,” or isolated processing, with little interactions between modules. Biological systems, on the other hand, fuse multi-sensory information to identify nearby objects of interest more quickly, more efficiently, and with higher signal-to-noise ratios. Similarly, examples of the OpenSense technology disclosed herein use neurally inspired processing to identify and locate objects in a robot'"'"'s environment. This enables the robot to navigate its environment more quickly and with lower computational and power requirements.

Claims

1. A system comprising:

an image sensor to acquire a plurality of images of at least a portion of an environment surrounding a robot; and

a processor, operably coupled to the image sensor, to;

translate each image in the plurality of images from a frame of reference of the image sensor to an allocentric frame of reference;

identify a position, in the allocentric frame of reference, of an object appearing in at least one image in the plurality of images; and

determine if the object appears in at least one other image in the plurality of images based on the position, in the allocentric frame of reference, of the object.

2. The system of claim 1, wherein the processor is configured to translate the at least one image from the frame of reference of the image sensor to an allocentric frame of reference by:

translating each image in the plurality of images from the frame of reference of the image sensor to an egocentric frame of reference based on a position and/or an orientation of the image sensor in the egocentric frame of reference, the egocentric frame of reference being defined with respect to the robot; and

translating each image in the plurality of images from the egocentric frame of reference to the allocentric frame of reference.

3. The system of claim 1, wherein the processor is configured to identify the position in the allocentric frame of reference of the object by:

generating a segmented version of the at least one image in the plurality of images; and

determining at least one spatial shroud fitting a form of the object based at least in part on the segmented version of the at least one image.

4. The system of claim 3, wherein the processor is configured to determine if the object appears in at least one other image in the plurality of images at least in part on by:

applying the at least one spatial shroud to the other image in the plurality of images.

5. The system of claim 1, wherein the processor is configured to:

map the position, in the allocentric frame of reference, of the object to coordinates in the frame of reference of the image sensor; and

determine a change to a position and/or an orientation of the image sensor based at least in part on the coordinates in the frame of reference of the image sensor.

6. The system of claim 5, further comprising:

an actuator, operably coupled to the processor and to the image sensor, to adjust a field of view of the image sensor based at least in part on the change to the position and/or the orientation of the image sensor, and

wherein the image sensor is configured to acquire a subsequent image in the plurality of images in response to adjustment of the field of view.

7. A method of locating an object with respect to a robot, the method comprising:

(A) acquiring, with a image sensor coupled to the robot, a plurality of images of at least a portion of an environment surrounding the robot;

(B) automatically translating each image in the plurality of images from a frame of reference of the image sensor to an allocentric frame of reference;

(C) identifying a position, in the allocentric frame of reference, of an object appearing in at least one image in the plurality of images; and

(D) determining if the object appears in at least one other image in the plurality of images based on the position, in the allocentric frame of reference, of the object.

8. The method of claim 7, wherein (B) comprises:

(B1) translating each image in the plurality of images from the frame of reference of the image sensor to an egocentric frame of reference based on a position and/or an orientation of the image sensor in the egocentric frame of reference, the egocentric frame of reference being defined with respect to the robot; and

(B2) translating each image in the plurality of images from the egocentric frame of reference to the allocentric frame of reference.

9. The method of claim 7, wherein (C) comprises:

(C1) generating a segmented version of a first image in the plurality of images; and

(C2) determining a spatial shroud fitting a form of the object based at least in part on the segmented version of the first image.

10. The method of claim 9, further comprising:

(E) mapping the position, in the allocentric frame of reference, of the object to coordinates in the frame of reference of the image sensor; and

(F) determining a change to a position and/or an orientation of the image sensor based at least in part on the coordinates in the frame of reference of the image sensor.

11. The method of claim 10, wherein (F) further comprises positioning and/or orienting the image sensor away from the object.

12. The method of claim 10, wherein (F) further comprises positioning and/or orienting the image sensor to acquire another image of the object.

13. The method of claim 10, wherein:

(D) comprises translating and/or transforming the spatial shroud based at least in part on the change in the position and/or the orientation of the image sensor determined in (F), and

(C) comprises determining if the spatial shroud fits the form of the object in a segmented version of a second image in the plurality of images.

14. The method of claim 13, wherein (D) further comprises:

(D3) identifying at least one feature of the object in the first image; and

(D4) comparing the at least one feature to a plurality of features identified in other images in the plurality of images.

15. The method of claim 10, further comprising:

(G) adjusting a field of view of the image sensor based at least in part on the change to the position and/or the orientation of the image sensor.

16. The method of claim 15, wherein (A) comprises:

acquiring a subsequent image in the plurality of images in response to adjustment of the field of view.