Systems and methods for detecting and picking up a waste receptacle

• US 9,403,278 B1
• Filed: 03/19/2015
• Issued: 08/02/2016
• Est. Priority Date: 03/19/2015
• Status: Active Grant

• Current Assignee: McNeilus Truck and Manufacturing Inc. (Oshkosh Corporation)
• Original Assignee: Waterloo Controls Inc.
• Inventors: Van Kampen, Leo Peter, Szoke-Sieswerda, Justin, Castellano, Brandon, McIsaac, Kenneth Alexander
• Primary Examiner(s): Cunningham, Gregory F
• Application Number:
  US14/662,791
• Time in Patent Office: 502 Days
• Field of Search: 382/103
• US Class Current: 1/1
• CPC Class Codes: B25J 9/1612 characterised by the hand, wrist, grip control
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: B65F 2003/023 Gripper arms for embracing the receptacle
• CPC Class Codes: B65F 3/04 Linkages, pivoted arms, or pivoted carriers for raising and subsequently tipping receptacles
• CPC Class Codes: G05B 2219/39505 Control of gripping, grasping, contacting force, force distribution
• CPC Class Codes: G05B 2219/40607 Fixed camera to observe workspace, object, workpiece, global
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/46 Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features colour feature extraction G06V10/56
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/751 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching

Abstract

Systems and methods for detecting and picking up a waste receptacle, the system being mountable on a waste-collection vehicle, and including an arm for grasping the waste receptacle, a processor, a camera, a non-transitory computer-readable medium, and an arm-actuation module. The processor is configured for generating a pose candidate based on an image captured by the camera, verifying that the pose candidate matches a template representation stored on the medium, and calculating a location of the waste receptacle. The arm-actuation module can be configured to automatically move the arm in response to the calculated location, in order to grasp the waste receptacle, lift, and dump the waste receptacle into a waste-collection vehicle.

Claims

1. A system for detecting and picking up a waste receptacle, comprising:

a) an arm for grasping the waste receptacle, the arm mountable on a waste-collection vehicle;

b) a camera for capturing an image;

c) a database for storing a template representation corresponding to the waste receptacle;

d) an arm-actuation module connected to the arm; and

e) a processor mounted on the waste-collection vehicle, in communication with the camera and the database and the arm-actuation module;

f) wherein the processor is configured for;

i) generating a pose candidate based on the image;

ii) verifying whether the pose candidate matches the template representation; and

iii) if the pose candidate matches the template representation, calculating a location of the waste receptacle; and

g) wherein the arm-actuation module is configured for automatically moving the arm in response to the calculated location of the waste receptacle.

2. The system of claim 1, wherein the processor is configured so that the step of generating the pose candidate comprises filtering the image and generating a set of gradient-response maps.

3. The system of claim 2, wherein the filtering the image comprises:

a) converting the image to a frequency domain from a spatial domain in order to obtain a spectral component of the image;

b) applying a high-pass Gaussian filter to the spectral component; and

,

c) converting the image back to the spatial domain after filtering.

4. The system of claim 2, wherein the processor is configured so that the step of generating the pose candidate further comprises suppressing noise via polling, the polling being performed at least twice.

5. The system of claim 2, wherein the processor is configured so that the step of generating the set of gradient-response maps comprises referencing at least one look-up table comprising linearized cosine-response values.

6. The system of claim 1, wherein the processor is configured so that the step of verifying whether the pose candidate matches the template representation comprises:

a) comparing a histogram of orientated gradients (HOG) of the template image with an HOG of the pose candidate;

b) determining that the pose candidate matches the template representation if the difference between the HOG of the template representation and the HOG of the pose candidate is below a pre-defined threshold; and

,

c) determining that the pose candidate does not match the template representation if the difference between the HOG of the template representation and the HOG of the pose candidate is above the pre-defined threshold.

7. The system of claim 1, wherein the processor is configured so that the step of calculating the location of the waste receptacle is based on pose metadata stored in the template representation.

8. The system of claim 1, wherein the arm-actuation module is configured so that the moving the arm comprises grasping the waste receptacle.

9. The system of claim 8, wherein the moving the arm further comprises lifting the waste receptacle and dumping contents of the waste receptacle into the waste-collection vehicle.

10. The system of claim 9, further comprising the waste-collection vehicle.

11. A method for detecting and picking up a waste receptacle comprising:

a) capturing an image with a camera;

b) generating a pose candidate based on the image;

c) verifying whether the pose candidate matches a template representation;

d) if and only if the pose candidate matches the template representation, calculating a location of the waste receptacle; and

,

e) moving an arm mounted on a waste-collection vehicle in response to the calculated location of the waste receptacle.

12. The method of claim 11, wherein the generating a pose candidate comprises filtering the image and generating a set of gradient-response maps.

13. The method of claim 12, wherein the filtering the image comprises:

a) converting the image to a frequency domain from a spatial domain in order to obtain a spectral component of the image;

b) applying a high-pass Gaussian filter to the spectral component; and

,

c) converting the image back to the spatial domain after filtering.

14. The method of claim 12, wherein the generating the pose candidate further comprises suppressing noise via polling, the polling being performed at least twice.

15. The method of claim 12, wherein the step of generating the set of gradient-response maps comprises referencing at least one look-up table comprising linearized cosine-response values.

16. The method of claim 12, wherein the verifying whether the pose candidate matches the template representation comprises:

a) comparing a histogram of oriented gradients (HOG) of the template representation with an HOG of the pose candidate;

b) determining that the pose candidate matches the template representation if the difference between the HOG of the template representation and the HOG of the pose candidate is below a pre-defined threshold; and

,

c) determining that the pose candidate does not match the template representation if the difference between the HOG of the template representation and the HOG of the pose candidate is above the pre-defined threshold.

17. The method of claim 12, wherein the calculating a location of the waste receptacle is based on pose metadata stored in the template representation.

18. The method of claim 12, wherein the moving the arm comprises grasping the waste receptacle.

19. A method for creating a representation of an object, comprising:

a) capturing a plurality of images of the object at a plurality of angles and depths;

b) deriving gradient information corresponding to the boundary of the object based on the plurality of images;

c) obtaining pose information corresponding to the plurality of angles and depths;

d) deriving pose metadata based on the pose information; and

,

e) composing a template representation corresponding to the object based on the gradient information and the pose metadata.

20. The method of claim 19, further comprising storing the template representation in a database.