Method For New Package Detection

• US 20190065861A1
• Filed: 08/22/2018
• Published: 02/28/2019
• Est. Priority Date: 08/23/2017
• Status: Active Grant

• Current Assignee: Shanghai Hanshi Information Technology Company Limited
• Original Assignee: Bossa Nova Robotics IP, Inc.
• Inventors: Savvides, Marios, Lin, An Pang, Venugopalan, Shreyas, Thanikkal, Ajmal, Singh, Karanhaar, Adler, Gavriel, Neblett, Kyle, Palmer, Christopher
• Granted Patent: US 10,783,379 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/784 the detected or recognised objects being people
• CPC Class Codes: G06V 10/95 structured as a network, e.g. client-server architectures
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

A method for processing arrival or removal of packages within the field of view of a video camera includes providing a database for recording packages placed in the field of view. Based on real-time analysis of successive image frames in the camera, a human person'"'"'s entry and exit from the field of view of the camera is also detected. Delivery or removal of objects is recorded in the database. In one embodiment, the method also determines whether or not a newly arrived package is placed alongside or on top of an existing package.

Claims

1. A method for processing arrival or removal of an object within the field of view of a video camera, comprising:

providing a database for recording objects residing in the field of view;

based on real-time analysis of successive image frames in the camera, detecting a human person'"'"'s entry into the field of view of the camera;

saving a first set of image frames, corresponding image frames captured by the video camera immediately prior to the detection of the human person'"'"'s entry into the field of view of the camera;

based on real-time analysis of successive image frames in the camera, detecting a human person'"'"'s exit from the field of view of the camera;

saving a second set of image frames, corresponding to image frames captured by the video camera immediately after the detection of the human person'"'"'s exit from the field of view of the camera;

comparing the first set of image frames with the second set of image frames to identify one or more objects that appear or disappear from the field of view based on difference between the first and the set of image frames;

determining and eliminating objects identified by comparing the first and second sets of image frames as artifacts;

determining from the objects remaining and the objects residing in the database;

(a) whether or not one of the objects residing in the database has been removed; and

(b) whether or not an object has newly arrived; and

entering any newly arrived objects into the database and removing any objects residing in the database that have been determined to have been removed.

2. The method of claim 1, wherein the objects comprise packages that are delivered or collected.

3. The method of claim 1, wherein the artifacts correspond to change in lighting conditions.

4. The method of claim 1, wherein the artifacts correspond to change in lighting conditions due to the opening or shutting of a door.

5. The method of claim 1, further comprising determining whether or not the newly arrived object is placed alongside or on top of an existing object residing in the database.

6. The method of claim 1, wherein comparing the first and second sets of image frames comprises computing a block-based discrete cosine transform on each of the first set of image frames and the second set of image frames to determine therefrom a change in spatial frequencies.

7. The method of claim 1, wherein comparing the first and second sets of image frames comprises computing differences in light intensity between corresponding pixels in the first set of image frames and the second set of image frames.

8. The method of claim 7, further comprising computing accumulated differences for predetermined corresponding regions in the computed differences in light intensity.

9. The method of claim 1, wherein determining from the objects remaining and the objects residing in the database comprises computing one or more intersections between each object remaining and each object residing in the database.

10. The method of claim 9, wherein the computed intersections comprise an intersection in shape between objects.

11. The method of claim 9, wherein the computed intersections comprise an intersection in location between objects.

12. The method of claim 1, wherein the determining from the objects remaining and the objects residing in the database comprises comparing color histograms.

13. A method for package detection, comprising:

providing a database for recording packages in the field of view;

based on real-time analysis of successive image frames in the camera, detecting a human person'"'"'s entry and exit from a field of view of the camera;

after the human person'"'"'s entry and exit, determining whether packages have been removed or added by computing one or more intersections between each package remaining and each package residing in the database; and

entering any added packages into the database and removing packages residing in the database that have been determined to have been removed.

14. The method of claim 13, wherein the computed intersections comprise an intersection in shape between objects.

15. The method of claim 13, wherein the computed intersections comprise an intersection in location between objects.

16. The method of claim 13, further comprising determining whether or not the newly arrived object is placed alongside or on top of an existing object residing in the database.

17. The method of claim 13, wherein comparing the first and second sets of image frames comprises computing a block-based discrete cosine transform on each of the first set of image frames and the second set of image frames to determine therefrom a change in spatial frequencies.

18. The method of claim 13, further comprising use of a change detection module to define a bounding box around an object and determine whether it is a package.

19. The method of claim 13, further comprising a change detection module and a bounding box filtering module to eliminate bounding boxes unlikely to surround packages.

20. A system for processing arrival or removal of an object within the field of view of a video camera, comprising:

a database module for recording objects residing in the field of view;

a detection module to determine a human person'"'"'s entry into the field of view of the camera based on real-time analysis of successive image frames in the camera, with the detection module saving a first set of image frames corresponding to image frames captured by the video camera immediately prior to the detection of the human person'"'"'s entry into the field of view of the camera and saving a second set of image frames corresponding to image frames captured by the video camera immediately after the detection of the human person'"'"'s exit from the field of view of the camera;

a comparison module to compare the first set of image frames with the second set of image frames to identify one or more objects that appear or disappear from the field of view based on differences between the first and the set of image frames;

an artifact elimination module to determine and eliminate objects identified by comparing the first and second sets of image frames as artifacts; and

wherein the database module records (a) whether or not one of the objects residing in the database has been removed; and

(b) whether or not an object is newly arrived; and

c) enters any newly arrived objects into the database and removes any objects residing in the database that have been determined to have been removed.