System and method for monitoring handrail entrance of passenger conveyor

• US 10,214,391 B2
• Filed: 07/28/2017
• Issued: 02/26/2019
• Est. Priority Date: 07/29/2016
• Status: Active Grant

• Current Assignee: Otis Elevator Company (Otis Worldwide Corp.)
• Original Assignee: Otis Elevator Company (Otis Worldwide Corp.)
• Inventors: Wang, LongWen, Hu, ZhaoXia, Fang, Hui, Jia, Zhen, Zhao, Jianwei, Li, Qiang, Su, Anna, Finn, Alan Matthew, Gschwendtner, Gero
• Primary Examiner(s): Hess, Douglas A
• Application Number:
  US15/663,463
• Publication Number:
  US 20180029840A1
• Time in Patent Office: 578 Days
• Field of Search: 198322, 198323, 198324, 198325
• CPC Class Codes: B66B 21/02 Escalators
• CPC Class Codes: B66B 25/003 Methods or algorithms therefor
• CPC Class Codes: B66B 29/04 for balustrades or handrails

Abstract

The present invention provides a handrail entry monitoring system of a passenger conveyor and a monitoring method thereof, and belongs to the field of passenger conveyor technologies. In the handrail entry monitoring system and the monitoring method of the present invention, at least part of a handrail entry region of the passenger conveyor is sensed by using an imaging sensor and/or a depth sensing sensor, to acquire a data frame, and the data frame is analyzed to monitor whether a handrail entry of the operating passenger conveyor is in a normal state or an abnormal state. The monitoring system of the present invention and the monitoring method thereof can timely and effectively detect a danger that a foreign matter is about to be entrapped into the handrail entry, helping prevent foreign matters from being entrapped into the handrail entry, thereby improving safety of the passenger conveyor.

Claims

1. A handrail entry monitoring system of a passenger conveyor, comprising:

an imaging sensor and/or a depth sensing sensor configured to sense at least part of a handrail entry region of the passenger conveyor to acquire a data frame; and

a processing apparatus configured to analyze the data frame to monitor whether a handrail entry of the operating passenger conveyor is in a normal state or an abnormal state, wherein the normal state refers to that no foreign matter is about to enter or is already at least partially in a dangerous region of the handrail entry, and the abnormal state refers to that a foreign matter is about to enter or is already at least partially in the dangerous region of the handrail entry;

wherein the processing apparatus further comprises;

a background acquisition module configured to acquire a background model based on a data frame sensed when the handrail entry of the passenger conveyor is in the normal state;

a foreground detection module configured to compare a data frame sensed in real time with the background model to obtain a foreground object;

a foreground feature extraction module configured to extract a corresponding position feature from the foreground object; and

a state judgment module configured to judge, at least based on the position feature, whether the foreground object is in the dangerous region of the handrail entry, and determine, when the judgment result is “

yes”

, that the handrail entry is in the abnormal state.

2. The handrail entry monitoring system of claim 1, wherein the processing apparatus is configured to comprise:

a track generation module configured to generate, according to foreground objects obtained corresponding to multiple continuous data frames, a movement track of a target foreground object.

3. The handrail entry monitoring system of claim 2, wherein the state judgment module is further configured to:

pre-judge, based on the movement track of the target foreground object, whether the target foreground object is about to enter the dangerous region of the handrail entry, and determine, when the judgment result is “

yes”

, that the handrail entry is in the abnormal state.

4. The handrail entry monitoring system of claim 2, wherein the track generation module is further configured to:

track a same foreground target in the multiple continuous data frames by using a Bayesian filter technology, and generate the movement track by using position features of the same foreground target extracted by the foreground feature extraction module from the multiple continuous data frames respectively.

5. The handrail entry monitoring system of claim 1, wherein the processing apparatus is configured to further comprise:

a scene model generation module configured to define, based on the data frame sensed when the handrail entry of the passenger conveyor is in the normal state, the monitored dangerous region.

6. The handrail entry monitoring system of claim 1, wherein the foreground detection module further comprises:

a foreground filtering sub-module configured to filter the foreground object.

7. The handrail entry monitoring system of claim 1, wherein the foreground feature extraction module is further configured to extract a foreground object, an object texture, and one or more of a shape feature, a color feature, a size feature, a scale invariant feature transform feature, a corner feature, and a principal component feature;

wherein the processing apparatus is configured to further comprise;

a foreground object judgment module configured to judge, based on the foreground object, the object texture, and one or more of the shape feature, the color feature, the size feature, the scale invariant feature transform feature, the corner feature, and the principal component feature, whether the foreground object is a foreign matter that needs to completely avoid being entrapped.

8. The handrail entry monitoring system of claim 1, wherein, in the background acquisition module, the background model is established by using a Gaussian mixture model, code book model learning, principle components analysis, robust principle components analysis, mean filtering, neural network methods, kernel density estimation, adaptive kernel density estimation, recursive modeling or support vector data description modeling.

9. The handrail entry monitoring system of claim 1, wherein the imaging sensor/depth sensing sensor comprises one or more imaging sensors/depth sensing sensors mounted around the handrail entry.

10. The handrail entry monitoring system of claim 1, wherein the handrail entry monitoring system further comprises an alarm unit, and the processing apparatus triggers the alarm unit to operate when determining that the handrail entry is in the abnormal state.

11. The handrail entry monitoring system of claim 1, wherein the processing apparatus is further configured to trigger an output signal to enable a braking part of the passenger conveyor to operate when determining that the handrail entry is in the abnormal state.

12. A passenger conveying system, comprising a passenger conveyor and the handrail entry monitoring system of claim 1.

13. A handrail entry monitoring method of a passenger conveyor, comprising steps of:

sensing, by an imaging sensor and/or a depth sensing sensor, at least part of a handrail entry region of the passenger conveyor to acquire a data frame; and

analyzing the data frame to monitor whether a handrail entry of the operating passenger conveyor is in a normal state or an abnormal state;

wherein the normal state refers to that no foreign matter is about to enter or is already at least partially in a dangerous region of the handrail entry, and the abnormal state refers to that a foreign matter is about to enter or is already at least partially in the dangerous region of the handrail entry;

wherein analyzing the data further comprises;

acquiring a background model based on a data frame sensed when the handrail entry of the passenger conveyor is in the normal state;

comparing a data frame sensed in real time with the background model to obtain a foreground object;

extracting a corresponding position feature from the foreground object; and

judging, at least based on the position feature, whether the foreground object is in the dangerous region of the handrail entry, and determining, when the judgment result is “

yes”

, that the handrail entry is in the abnormal state.

14. The handrail entry monitoring method of claim 13, wherein the analyzing step further comprises:

generating, according to foreground objects obtained corresponding to multiple continuous data frames, a movement track of a target foreground object.

15. The handrail entry monitoring method of claim 14, wherein, in the step of judging whether the foreground object is in the dangerous region of the handrail entry, whether the target foreground object is about to enter the dangerous region of the handrail entry is pre-judged based on the movement track of the target foreground object, and it is determined that the handrail entry is in the abnormal state when the judgment result is “

yes”

.

16. The handrail entry monitoring method of claim 14, wherein, in the step of generating a movement track, a same foreground target in the multiple continuous data frames is tracked by using a Bayesian filter technology, and the movement track is generated by using position features of the same foreground target extracted by the foreground feature extraction module from the multiple continuous data frames respectively.

17. The handrail entry monitoring method of claim 13, wherein the analyzing step further comprises:

defining, based on the data frame sensed when the handrail entry of the passenger conveyor is in the normal state, the monitored dangerous region.

18. The handrail entry monitoring method of claim 13, wherein, in the step of obtaining a foreground object, the foreground object is filtered.

19. The handrail entry monitoring method of claim 13, wherein, in the extracting step, a foreground object, an object texture, and one or more of a shape feature, a color feature, a size feature, a scale invariant feature transform feature, a corner feature, and a principal component feature are further extracted;

and

the analyzing step further comprises;

judging, based on the foreground object, the object texture, and one or more of the shape feature, the color feature, the size feature, the scale invariant feature transform feature, the corner feature, and the principal component feature, whether the foreground object is a foreign matter that needs to completely avoid being entrapped.

20. The handrail entry monitoring method of claim 13, wherein the background model is established by using a Gaussian mixture model, code book model learning, principle components analysis, robust principle components analysis, mean filtering, neural network methods, kernel density estimation, adaptive kernel density estimation, recursive modeling or support vector data description modeling.

21. The handrail entry monitoring method of claim 13, wherein the analyzing step further comprises:

directly determining that the handrail entry is in the normal state when the foreground object is basically absent.

22. The handrail entry monitoring method of claim 13, wherein an alarm unit is triggered to operate when it is determined that the handrail entry is in the abnormal state.

23. The handrail entry monitoring method of claim 13, wherein an output signal is triggered to enable a braking part of the passenger conveyor to operate when it is determined that the handrail entry is in the abnormal state.