Wheelchair detection using stereo vision

• US 7,043,084 B2
• Filed: 07/30/2002
• Issued: 05/09/2006
• Est. Priority Date: 07/30/2002
• Status: Expired due to Fees

• Current Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Inventors: Beardsley, Paul
• Primary Examiner(s): Bella, Matthew C.
• Assistant Examiner(s): GORADIA, SHEFALI DINESH
• Application Number:
  US10/210,845
• Publication Number:
  US 20040022439A1
• Time in Patent Office: 1,379 Days
• Field of Search: 187391-392, 382103-104, 382/107, 382/173, 382/175, 382/154, 382/224, 382/266, 382/190, 382/199, 382128-134
• US Class Current: 382/224
• CPC Class Codes: B66B 1/2408 where the allocation of a call to an elevator car is of importance, i.e. by means of a supervisory or group controller
• CPC Class Codes: B66B 1/468 Call registering systems
• CPC Class Codes: B66B 2201/4669 using passenger condition detectors
• CPC Class Codes: E05F 15/73 responsive to movement or presence of persons or objects
• CPC Class Codes: E05F 2015/767 using cameras
• CPC Class Codes: E05Y 2400/454 for accommodating handicapped users
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Abstract

A method classifies an object in a scene as either an ambulatory person or a wheelchair user. Images of the scene are acquired with a set of stereo cameras. Moving objects in the scene are segmented from the background using detected edges. A 3D reconstruction of the object is generated from the edges, and tests are performed on height and 3D shape of the 3D reconstruction to classify the object as the ambulatory person or the wheelchair user.

Claims

1. A method for classifying an object in a scene comprising:

acquiring a set of stereo images of the object in the scene with a set of stereo cameras;

segmenting the object from the background using detected edges of the stereo images;

generating a 3D reconstruction of the object from the detected edges of the stereo images in which the reconstruction further comprises;

fitting a vertical plane to the 3D reconstruction;

measuring a goodness-of-fit of the plane to the 3D reconstruction;

taking a ratio of the goodness-of-fit over the height, where the height is a difference between a topmost point of the 3D reconstruction and a ground plane; and

testing the ratio to classify the object as an ambulatory person or a wheelchair user; and

performing tests on height and 3D shape of the 3D reconstruction to classify the object as an ambulatory person or a wheelchair user.

2. The method of claim 1 wherein the segmentation uses edge detection based on background change.

3. The method of claim 2 wherein the 3D reconstruction is created via edge chaining and edge matching based on smoothly-varying stereo disparity.

4. The method of claim 1 further comprising:

classifying the object as an ambulatory person if the height is greater than a threshold, where the height is the difference between the topmost point of the 3D reconstruction and a ground plane.

5. The method of claim 4 further comprising:

determining the threshold from statistics about ambulatory people and wheelchair users.

6. The method of claim 1 further comprising:

measuring the goodness-of-fit of the plane as the median perpendicular distance between the plane and the points in the 3D reconstruction.

7. The method of claim 1 further comprising:

testing the ratio to classifying the object based on statistics about ambulatory people and wheelchair users.

8. The method of claim 1 wherein a sequence of sets of stereo images is analyzed:

classifying the object as the wheelchair user if the 3D reconstruction near a base of the object is not exhibiting oscillatory motion.

9. The method of claim 8 further comprising:

projecting 3D points near a base of the object to a ground plane;

fitting a straight line to the projected points;

fitting a sine wave to an angle of the straight line over time, to detect oscillation in the projected points.

10. The method of claim 1 wherein a sequence of sets of stereo images is analyzed, and further comprising:

classifying the object as the wheelchair user if object is moving and the 3D reconstruction near the base of the object is rigid.

11. The method of claim 1 wherein the scene includes multiple objects, and further comprising:

segmenting the 3D reconstruction into individual reconstructions for each object in the scene.

12. A system for classifying an object in a scene comprising:

a set of cameras acquiring a set of stereo images of the object in the scene;

means for segmenting the object from the background using detected edges of the stereo images;

means for generating a 3D reconstruction of the object from the detected edges of the stereo images, the reconstruction further comprising;

fitting a vertical plane to the 3D reconstruction;

measuring a goodness-of-fit of the plane to the 3D reconstruction;

taking a ratio of the goodness-of-fit over the height, where the height is a difference between a topmost point of the 3D reconstruction and a ground plane; and

testing the ratio to classify the object as an ambulatory person or a wheelchair user; and

means for performing tests on height and 3D shape of the 3D reconstruction to classify the object as an ambulatory person or a wheelchair user.