Processing apparatus, processing system, processing program, and processing method

• US 10,402,664 B2
• Filed: 05/19/2015
• Issued: 09/03/2019
• Est. Priority Date: 05/19/2014
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Yokota, Soichiro, Lu, Yaojie, Ren, Jie, Takahashi, Sadao, Ishigaki, Tomoko
• Primary Examiner(s): Perungavoor, Sathyanaraya V
• Assistant Examiner(s): Dhillon, Puneet S
• Application Number:
  US15/311,946
• Publication Number:
  US 20170098132A1
• Time in Patent Office: 1,568 Days
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/285 using a sequence of stereo image pairs
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G08G 1/16 Anti-collision systems road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08

Abstract

A processing apparatus includes a distance image acquirer, a moving-object detector, a background recognizer. The distance image acquirer acquires a distance image containing distance information of each pixel. The moving-object detector detects a moving object from the distance image. The background recognizer generates a background model in which a stationary object recognized as background of the moving object is modeled, from the distance image acquired by the distance image acquirer. The moving-object detector changes a method for detecting the moving object, based on a relative positional relation between the moving object and the background model.

Claims

1. A processing apparatus comprising:

processing circuitry configured to

acquire a distance image containing distance information of each pixel;

detect a moving object from the distance image; and

generate a background model in which a stationary object recognized as background of the moving object is modeled, from the acquired distance image, wherein

the processing circuitry changes a method for detecting the moving object, based on a relative positional relation between the moving object and the background model,

wherein the detection of the moving object includes continuously tracking the moving object after it is initially recognized in image data using the method thus changed,

wherein, from a surrounding situation recognized by the processing circuitry, the processing circuitry is configured to

detect that the moving object has had a sudden change in orientation in a direction that is sideways to an initial traveling direction of the moving object, and

control display, on a display device, of a group of motion vectors of predicted motions of the moving object, the group of motion vectors being represented as an image of wave rings projecting away from the moving object, wherein each of the wave rings has a different display characteristic based on a respective distance of the wave ring from the moving object.

2. The processing apparatus according to claim 1, the processing circuitry being further configured to:

generate a travel-surface situation model in which a travel surface on which the moving object is to travel is modeled, from the distance image; and

recognize a surrounding situation by use of at least one of the positional relation between the moving object and the background model, and a positional relation between the moving object and the travel-surface situation model, wherein

the processing circuitry detects the moving object, based on a result of the situation recognition.

4. The processing apparatus according to claim 1, wherein, from a surrounding situation recognized by the processing circuitry, the processing circuitry detects the moving object that has partially hidden.

5. The processing apparatus according to claim 1, further comprising a memory to store therein history of a three-dimensional positional relation among the moving object, the travel-surface situation model, and the background model.

6. The processing apparatus according to claim 5, wherein, from the history of the three-dimensional positional relation among the moving object, the travel-surface situation model, and the background model stored in the memory, the processing circuitry detects the moving object that has had a change in orientation thereof or has partially hidden.

13. The processing apparatus according to claim 1, wherein the method for detecting the moving object is configured to be changed to one of a plurality of predetermined processing methods that include at least recognition processing for curves and sloping roads, detection processing according to a blocked area, and detection processing for tracking a recognition target with a predetermined threshold using a predicted position of the recognition target.

3. A processing apparatus comprising:

processing circuitry configured to

acquire a distance image containing distance information of each pixel;

detect a moving object from the acquired distance image;

generate a background model in which a stationary object recognized as background of the moving object is modeled, from the acquired distance image;

generate a travel-surface situation model in which a travel surface on which the moving object is to travel is modeled, from the acquired distance image; and

recognize a surrounding situation by use of at least one of a positional relation between the moving object and the background model, and a positional relation between the moving object and the travel-surface situation model,

wherein the detection of the moving object includes continuously tracking the moving object after it is initially recognized in image data,

wherein, from the surrounding situation recognized by the processing circuitry, the processing circuitry is configured to

detect that the moving object has had a sudden change in orientation in a direction that is sideways to an initial traveling direction of the moving object, and

control display, on a display device, of a group of motion vectors of predicted motions of the moving object, the group of motion vectors being represented as an image of wave rings projecting away from the moving object, wherein each of the wave rings has a different display characteristic based on a respective distance of the wave ring from the moving object.

10. The processing apparatus according to claim 3, wherein, from a surrounding situation recognized by the processing circuitry, the processing circuitry detects the moving object that has partially hidden.

11. The processing apparatus according to claim 3, further comprising a memory to store therein history of a three-dimensional positional relation among the moving object, the travel-surface situation model, and the background model.

12. The processing apparatus according to claim 11, wherein, from the history of the three-dimensional positional relation among the moving object, the travel-surface situation model, and the background model stored in the memory, the processing circuitry detects the moving object that has had a change in orientation thereof or has partially hidden.

7. A processing system comprising:

processing circuitry configured to

acquire a distance image containing distance information of each pixel;

generate a background model in which a stationary object recognized as background are modeled, from the acquired distance image;

change a detection method, based on a relative positional relation between a moving object detected from the distance image and the background model, and detect the moving object; and

in accordance with a result of the detection of the moving object, control a controlled device,

wherein the detection of the moving object includes continuously tracking the moving object after it is initially recognized in image data using the method thus changed,

wherein, from a surrounding situation recognized by the processing circuitry, the processing circuitry is configured to

detect that the moving object has had a sudden change in orientation in a direction that is sideways to an initial traveling direction of the moving object, and

control display, on a display device, of a group of motion vectors of predicted motions of the moving object, the group of motion vectors being represented as an image of wave rings projecting away from the moving object, wherein each of the wave rings has a different display characteristic based on a respective distance of the wave ring from the moving object.

8. A computer program product comprising a non-transitory computer-readable medium containing a processing program that causes a computer to perform a method comprising:

acquiring, by a distance image acquirer, a distance image containing distance information of each pixel;

generating, by a background recognizer, a background model in which a stationary object recognized as background is modeled, from the distance image acquired by the distance image acquirer; and

changing, based on a relative positional relation between a moving object detected from the distance image and the background model, a detection method so as to detect the moving object, by a moving-object detector,

wherein the detection of the moving object includes continuously tracking the moving object after it is initially recognized in image data using the method thus changed, and

wherein the method further includes

recognizing a surrounding situation,

detecting, based on the recognized surround situation, that the moving object has had a sudden change in orientation in a direction that is sideways to an initial traveling direction of the moving object, and

controlling display, on a display device, of a group of motion vectors of predicted motions of the moving object, the group of motion vectors being represented as an image of wave rings projecting away from the moving object, wherein each of the wave rings has a different display characteristic based on a respective distance of the wave ring from the moving object.

9. A processing method comprising:

acquiring, by a distance image acquirer, a distance image containing distance information of each pixel;

generating, by a background recognizer, a background model in which a stationary object recognized as background is modeled, from the distance image acquired by the distance image acquirer; and

changing, based on a relative positional relation between a moving object detected from the distance image and the background model, a detection method so as to detect the moving object, by a moving-object detector,

wherein the detection of the moving object includes continuously tracking the moving object after it is initially recognized in image data using the detection method thus changed, and

wherein the method further includes

recognizing a surrounding situation,

detecting, based on the recognized surround situation, that the moving object has had a sudden change in orientation in a direction that is sideways to an initial traveling direction of the moving object, and

controlling display, on a display device, of a group of motion vectors of predicted motions of the moving object, the group of motion vectors being represented as an image of wave rings projecting away from the moving object, wherein each of the wave rings has a different display characteristic based on a respective distance of the wave ring from the moving object.