Learning method and testing method for monitoring blind spot of vehicle, and learning device and testing device using the same

• US 10,474,930 B1
• Filed: 10/05/2018
• Issued: 11/12/2019
• Est. Priority Date: 10/05/2018
• Status: Active Grant

• Current Assignee: Stradvision, Inc.
• Original Assignee: Stradvision, Inc.
• Inventors: Kim, Kye-Hyeon, Kim, Yongjoong, Kim, Insu, Kim, Hak-Kyoung, Nam, Woonhyun, Boo, Sukhoon, Sung, Myungchul, Yeo, Donghun, Ryu, Wooju, Jang, Taewoong, Jeong, Kyungjoong, Je, Hongmo, Cho, Hojin
• Primary Examiner(s): Alavi, Amir
• Application Number:
  US16/152,699
• Time in Patent Office: 403 Days
• CPC Class Codes: G06F 18/217 Validation; Performance evaluation; Active pattern learning techniques
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06N 20/20 Ensemble learning
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06N 5/01 Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

A learning method of a CNN (Convolutional Neural Network) for monitoring one or more blind spots of a monitoring vehicle is provided. The learning method includes steps of: a learning device instructing a detector to output class information and location information on a monitored vehicle in a training image; instructing a cue information extracting layer to output cue information on the monitored vehicle by using the outputted information, and instructing an FC layer to determine whether the monitored vehicle is located on the blind spots by neural-network operations with the cue information or its processed values; and learning parameters of the FC layer and parameters of the detector, by backpropagating loss values for the blind spots by referring to the determination and its corresponding GT and backpropagating loss values for the vehicle detection by referring to the class information and the location information and their corresponding GT, respectively.

Claims

1. A learning method of a CNN (Convolutional Neural Network) for monitoring one or more blind spots of a monitoring vehicle, comprising steps of:

(a) a learning device, if a training image corresponding to at least one video image taken from the monitoring vehicle is inputted, instructing a detector on the monitoring vehicle to output class information and location information on a monitored vehicle included in the training image;

(b) the learning device instructing a cue information extracting layer to perform one or more operations by using the class information and the location information on the monitored vehicle, to thereby output one or more pieces of cue information on the monitored vehicle, and instructing an FC layer (fully connected layer) for monitoring the blind spots to perform one or more neural network operations by using said pieces of cue information or their processed values on the monitored vehicle, to thereby output a result of determining whether the monitored vehicle is located on one of the blind spots of the monitoring vehicle; and

(c) the learning device instructing a first loss layer to generate one or more loss values for the blind spots by referring to the result and its corresponding first GT (Ground Truth), to thereby learn one or more parameters of the FC layer for monitoring the blind spots by backpropagating the loss values for the blind spots, and instructing a second loss layer to generate one or more loss values for vehicle detection by referring to the class information and the location information on the monitored vehicle and their corresponding second GT, to thereby learn one or more parameters of the detector by backpropagating the loss values for the vehicle detection;

wherein the detector is a vehicle detector based on an R-CNN (Region-based Convolutional Neural Network) including;

one or more convolutional layers which generate a feature map from the training image,

an RPN (Region Proposal Network) which generates an ROI (Region Of Interest) of the monitored vehicle from the feature map,

a pooling layer which generates a feature vector by pooling an area, in the feature map, corresponding to the ROI,

at least one FC layer for the vehicle detection which performs at least one fully connected operation on the feature vector, to thereby generate one or more FC output values,

a classification layer which outputs the class information on the monitored vehicle by referring to the FC output values, and

a regression layer which outputs the location information on the monitored vehicle by referring to the FC output values.

2. The learning method of claim 1, wherein the loss values for the vehicle detection include one or more class loss values and one or more location loss values of the monitored vehicle, and

wherein the learning device learns one or more parameters of the FC layer for the vehicle detection and one or more parameters of the convolutional layers by backpropagating the class loss values and the location loss values.

3. The learning method of claim 1, wherein the learning device instructs the CNN to receive a concatenated value acquired by using said pieces of cue information on the monitored vehicle generated by the cue information extracting layer and the feature vector generated by the pooling layer of the detector, as an input to the FC layer for monitoring the blind spots.

4. A learning method of a CNN (Convolutional Neural Network) for monitoring one or more blind spots of a monitoring vehicle, comprising steps of:

(a) a learning device, if a training image corresponding to at least one video image taken from the monitoring vehicle is inputted, instructing a detector on the monitoring vehicle to output class information and location information on a monitored vehicle included in the training image;

(b) the learning device instructing a cue information extracting layer to perform one or more operations by using the class information and the location information on the monitored vehicle, to thereby output one or more pieces of cue information on the monitored vehicle, and instructing an FC layer (fully connected layer) for monitoring the blind spots to perform one or more neural network operations by using said pieces of cue information or their processed values on the monitored vehicle, to thereby output a result of determining whether the monitored vehicle is located on one of the blind spots of the monitoring vehicle; and

(c) the learning device instructing a first loss layer to generate one or more loss values for the blind spots by referring to the result and its corresponding first GT (Ground Truth), to thereby learn one or more parameters of the FC layer for monitoring the blind spots by backpropagating the loss values for the blind spots, and instructing a second loss layer to generate one or more loss values for vehicle detection by referring to the class information and the location information on the monitored vehicle and their corresponding second GT, to thereby learn one or more parameters of the detector by backpropagating the loss values for the vehicle detection,

wherein the FC layer for monitoring the blind spots includes a neural network capable of outputting a result value of whether the monitored vehicle is located on one of the blind spots by a multilayer perceptron to which said pieces of cue information on the monitored vehicle are inputted.

5. A learning method of a CNN (Convolutional Neural Network) for monitoring one or more blind spots of a monitoring vehicle, comprising steps of:

(a) a learning device, if a training image corresponding to at least one video image taken from the monitoring vehicle is inputted, instructing a detector on the monitoring vehicle to output class information and location information on a monitored vehicle included in the training image;

(b) the learning device instructing a cue information extracting layer to perform one or more operations by using the class information and the location information on the monitored vehicle, to thereby output one or more pieces of cue information on the monitored vehicle, and instructing an FC layer (fully connected layer) for monitoring the blind spots to perform one or more neural network operations by using said pieces of cue information or their processed values on the monitored vehicle, to thereby output a result of determining whether the monitored vehicle is located on one of the blind spots of the monitoring vehicle; and

(c) the learning device instructing a first loss layer to generate one or more loss values for the blind spots by referring to the result and its corresponding first GT (Ground Truth), to thereby learn one or more parameters of the FC layer for monitoring the blind spots by backpropagating the loss values for the blind spots, and instructing a second loss layer to generate one or more loss values for vehicle detection by referring to the class information and the location information on the monitored vehicle and their corresponding second GT, to thereby learn one or more parameters of the detector by backpropagating the loss values for the vehicle detection,

wherein said pieces of cue information on the monitored vehicle include at least part of (i) the class information on the monitored vehicle, (ii) the location information on the monitored vehicle, (iii) size information on the monitored vehicle corresponding to an ROI (Region Of Interest) size, (iv) aspect ratio information on the monitored vehicle, and (v) distance information between a center of the monitored vehicle and one of outer sides of the blind spots.

6. The learning method of claim 5, wherein the cue information extracting layer determines a smaller value among either of (i) a distance of one outer boundary of the blind spot BS-L from the center of the monitored vehicle and (ii) a distance of the other outer boundary of the blind spot BS-R from the center thereof as the distance information between the center of the monitored vehicle and said one of the outer sides of the blind spots, and further outputs relative location information in addition to the distance information in order to determine whether the center of the monitored vehicle is located inside or outside of said one of the outer sides of the blind spots.

7. A testing method of a CNN (Convolutional Neural Network) for monitoring one or more blind spots of a monitoring vehicle, comprising steps of:

(a) on condition that a learning device, (i) has instructed a detector on the monitoring vehicle to output class information for training and location information for training on a monitored vehicle included in a training image corresponding to at least one video image taken from the monitoring vehicle, (ii) has instructed a cue information extracting layer to perform one or more operations by using the class information for training and the location information for training on the monitored vehicle, to thereby output one or more pieces of cue information for training on the monitored vehicle, (iii) has instructed an FC layer (fully connected layer) for monitoring the blind spots to perform one or more neural network operations by using said pieces of cue information for training or their processed values, to thereby output a result for training of determining whether the monitored vehicle is located on one of the blind spots, and (iv) has instructed a first loss layer to generate one or more loss values for the blind spots by referring to the result for training and its corresponding first GT (Ground Truth), to thereby learn one or more parameters of the FC layer for monitoring the blind spots by backpropagating the loss values for the blind spots, and has instructed a second loss layer to generate one or more loss values for vehicle detection by referring to the class information for training and the location information for training and their corresponding second GT, to thereby learn one or more parameters of the detector by backpropagating the loss values for the vehicle detection, a testing device instructing a detector on the monitoring vehicle to output class information for testing and location information for testing on a monitored vehicle included in a test image taken from the monitoring vehicle; and

(b) the testing device instructing the cue information extracting layer to perform the operations by using the class information for testing and the location information for testing on the monitored vehicle, to thereby output one or more pieces of cue information for testing on the monitored vehicle, and instructing the FC layer for monitoring the blind spots to perform the neural network operations by using said pieces of cue information for testing or their processed values on the monitored vehicle, to thereby output a result for testing of determining whether the monitored vehicle is located on one of the blind spots of the monitoring vehicle;

wherein the detector is a vehicle detector based on an R-CNN (Region-based Convolutional Neural Network) including;

one or more convolutional layers which generate a feature map for testing from the test image,

an RPN (Region Proposal Network) which generates an ROI (Region Of Interest) for testing of the monitored vehicle from the feature map for testing,

a pooling layer which generates a feature vector for testing by pooling an area, in the feature map for testing, corresponding to the ROI for testing,

at least one FC layer for the vehicle detection which performs at least one fully connected operation on the feature vector for testing, to thereby generate one or more FC output values,

a classification layer which outputs the class information for testing on the monitored vehicle by referring to the FC output values, and

a regression layer which outputs the location information for testing on the monitored vehicle by referring to the FC output values.

8. The testing method of claim 7, wherein the testing device instructs the CNN to receive a concatenated value for testing acquired by using said pieces of cue information for testing on the monitored vehicle generated by the cue information extracting layer and the feature vector for testing generated by the pooling layer of the detector, as an input to the FC layer for monitoring the blind spots.

9. A testing method of a CNN (Convolutional Neural Network) for monitoring one or more blind spots of a monitoring vehicle, comprising steps of:

(a) on condition that a learning device, (i) has instructed a detector on the monitoring vehicle to output class information for training and location information for training on a monitored vehicle included in a training image corresponding to at least one video image taken from the monitoring vehicle, (ii) has instructed a cue information extracting layer to perform one or more operations by using the class information for training and the location information for training on the monitored vehicle, to thereby output one or more pieces of cue information for training on the monitored vehicle, (iii) has instructed an FC layer (fully connected layer) for monitoring the blind spots to perform one or more neural network operations by using said pieces of cue information for training or their processed values, to thereby output a result for training of determining whether the monitored vehicle is located on one of the blind spots, and (iv) has instructed a first loss layer to generate one or more loss values for the blind spots by referring to the result for training and its corresponding first GT (Ground Truth), to thereby learn one or more parameters of the FC layer for monitoring the blind spots by backpropagating the loss values for the blind spots, and has instructed a second loss layer to generate one or more loss values for vehicle detection by referring to the class information for training and the location information for training and their corresponding second GT, to thereby learn one or more parameters of the detector by backpropagating the loss values for the vehicle detection, a testing device instructing a detector on the monitoring vehicle to output class information for testing and location information for testing on a monitored vehicle included in a test image taken from the monitoring vehicle; and

(b) the testing device instructing the cue information extracting layer to perform the operations by using the class information for testing and the location information for testing on the monitored vehicle, to thereby output one or more pieces of cue information for testing on the monitored vehicle, and instructing the FC layer for monitoring the blind spots to perform the neural network operations by using said pieces of cue information for testing or their processed values on the monitored vehicle, to thereby output a result for testing of determining whether the monitored vehicle is located on one of the blind spots of the monitoring vehicle;

wherein the FC layer for monitoring the blind spots includes a neural network capable of outputting a result value for testing of whether the monitored vehicle is located on one of the blind spots by a multilayer perceptron to which said pieces of cue information for testing or their processed values on the monitored vehicle are inputted.

10. A testing method of a CNN (Convolutional Neural Network) for monitoring one or more blind spots of a monitoring vehicle, comprising steps of:

(a) on condition that a learning device, (i) has instructed a detector on the monitoring vehicle to output class information for training and location information for training on a monitored vehicle included in a training image corresponding to at least one video image taken from the monitoring vehicle, (ii) has instructed a cue information extracting layer to perform one or more operations by using the class information for training and the location information for training on the monitored vehicle, to thereby output one or more pieces of cue information for training on the monitored vehicle, (iii) has instructed an FC layer (fully connected layer) for monitoring the blind spots to perform one or more neural network operations by using said pieces of cue information for training or their processed values, to thereby output a result for training of determining whether the monitored vehicle is located on one of the blind spots, and (iv) has instructed a first loss layer to generate one or more loss values for the blind spots by referring to the result for training and its corresponding first GT (Ground Truth), to thereby learn one or more parameters of the FC layer for monitoring the blind spots by backpropagating the loss values for the blind spots, and has instructed a second loss layer to generate one or more loss values for vehicle detection by referring to the class information for training and the location information for training and their corresponding second GT, to thereby learn one or more parameters of the detector by backpropagating the loss values for the vehicle detection, a testing device instructing a detector on the monitoring vehicle to output class information for testing and location information for testing on a monitored vehicle included in a test image taken from the monitoring vehicle; and

(b) the testing device instructing the cue information extracting layer to perform the operations by using the class information for testing and the location information for testing on the monitored vehicle, to thereby output one or more pieces of cue information for testing on the monitored vehicle, and instructing the FC layer for monitoring the blind spots to perform the neural network operations by using said pieces of cue information for testing or their processed values on the monitored vehicle, to thereby output a result for testing of determining whether the monitored vehicle is located on one of the blind spots of the monitoring vehicle;

wherein said pieces of cue information for testing on the monitored vehicle include at least part of (i) the class information for testing on the monitored vehicle, (ii) the location information for testing on the monitored vehicle, (iii) size information for testing on the monitored vehicle corresponding to an ROI size, (iv) aspect ratio information for testing on the monitored vehicle, and (v) distance information for testing between a center of the monitored vehicle and one of outer sides of the blind spots.

11. The testing method of claim 10, wherein the cue information extracting layer determines a smaller value among either of (i) a distance of one outer boundary of the blind spot BS-L from the center of the monitored vehicle and (ii) a distance of the other outer boundary of the blind spot BS-R from the center thereof as the distance information for testing between the center of the monitored vehicle and said one of the outer sides of the blind spots, and further outputs relative location information for testing in addition to the distance information for testing in order to determine whether the center of the monitored vehicle is located inside or outside of said one of the outer sides of the blind spots.

12. A testing method of a CNN (Convolutional Neural Network) for monitoring one or more blind spots of a monitoring vehicle, comprising steps of:

(a) on condition that a learning device, (i) has instructed a detector on the monitoring vehicle to output class information for training and location information for training on a monitored vehicle included in a training image corresponding to at least one video image taken from the monitoring vehicle, (ii) has instructed a cue information extracting layer to perform one or more operations by using the class information for training and the location information for training on the monitored vehicle, to thereby output one or more pieces of cue information for training on the monitored vehicle, (iii) has instructed an FC layer (fully connected layer) for monitoring the blind spots to perform one or more neural network operations by using said pieces of cue information for training or their processed values, to thereby output a result for training of determining whether the monitored vehicle is located on one of the blind spots, and (iv) has instructed a first loss layer to generate one or more loss values for the blind spots by referring to the result for training and its corresponding first GT (Ground Truth), to thereby learn one or more parameters of the FC layer for monitoring the blind spots by backpropagating the loss values for the blind spots, and has instructed a second loss layer to generate one or more loss values for vehicle detection by referring to the class information for training and the location information for training and their corresponding second GT, to thereby learn one or more parameters of the detector by backpropagating the loss values for the vehicle detection, a testing device instructing a detector on the monitoring vehicle to output class information for testing and location information for testing on a monitored vehicle included in a test image taken from the monitoring vehicle; and

(b) the testing device instructing the cue information extracting layer to perform the operations by using the class information for testing and the location information for testing on the monitored vehicle, to thereby output one or more pieces of cue information for testing on the monitored vehicle, and instructing the FC layer for monitoring the blind spots to perform the neural network operations by using said pieces of cue information for testing or their processed values on the monitored vehicle, to thereby output a result for testing of determining whether the monitored vehicle is located on one of the blind spots of the monitoring vehicle,

wherein the monitoring vehicle for the testing device is not identical to the monitoring vehicle for the learning device.

13. A learning device of a CNN (Convolutional Neural Network) for monitoring one or more blind spots of a monitoring vehicle, comprising:

a communication part for receiving a training image corresponding to at least one video image taken from the monitoring vehicle; and

a processor for (I) instructing a detector on the monitoring vehicle to output class information and location information on a monitored vehicle included in the training image, (II) instructing a cue information extracting layer to perform one or more operations by using the class information and the location information on the monitored vehicle, to thereby output one or more pieces of cue information on the monitored vehicle, and instructing an FC layer (fully connected layer) for monitoring the blind spots to perform one or more neural network operations by using said pieces of cue information or their processed values on the monitored vehicle, to thereby output a result of determining whether the monitored vehicle is located on one of the blind spots of the monitoring vehicle, and (III) instructing a first loss layer to generate one or more loss values for the blind spots by referring to the result and its corresponding first GT (Ground Truth), to thereby learn one or more parameters of the FC layer for monitoring the blind spots by backpropagating the loss values for the blind spots, and instructing a second loss layer to generate one or more loss values for vehicle detection by referring to the class information and the location information on the monitored vehicle and their corresponding second GT, to thereby learn one or more parameters of the detector by backpropagating the loss values for the vehicle detection,

wherein the detector is a vehicle detector based on an R-CNN (Region-based Convolutional Neural Network) including;

one or more convolutional layers which generate a feature map from the training image,

an RPN (Region Proposal Network) which generates an ROI (Region Of Interest) of the monitored vehicle from the feature map,

a pooling layer which generates a feature vector by pooling an area, in the feature map, corresponding to the ROI,

at least one FC layer for the vehicle detection which performs at least one fully connected operation on the feature vector, to thereby generate one or more FC output values,

a classification layer which outputs the class information on the monitored vehicle by referring to the FC output values, and

a regression layer which outputs the location information on the monitored vehicle by referring to the FC output values.

14. The learning device of claim 13, wherein the loss values for the vehicle detection include one or more class loss values and one or more location loss values of the monitored vehicle, and

wherein the processor learns one or more parameters of the FC layer for the vehicle detection and one or more parameters of the convolutional layers by backpropagating the class loss values and the location loss values.

15. The learning device of claim 13, wherein the processor instructs the CNN to receive a concatenated value acquired by using said pieces of cue information on the monitored vehicle generated by the cue information extracting layer and the feature vector generated by the pooling layer of the detector, as an input to the FC layer for monitoring the blind spots.

16. A learning device of a CNN (Convolutional Neural Network) for monitoring one or more blind spots of a monitoring vehicle, comprising:

a communication part for receiving a training image corresponding to at least one video image taken from the monitoring vehicle; and

a processor for (I) instructing a detector on the monitoring vehicle to output class information and location information on a monitored vehicle included in the training image, (II) instructing a cue information extracting layer to perform one or more operations by using the class information and the location information on the monitored vehicle, to thereby output one or more pieces of cue information on the monitored vehicle, and instructing an FC layer (fully connected layer) for monitoring the blind spots to perform one or more neural network operations by using said pieces of cue information or their processed values on the monitored vehicle, to thereby output a result of determining whether the monitored vehicle is located on one of the blind spots of the monitoring vehicle, and (III) instructing a first loss layer to generate one or more loss values for the blind spots by referring to the result and its corresponding first GT (Ground Truth), to thereby learn one or more parameters of the FC layer for monitoring the blind spots by backpropagating the loss values for the blind spots, and instructing a second loss layer to generate one or more loss values for vehicle detection by referring to the class information and the location information on the monitored vehicle and their corresponding second GT, to thereby learn one or more parameters of the detector by backpropagating the loss values for the vehicle detection,

wherein the FC layer for monitoring the blind spots includes a neural network capable of outputting a result value of whether the monitored vehicle is located on one of the blind spots by a multilayer perceptron to which said pieces of cue information on the monitored vehicle are inputted.

17. A learning device of a CNN (Convolutional Neural Network) for monitoring one or more blind spots of a monitoring vehicle, comprising:

a communication part for receiving a training image corresponding to at least one video image taken from the monitoring vehicle; and

a processor for (I) instructing a detector on the monitoring vehicle to output class information and location information on a monitored vehicle included in the training image, (II) instructing a cue information extracting layer to perform one or more operations by using the class information and the location information on the monitored vehicle, to thereby output one or more pieces of cue information on the monitored vehicle, and instructing an FC layer (fully connected layer) for monitoring the blind spots to perform one or more neural network operations by using said pieces of cue information or their processed values on the monitored vehicle, to thereby output a result of determining whether the monitored vehicle is located on one of the blind spots of the monitoring vehicle, and (III) instructing a first loss layer to generate one or more loss values for the blind spots by referring to the result and its corresponding first GT (Ground Truth), to thereby learn one or more parameters of the FC layer for monitoring the blind spots by backpropagating the loss values for the blind spots, and instructing a second loss layer to generate one or more loss values for vehicle detection by referring to the class information and the location information on the monitored vehicle and their corresponding second GT, to thereby learn one or more parameters of the detector by backpropagating the loss values for the vehicle detection,

wherein said pieces of cue information on the monitored vehicle include at least part of (i) the class information on the monitored vehicle, (ii) the location information on the monitored vehicle, (iii) size information on the monitored vehicle corresponding to an ROI (Region Of Interest) size, (iv) aspect ratio information on the monitored vehicle, and (v) distance information between a center of the monitored vehicle and one of outer sides of the blind spots.

18. The learning device of claim 17, wherein the cue information extracting layer determines a smaller value among either of (i) a distance of one outer boundary of the blind spot BS-L from the center of the monitored vehicle and (ii) a distance of the other outer boundary of the blind spot BS-R from the center thereof as the distance information between the center of the monitored vehicle and said one of the outer sides of the blind spots, and further outputs relative location information in addition to the distance information in order to determine whether the center of the monitored vehicle is located inside or outside of said one of the outer sides of the blind spots.

19. A testing device of a CNN (Convolutional Neural Network) for monitoring one or more blind spots of a monitoring vehicle, comprising:

a communication part for, on condition that a learning device, (i) has instructed a detector on the monitoring vehicle to output class information for training and location information for training on a monitored vehicle included in a training image corresponding to at least one video image taken from the monitoring vehicle, (ii) has instructed a cue information extracting layer to perform one or more operations by using the class information for training and the location information for training on the monitored vehicle, to thereby output one or more pieces of cue information for training on the monitored vehicle, (iii) has instructed an FC layer (fully connected layer) for monitoring the blind spots to perform one or more neural network operations by using said pieces of cue information for training or their processed values, to thereby output a result for training of determining whether the monitored vehicle is located on one of the blind spots, and (iv) has instructed a first loss layer to generate one or more loss values for the blind spots by referring to the result for training and its corresponding first GT (Ground Truth), to thereby learn one or more parameters of the FC layer for monitoring the blind spots by backpropagating the loss values for the blind spots, and has instructed a second loss layer to generate one or more loss values for vehicle detection by referring to the class information for training and the location information for training and their corresponding second GT, to thereby learn one or more parameters of the detector by backpropagating the loss values for the vehicle detection, acquiring class information for testing and location information for testing on the monitored vehicle form the detector which detects the monitored vehicle in a test image taken from the monitoring vehicle; and

a processor for (I) instructing the cue information extracting layer to perform the operations by using the class information for testing and the location information for testing on the monitored vehicle, to thereby output one or more pieces of cue information for testing on the monitored vehicle, and (II) instructing the FC layer for monitoring the blind spots to perform the neural network operations by using said pieces of cue information for testing or their processed values on the monitored vehicle, to thereby output a result for testing of determining whether the monitored vehicle is located on one of the blind spots of the monitoring vehicle,

wherein the detector is a vehicle detector based on an R-CNN (Region-based Convolutional Neural Network) including;

one or more convolutional layers which generate a feature map for testing from the test image,

an RPN (Region Proposal Network) which generates an ROI (Region Of Interest) for testing of the monitored vehicle from the feature map for testing,

a pooling layer which generates a feature vector for testing by pooling an area, in the feature map for testing, corresponding to the ROI for testing,

at least one FC layer for the vehicle detection which performs at least one fully connected operation on the feature vector for testing, to thereby generate one or more FC output values,

a classification layer which outputs the class information for testing on the monitored vehicle by referring to the FC output values, and

a regression layer which outputs the location information for testing on the monitored vehicle by referring to the FC output values.

20. The testing device of claim 19, wherein the processor instructs the CNN to receive a concatenated value for testing acquired by using said pieces of cue information for testing on the monitored vehicle generated by the cue information extracting layer and the feature vector for testing generated by the pooling layer of the detector, as an input to the FC layer for monitoring the blind spots.

21. A testing device of a CNN (Convolutional Neural Network) for monitoring one or more blind spots of a monitoring vehicle, comprising:

a communication part for, on condition that a learning device, (i) has instructed a detector on the monitoring vehicle to output class information for training and location information for training on a monitored vehicle included in a training image corresponding to at least one video image taken from the monitoring vehicle, (ii) has instructed a cue information extracting layer to perform one or more operations by using the class information for training and the location information for training on the monitored vehicle, to thereby output one or more pieces of cue information for training on the monitored vehicle, (iii) has instructed an FC layer (fully connected layer) for monitoring the blind spots to perform one or more neural network operations by using said pieces of cue information for training or their processed values, to thereby output a result for training of determining whether the monitored vehicle is located on one of the blind spots, and (iv) has instructed a first loss layer to generate one or more loss values for the blind spots by referring to the result for training and its corresponding first GT (Ground Truth), to thereby learn one or more parameters of the FC layer for monitoring the blind spots by backpropagating the loss values for the blind spots, and has instructed a second loss layer to generate one or more loss values for vehicle detection by referring to the class information for training and the location information for training and their corresponding second GT, to thereby learn one or more parameters of the detector by backpropagating the loss values for the vehicle detection, acquiring class information for testing and location information for testing on the monitored vehicle form the detector which detects the monitored vehicle in a test image taken from the monitoring vehicle; and

a processor for (I) instructing the cue information extracting layer to perform the operations by using the class information for testing and the location information for testing on the monitored vehicle, to thereby output one or more pieces of cue information for testing on the monitored vehicle, and (II) instructing the FC layer for monitoring the blind spots to perform the neural network operations by using said pieces of cue information for testing or their processed values on the monitored vehicle, to thereby output a result for testing of determining whether the monitored vehicle is located on one of the blind spots of the monitoring vehicle,

wherein the FC layer for monitoring the blind spots includes a neural network capable of outputting a result value for testing of whether the monitored vehicle is located on one of the blind spots by a multilayer perceptron to which said pieces of cue information for testing or their processed values on the monitored vehicle are inputted.

22. A testing device of a CNN (Convolutional Neural Network) for monitoring one or more blind spots of a monitoring vehicle, comprising:

a communication part for, on condition that a learning device, (i) has instructed a detector on the monitoring vehicle to output class information for training and location information for training on a monitored vehicle included in a training image corresponding to at least one video image taken from the monitoring vehicle, (ii) has instructed a cue information extracting layer to perform one or more operations by using the class information for training and the location information for training on the monitored vehicle, to thereby output one or more pieces of cue information for training on the monitored vehicle, (iii) has instructed an FC layer (fully connected layer) for monitoring the blind spots to perform one or more neural network operations by using said pieces of cue information for training or their processed values, to thereby output a result for training of determining whether the monitored vehicle is located on one of the blind spots, and (iv) has instructed a first loss layer to generate one or more loss values for the blind spots by referring to the result for training and its corresponding first GT (Ground Truth), to thereby learn one or more parameters of the FC layer for monitoring the blind spots by backpropagating the loss values for the blind spots, and has instructed a second loss layer to generate one or more loss values for vehicle detection by referring to the class information for training and the location information for training and their corresponding second GT, to thereby learn one or more parameters of the detector by backpropagating the loss values for the vehicle detection, acquiring class information for testing and location information for testing on the monitored vehicle form the detector which detects the monitored vehicle in a test image taken from the monitoring vehicle; and

a processor for (I) instructing the cue information extracting layer to perform the operations by using the class information for testing and the location information for testing on the monitored vehicle, to thereby output one or more pieces of cue information for testing on the monitored vehicle, and (II) instructing the FC layer for monitoring the blind spots to perform the neural network operations by using said pieces of cue information for testing or their processed values on the monitored vehicle, to thereby output a result for testing of determining whether the monitored vehicle is located on one of the blind spots of the monitoring vehicle,

wherein said pieces of cue information for testing on the monitored vehicle include at least part of (i) the class information for testing on the monitored vehicle, (ii) the location information for testing on the monitored vehicle, (iii) size information for testing on the monitored vehicle corresponding to an ROI size, (iv) aspect ratio information for testing on the monitored vehicle, and (v) distance information for testing between a center of the monitored vehicle and one of outer sides of the blind spots.

23. The testing device of claim 22, wherein the cue information extracting layer determines a smaller value among either of (i) a distance of one outer boundary of the blind spot BS-L from the center of the monitored vehicle and (ii) a distance of the other outer boundary of the blind spot BS-R from the center thereof as the distance information for testing between the center of the monitored vehicle and said one of the outer sides of the blind spots, and further outputs relative location information for testing in addition to the distance information for testing in order to determine whether the center of the monitored vehicle is located inside or outside of said one of the outer sides of the blind spots.

24. A testing device of a CNN (Convolutional Neural Network) for monitoring one or more blind spots of a monitoring vehicle, comprising:

a communication part for, on condition that a learning device, (i) has instructed a detector on the monitoring vehicle to output class information for training and location information for training on a monitored vehicle included in a training image corresponding to at least one video image taken from the monitoring vehicle, (ii) has instructed a cue information extracting layer to perform one or more operations by using the class information for training and the location information for training on the monitored vehicle, to thereby output one or more pieces of cue information for training on the monitored vehicle, (iii) has instructed an FC layer (fully connected layer) for monitoring the blind spots to perform one or more neural network operations by using said pieces of cue information for training or their processed values, to thereby output a result for training of determining whether the monitored vehicle is located on one of the blind spots, and (iv) has instructed a first loss layer to generate one or more loss values for the blind spots by referring to the result for training and its corresponding first GT (Ground Truth), to thereby learn one or more parameters of the FC layer for monitoring the blind spots by backpropagating the loss values for the blind spots, and has instructed a second loss layer to generate one or more loss values for vehicle detection by referring to the class information for training and the location information for training and their corresponding second GT, to thereby learn one or more parameters of the detector by backpropagating the loss values for the vehicle detection, acquiring class information for testing and location information for testing on the monitored vehicle form the detector which detects the monitored vehicle in a test image taken from the monitoring vehicle; and

a processor for (I) instructing the cue information extracting layer to perform the operations by using the class information for testing and the location information for testing on the monitored vehicle, to thereby output one or more pieces of cue information for testing on the monitored vehicle, and (II) instructing the FC layer for monitoring the blind spots to perform the neural network operations by using said pieces of cue information for testing or their processed values on the monitored vehicle, to thereby output a result for testing of determining whether the monitored vehicle is located on one of the blind spots of the monitoring vehicle,

wherein the monitoring vehicle for the testing device is not identical to the monitoring vehicle for the learning device.