Method and apparatus for image segmentation using an event sensor
First Claim
1. A method for image segmentation in an electronic device, the method comprising:
- obtaining a first RGB frame of a scene using a sensor, the sensor comprising;
a red, green, blue (RGB) sensor generating RGB data, andan event sensor generating event data corresponding to the first RGB frame;
classifying objects in the scene to one or more classes of objects by identifying the one or more classes of objects in the scene;
obtaining event data corresponding to a second RGB frame with the event sensor at multiple sensitivity levels by changing exposure values of the event sensor;
identifying motion information of the scene with the classified objects based on the first RGB frame and the event data of the second RGB frame;
predicting an object boundary of the second RGB frame using the one or more classes of objects in the scene; and
estimating the object boundary for the second RGB frame using the motion information.
1 Assignment
0 Petitions
Accused Products
Abstract
A method for real-time semantic image segmentation using a monocular event-based sensor includes capturing a scene using a red, green, blue (RGB) sensor to obtain a plurality of RGB frames and an event sensor to obtain event data corresponding to each of the plurality of RGB frames, performing object labeling for objects in a first RGB frame among the plurality of RGB frames by identifying one or more object classes, obtaining an event velocity of the scene by fusing the event data corresponding to the first RGB frame and at least one subsequent RGB frame among the plurality of RGB frames, determining whether the event velocity is greater than a predefined threshold, and performing object labeling for objects in the at least one subsequent RGB frame based on the determination.
-
Citations
17 Claims
-
1. A method for image segmentation in an electronic device, the method comprising:
-
obtaining a first RGB frame of a scene using a sensor, the sensor comprising; a red, green, blue (RGB) sensor generating RGB data, and an event sensor generating event data corresponding to the first RGB frame; classifying objects in the scene to one or more classes of objects by identifying the one or more classes of objects in the scene; obtaining event data corresponding to a second RGB frame with the event sensor at multiple sensitivity levels by changing exposure values of the event sensor; identifying motion information of the scene with the classified objects based on the first RGB frame and the event data of the second RGB frame; predicting an object boundary of the second RGB frame using the one or more classes of objects in the scene; and estimating the object boundary for the second RGB frame using the motion information. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. A method for image segmentation in an electronic device, the method comprising:
-
capturing a scene using a red, green, blue (RGB) sensor to obtain a plurality of RGB frames and an event sensor to obtain event data corresponding to each of the plurality of RGB frames; performing object labeling for objects in a first RGB frame among the plurality of RGB frames by identifying one or more object classes; obtaining an event velocity of the scene by fusing the event data corresponding to the first RGB frame and at least one subsequent RGB frame among the plurality of RGB frames; determining whether the event velocity is greater than a predefined threshold; and performing object labeling for objects in the at least one subsequent RGB frame based on the determination, wherein the performing of the object labeling comprises classifying the objects into the one or more object classes. - View Dependent Claims (12, 13, 14, 15)
-
-
16. An apparatus for image segmentation, the apparatus comprising:
-
a sensor unit configured to obtain a first red, green, blue (RGB) frame of a scene, the sensor unit comprising; an RGB sensor configured to generate RGB data, and an event sensor configured to generate event data of the first RGB frame and a second RGB frame, wherein the event sensor is configured to obtain the second RGB frame at multiple sensitivity levels by changing exposure values of the event sensor; and a processor configured to; classify objects in the scene to one or more classes of objects by identifying the one or more classes of objects in the scene, identify motion information of the scene with the classified objects based on the first RGB frame and the event data of the second RGB frame; predict an object boundary of the second RGB frame using the one or more classes of objects in the scene; and estimate the object boundary for the second RGB frame using the motion information. - View Dependent Claims (17)
-
Specification