METHOD AND DEVICE FOR GENERATING BINARY DESCRIPTORS IN VIDEO FRAMES

• US 20170177980A1
• Filed: 12/16/2016
• Published: 06/22/2017
• Est. Priority Date: 12/17/2015
• Status: Active Grant

• Current Assignee: Stmicroelectronics France (STMicroelectronics NV)
• Original Assignee: STMicroelectronics SA (STMicroelectronics NV)
• Inventors: ALIBAY, Manu, AUBERGER, Stphane, STANCIULESCU, Bogdan-Florin
• Granted Patent: US 10,268,929 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/251 of input or preprocessed data
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/40 Extraction of image or video features
• CPC Class Codes: G06V 10/467 Encoded features or binary features, e.g. local binary patterns [LBP]
• CPC Class Codes: G06V 10/803 of input or preprocessed data
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 23/689 Motion occurring during a rolling shutter mode
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

A method generates a binary descriptor associated with a given point in a current frame of a succession of video frames obtained by an apparatus such as an image sensor. The method includes determining a pattern of points pairs around said given point in the current frame, and performing intensity comparison processing between the two points of each pair. The apparatus is likely to move in a rotation between the previous frame and the current frame. The method includes processing the pattern of points of the current frame with tridimensional rotation information representative of the apparatus rotation between the previous frame and the current frame and obtained from inertial measurements provided by at least one inertial sensor.

Claims

1. A method, comprising generating a binary descriptor associated with a given point in a current frame of a succession of video frames obtained by an apparatus configured to move in a rotation between a previous frame and the current frame, the generating including determining a pattern of point pairs of points around said given point in the current frame, and performing intensity comparison processing between the points of each pair, and processing said pattern of point pairs of said current frame with tridimensional rotation information representative of said apparatus rotation between the previous frame and the current frame.

2. The method according to claim 1, wherein said tridimensional rotation information is obtained from inertial measurements.

3. The method according to claim 1, wherein processing said pattern of points of said current frame comprises, for each point of each point pair of said pattern:

determining bidimensional coordinates of said point from bidimensional coordinates of a homologue point in the previous frame and from an in-plane rotation matrix corresponding to an in-plane rotation, in a plane, of an in-plane angle around said given point,

converting said determined bidimensional coordinates of said point into tridimensional coordinates,

determining a second rotation matrix representative of said tridimensional rotation information,

applying said second rotation matrix to said tridimensional coordinates of said point to obtain new tridimensional coordinates, and

projecting said new tridimensional coordinates back onto said plane of said in-plane rotation to obtain a projected point having projected coordinates.

4. The method according to claim 3, wherein performing intensity comparison processing between the points of each point pair comprises determining intensities of the projected points of said point pair and comparing the intensities.

5. The method according to claim 1, wherein said given point is an origin of each point pair.

6. The method according to claim 1, wherein the points of said pattern of point pairs are included in a zone of L×

M points centered on said given point.

7. The method according to claim 1, wherein said given point is a keypoint of said current frame.

8. The method according to claim 1, wherein said apparatus belongs to a platform.

9. A device comprising generating means for generating a binary descriptor associated with a given point in a current frame of a succession of video frames obtained by an apparatus configured to move in a rotation between a previous frame and the current frame, said generating means including first means for determining a pattern of point pairs of points around said given point in the current frame, and comparison means for performing intensity comparison processing between the points of each point pair, and processing means for processing said pattern of point pairs of said current frame with tridimensional rotation information representative of said apparatus rotation between the previous frame and the current frame.

10. The device according to claim 9, wherein said processing means are configured to receive said tridimensional rotation information from inertial measurements provided by at least one inertial sensor.

11. The device according to claim 9, wherein said processing means comprises a processing module configured, for each point of each point pair of said pattern, to:

determine bidimensional coordinates of said point from bidimensional coordinates of a homologue point in the previous frame and from an in-plane rotation matrix corresponding to an in-plane rotation, in a plane, of an in-plane angle around said given point,

convert said determined bidimensional coordinates of said point into tridimensional coordinates,

determine a second rotation matrix representative of said tridimensional rotation information,

apply said second rotation matrix to said tridimensional coordinates of said point to obtain new tridimensional coordinates, and

project said new tridimensional coordinates back onto said plane of said in-plane rotation to obtain a projected point having projected coordinates.

12. The device according to claim 11, wherein said comparison means are configured to determine respective intensities of the projected points of said point pair and to compare the intensities.

13. The device according to claim 9, wherein said given point is an origin of each point pair.

14. The device according to claim 9, wherein said pattern of point pairs is included in a zone of L×

M points centered on said given point.

15. The device according to claim 9, wherein said given point is a keypoint of said current frame.

16. A tablet platform, comprising:

an apparatus configured to capture a succession of video frames including a current frame and a previous frame and configured to move in a rotation between the previous frame and the current frame; and

a device including generating means for generating a binary descriptor associated with a given point in the current frame, said generating means including first means for determining a pattern of point pairs of points around said given point in the current frame, and comparison means for performing intensity comparison processing between the points of each point pair, and processing means for processing said pattern of point pairs of said current frame with tridimensional rotation information representative of said apparatus rotation between the previous frame and the current frame.

17. The platform according to claim 16, further comprising at least one inertial sensor configured to provide said tridimensional rotation information.

18. The platform according to claim 16, wherein said processing means comprises a processing module configured, for each point of each point pair of said pattern, to:

determine bidimensional coordinates of said point from bidimensional coordinates of a homologue point in the previous frame and from an in-plane rotation matrix corresponding to an in-plane rotation, in a plane, of an in-plane angle around said given point,

convert said determined bidimensional coordinates of said point into tridimensional coordinates,

determine a second rotation matrix representative of said tridimensional rotation information,

apply said second rotation matrix to said tridimensional coordinates of said point to obtain new tridimensional coordinates, and

project said new tridimensional coordinates back onto said plane of said in-plane rotation to obtain a projected point having projected coordinates.

19. The platform according to claim 18, wherein said comparison means are configured to determine respective intensities of the projected points of said point pair and to compare the intensities.

20. The platform according to claim 16, wherein said pattern of point pairs is included in a zone of L×

M points centered on said given point.