Motion analysis method and apparatus

• US 9,552,644 B2
• Filed: 04/29/2015
• Issued: 01/24/2017
• Est. Priority Date: 11/17/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Ryu, Eric Hyunsurk, Chon, Sang-Gyun, Lee, Kyoobin, Lee, Jun Haeng
• Primary Examiner(s): Tran, Phuoc
• Application Number:
  US14/699,546
• Publication Number:
  US 20160140731A1
• Time in Patent Office: 636 Days
• US Class Current: 1/1
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A method and apparatus for analyzing a motion based on depth information are provided. The method includes: receiving event signals from a first vision sensor configured to generate at least one event signal by sensing at least a portion of an object in which a motion occurs; receiving a current frame from a second vision sensor configured to time-synchronously capture the object; synchronizing the received event signals and the received current frame; and obtaining depth information of the object based on the synchronized event signals and current frame.

Claims

1. A method of analyzing a motion based on depth information, the method comprising:

receiving event signals from a first vision sensor configured to generate at least one event signal by sensing at least a portion of an object in which a motion occurs;

receiving a current frame from a second vision sensor configured to time-synchronously capture the object;

synchronizing the received event signals and the received current frame; and

obtaining depth information of the object based on the synchronized event signals and current frame.

2. The method of claim 1, wherein the first vision sensor comprises an event-based vision sensor configured to generate the at least one event signal in response to an event in which light received from the object is changed non-time-synchronously.

3. The method of claim 1, wherein the synchronizing comprises matching the received current frame and the received event signals, received between a timing of a previous frame and a timing of the current frame.

4. The method of claim 1, wherein the synchronizing comprises matching the received current frame and the received event signals, received between a timing to which a predetermined latency is delayed from a timing of a previous frame and a timing to which the predetermined latency is delayed from a timing of the current frame.

5. The method of claim 1, wherein the synchronizing comprises determining, based on latencies of current event signals, at least one event signal to match a previous frame from among the current event signals received between a timing of the previous frame and a timing of the current frame.

6. The method of claim 1, wherein the synchronizing comprises determining, based on latencies of subsequent event signals, at least one event signal to match the current frame from among the subsequent event signals received between a timing of the current frame and a timing of a subsequent frame.

7. The method of claim 5, wherein the latencies are time differences between points in time at which the current event signals are generated and points in time at which the current event signals are received.

8. The method of claim 1, further comprising:

obtaining an edge image by detecting an edge from the received current frame,

wherein the synchronizing comprises synchronizing the received event signals and the obtained edge image.

9. The method of claim 1, wherein the obtaining comprises obtaining the depth information of the object based on a disparity calculated based on the synchronized event signals and current frame.

10. The method of claim 9, wherein the disparity is calculated by comparing pixel information on a pixel of one axis in the synchronized current frame and an event signal positioned on the one axis among the synchronized event signals.

11. The method of claim 1, further comprising:

analyzing the motion of the object based on the received event signals; and

compensating for the motion of the object based on the depth information of the object and the analyzed motion of the object.

12. The method of claim 11, wherein the analyzing comprises analyzing the motion of the object based on an optical flow including at least one velocity vector related to the received event signals.

13. The method of claim 12, further comprising:

compensating for the depth information of the object by using a velocity vector obtained based on the optical flow,

wherein the velocity vector corresponds to the received event signals.

14. The method of claim 1, wherein the receiving the current frame comprises receiving the current frame from a frame-based vision sensor configured to capture the object based on a predetermined number of frames per second.

15. The method of claim 1, wherein the first vision sensor and the second vision sensor are spaced apart at a predetermined interval and disposed in a same direction.

16. A non-transitory computer-readable recording medium having recorded thereon a program executable by a computer to implement the method of claim 1.

17. An apparatus for analyzing a motion based on depth information, the apparatus comprising:

a first vision sensor configured to generate at least one event signal by sensing at least a portion of an object in which a motion occurs;

a second vision sensor configured to generate a current frame by time-synchronously capturing the object; and

a processor configured to synchronize the generated at least one event signal received from the first vision sensor and the generated current frame received from the second vision sensor, and to obtain depth information of the object based on the synchronized event signals and current frame.

18. The apparatus of claim 17, wherein the first vision sensor comprises an event-based vision sensor configured to generate the at least one event signal in response to an event in which light received from the object is changed non-time-synchronously.

19. The apparatus of claim 17, wherein the processor is configured to match the received current frame and the received at least one event signal, received between a timing of a previous frame and a timing of the current frame.

20. The apparatus of claim 17, wherein the processor is configured to match the received current frame and the received at least one event signal, received between a timing to which a predetermined latency is delayed from a timing of a previous frame and a timing to which the predetermined latency is delayed from a timing of the current frame.