METHODS OF AND APPARATUSES FOR RECOGNIZING MOTION OF OBJECTS, AND ASSOCIATED SYSTEMS

• US 20140119603A1
• Filed: 10/28/2013
• Published: 05/01/2014
• Est. Priority Date: 10/30/2012
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Dong-Wook Kwon, Gi-Sang Lee, Jin-Kyung Lee, Jin-Wuk Choi, Kyung-Il Kim, Min Ho Kim, Sang-Bo Lee, Young-Gu Jin
• Inventors: JIN, Young-Gu, KIM, Jung-Seop, KWON, Dong-Wook, KIM, Kyung-Il, KIM, Min Ho, LEE, Gi-Sang, LEE, Sang-Bo, LEE, Jin-Kyung, CHOI, Jin-Wuk
• Granted Patent: US 9,715,740 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/254 involving subtraction of images

Abstract

A method of recognizing motion of an object may include periodically obtaining depth data of a first resolution and two-dimensional data of a second resolution with respect to a scene using an image capturing device, wherein the second resolution is higher than the first resolution; determining a motion tracking region by recognizing a target object in the scene based on the depth data, such that the motion tracking region corresponds to a portion of a frame and the portion includes the target object; periodically obtaining tracking region data of the second resolution corresponding to the motion tracking region; and/or analyzing the motion of the target object based on the tracking region data.

Claims

1. A method of recognizing motion of an object, the method comprising:

periodically obtaining depth data of a first resolution and two-dimensional data of a second resolution with respect to a scene using an image capturing device, wherein the second resolution is higher than the first resolution;

determining a motion tracking region by recognizing a target object in the scene based on the depth data, such that the motion tracking region corresponds to a portion of a frame and the portion includes the target object;

periodically obtaining tracking region data of the second resolution corresponding to the motion tracking region; and

analyzing the motion of the target object based on the tracking region data.

2. The method of claim 1, wherein periodically obtaining the depth data and the two-dimensional data includes:

providing the depth data corresponding to the frame with a first frame period using depth pixels of the first resolution; and

providing the two-dimensional data corresponding to the frame with a second frame period using color pixels of the second resolution.

3. The method of claim 2, further comprising:

synchronizing the depth data and the two-dimensional data to be matched with each other, when the first frame period is different from the second frame period.

4. The method of claim 3, wherein the tracking region data corresponding to the motion tracking region are provided with the first frame period or the second frame period.

5. The method of claim 1, wherein periodically obtaining the tracking region data of the second resolution includes:

extracting region image data of the second resolution corresponding to the motion tracking region from the two-dimensional data of the second resolution corresponding to the frame; and

providing the region image data of the second resolution as the tracking region data.

6. The method of claim 2, wherein periodically obtaining the tracking region data of the second resolution includes:

extracting region depth data of the first resolution corresponding to the motion tracking region from the depth data of the first resolution corresponding to the frame;

extracting region image data of the second resolution corresponding to the motion tracking region from the two-dimensional data of the second resolution corresponding to the frame;

compensating for the region depth data of the first resolution using the region image data of the second resolution to generate region depth data of the second resolution; and

providing the region depth data of the second resolution as the tracking region data.

7. The method of claim 2, wherein the depth pixels and the color pixels are arranged in a common pixel array.

8. The method of claim 2, wherein the depth pixels and the color pixels are arranged respectively in distinct pixel arrays that are spaced apart from each other.

9. The method of claim 1, wherein periodically obtaining the depth data and the two-dimensional data includes:

periodically providing raw data corresponding to the frame using time-of-flight (TOF) depth pixels, the TOF depth pixels operating in response to a plurality of demodulation signals having different phases from each other; and

calculating the depth data of the first resolution and the two-dimensional data of the second resolution based on the raw data.

10. The method of claim 9, wherein calculating the depth data of the first resolution and the two-dimensional data of the second resolution includes:

providing the depth data of the first resolution by combining every M bits of the raw data, where M is a positive integer equal to or greater than two; and

providing the two-dimensional data of the second resolution by combining every N bits of the raw data, where N is a positive integer equal to or smaller than M.

11. The method of claim 9, wherein the demodulation signals have phase difference of 0, 90, 180 and 270 degrees, respectively, with respect to transmission light radiated from the image capturing device.

12. The method of claim 11, wherein providing the depth data of the first resolution includes:

providing one bit value of the depth data based on four bit values of the raw data, the four bit values respectively corresponding to the four demodulation signals having the phase difference of 0, 90, 180 and 270 degrees, respectively.

13. The method of claim 11, wherein providing the two-dimensional data of the second resolution includes:

providing one bit value of the two-dimensional data by summing two bit values of the raw data, the two bit values respectively corresponding to the two demodulation signals having the phase differences of 0 and 180 degrees; and

providing another bit value of the two-dimensional data by summing other two bit values of the raw data, the other two bit values respectively corresponding to the two demodulation signals having the phase differences of 90 and 270 degrees.

14. The method of claim 1, wherein periodically obtaining the tracking region data of the second resolution includes:

extracting region depth data of the first resolution corresponding to the motion tracking region from the depth data of the first resolution corresponding to the frame;

extracting region image data of the second resolution corresponding to the motion tracking region from the two-dimensional data of the second resolution corresponding to the frame;

compensating for the region depth data of the first resolution using the region image data of the second resolution to generate region depth data of the second resolution; and

providing the region depth data of the second resolution as the tracking region data.

15. The method of claim 1, wherein determining the motion tracking region includes:

determining coordinates of a center point of the motion tracking region in the frame; and

determining a size of the motion tracking region in the frame.

16. The method of claim 1, further comprising:

upgrading the motion tracking region according to the motion of the target object.

17. The method of claim 16, wherein upgrading the motion tracking region includes:

detecting a change of a position of the target object in the scene based on the depth data; and

changing coordinates of a center point of the motion tracking region in the frame based on the change of the position of the target object in the scene.

18. The method of claim 16, wherein upgrading the motion tracking region includes:

detecting a change of distance to the target object based on the depth data;

decreasing a size of the motion tracking region when the distance to the target object increases; and

increasing the size of the motion tracking region when the distance to the target object decreases.

19. An apparatus for recognizing motion of an object, the apparatus comprising:

an image capturing device configured to periodically provide depth data of a first resolution and two-dimensional data of a second resolution with respect to a scene, wherein the second resolution is higher than the first resolution;

a motion region tracker configured to determine a motion tracking region by recognizing a target object in the scene based on the depth data, such that the motion tracking region corresponds to a portion of a frame and the portion includes the target object, and configured to periodically provide tracking region data of the second resolution corresponding to the motion tracking region; and

a motion analyzer configured to analyze the motion of the target object based on the tracking region data.

20. The apparatus of claim 19, wherein the image capturing device includes:

a pixel array in which depth pixels of the first resolution and color pixels of the second resolution are alternatively arranged, the depth pixels providing the depth data with a first frame period, and the color pixels providing the two-dimensional data with a second frame period.

21. The apparatus of claim 19, wherein the image capturing device includes:

a first pixel array in which depth pixels of the first resolution are arranged, the depth pixels providing the depth data with a first frame period; and

a second pixel array in which color pixels of the second resolution are arranged, the color pixels providing the two-dimensional data with a second frame period.

22. The apparatus of claim 19, wherein the image capturing device includes:

a pixel array in which time-of-flight (TOF) depth pixels are arranged, the TOF depth pixels operating in response to a plurality of demodulation signals having different phases from each other to periodically provide raw data corresponding to the frame.

23. The apparatus of claim 22, wherein the demodulation signals have phase difference of 0, 90, 180, and 270 degrees, respectively, with respect to transmission light radiated from the image capturing device, and one bit value of the depth data is provided based on four bit values of the raw data, the four bit values respectively corresponding to the four demodulation signals having the phase difference of 0, 90, 180, and 270 degrees, respectively.

24. A system, comprising:

an image capturing device configured to periodically provide depth data of a first resolution corresponding to a frame of a scene and two-dimensional data of a second resolution corresponding to the frame, wherein the second resolution is higher than the first resolution;

a motion region tracker configured to determine a motion tracking region by recognizing a target object in the scene based on the depth data, such that the motion tracking region corresponds to a portion of the frame and the portion includes the target object, and configured to periodically provide tracking region data of the second resolution corresponding to the motion tracking region;

a motion analyzer configured to analyze motion of the target object based on the tracking region data; and

a control device configured to generate an event corresponding to the motion of the target object based on an analysis result of the motion analyzer.

25. The system of claim 24, wherein the system is a user interface system that operates by recognizing motion of a user, and

wherein the target object includes a body of the user or a portion of the body of the user.

26. A method for recognizing motion of an object, the method comprising:

obtaining depth data of a first resolution with respect to a scene;

obtaining two-dimensional data of a second resolution with respect to the scene;

recognizing the object in the scene based on the depth data;

tracking the object using a motion tracking region to provide tracking region data of the second resolution; and

analyzing the motion of the object based on the tracking region data;

wherein the second resolution is higher than the first resolution.

27. The method of claim 26, wherein obtaining the depth data of the first resolution comprises using a depth pixel array of a sensing unit to output depth information.

28. The method of claim 26, wherein obtaining the two-dimensional data of the second resolution comprises using a color pixel array of a sensing unit to output color information.

29. The method of claim 26, wherein obtaining the depth data of the first resolution comprises using a depth pixel array of a sensing unit to output depth information, and

wherein obtaining the two-dimensional data of the second resolution comprises using a color pixel array of the sensing unit to output color information.

30. The method of claim 26, wherein obtaining the depth data of the first resolution comprises using a pixel array of a sensing unit to output depth information, and

wherein obtaining the two-dimensional data of the second resolution comprises using the pixel array of the sensing unit to output color information.