Motion determination system and method thereof

• US 10,229,505 B2
• Filed: 07/08/2016
• Issued: 03/12/2019
• Est. Priority Date: 06/29/2016
• Status: Active Grant

• Current Assignee: KAMI Vision Incorporated, Xiaoyi Technology Co., Ltd.
• Original Assignee: Shanghai Xiaoyi Technology Co., Ltd.
• Inventors: Ding, Dajun, Zhao, Lili, Cao, Taiqiang
• Primary Examiner(s): Bitar, Nancy
• Application Number:
  US15/205,396
• Publication Number:
  US 20180005384A1
• Time in Patent Office: 977 Days
• Field of Search: None
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle

Abstract

Methods and apparatus for determining the motion of a movable object are disclosed. The methods may include, in a view of the environment outside the movable object, identifying a first region of interest (ROI) corresponding to a static portion of the environment. The methods may also include, in the view of the environment, identifying a second region of interest (ROI) corresponding to an active portion of the environment. The methods may also include receiving first and second image data respectively representing the first and second ROIs. The methods may also include analyzing the first image data over time. The methods may also include analyzing the second image data over time. The methods may further include determining whether the movable object is in motion based on the analyzes of the first and second image data.

Claims

1. A method performed by a processor in communication with an imaging device mounted on a movable object, the method comprising:

in a view of an environment outside the movable object, identifying a first region of interest (ROI) corresponding to a static portion of the environment, the first ROI covering at least an area in which vehicle and pedestrian traffic cannot be observed;

in the view of the environment, identifying a second region of interest (ROI) corresponding to an active portion of the environment, the second ROI covering at least an area in which vehicle or pedestrian traffic can be observed;

receiving, from the imaging device, first and second image data respectively representing the first and second ROIs;

analyzing the first image data over time, wherein analyzing the first image data comprises performing a first comparison of an image property of the first ROI with a first threshold;

analyzing the second image data over time, wherein analyzing the second image data comprises performing a second comparison of an image property of the second ROI with a second threshold; and

determining whether the movable object is in motion based on the analyses of the first and second image data.

2. The method of claim 1, wherein the determining comprises concluding that the movable object is not in motion if a change over time of the first image data is less than a change over time of the second image data.

3. The method of claim 1, wherein

the movable object is a vehicle.

4. The method of claim 1, wherein the first ROI corresponds to a first static portion of the environment, the method further comprising:

in the view of the environment, identifying a third ROI corresponding to a second static portion of the environment, second static portion being non-overlapping with the first static portion;

receiving third image data representing the third ROI;

analyzing the third image data over time;

comparing the analyses of the first and third image data; and

determining whether the movable object is in motion at least partially based on the comparison result.

5. The method of claim 1, wherein:

determining whether the movable object is in motion comprises determining whether the movable object is in motion based on at least one of the first or second comparisons.

6. The method of claim 5, wherein the image property of a ROI comprises at least one of a change of pixel values over time, a complexity of pixel values in space, or a brightness in the ROI.

7. The method of claim 6, wherein the change of pixel values over time is calculated as a change over time of an averaged pixel value.

8. The method of claim 6, wherein the complexity of pixel values is calculated as a root-mean-square derivation of pixel values.

9. The method of claim 6, wherein the brightness in the ROI is calculated as an averaged pixel brightness value.

10. The method of claim 1, wherein receiving the first and second image data comprises:

receiving raw data corresponding to the first and second ROIs; and

down-sampling the raw data to generate the first and second image data.

11. The method of claim 10, wherein:

the raw data corresponding to the first and second ROIs includes a plurality of pixel values; and

down-sampling the raw data comprises;

sampling, according to a sampling rule, a subset of pixel values from the raw data corresponding to the first and second ROIs;

dividing the sampled pixel values into a predetermined number of groups; and

averaging the pixel values in the groups.

12. The method of claim 6, further comprising:

determining whether a change of pixel values over time in the first ROI exceeds a first amount of change; and

when the change of pixel values over time in the first ROI exceeds the first amount of change, concluding that the movable object is in motion.

13. The method of claim 12, further comprising:

when the change of pixel values over time in the first ROI does not exceed the first amount of change, determining whether a complexity of pixel values in the first ROI exceeds a first complexity level; and

when the complexity of pixel values in the first ROI exceeds the first complexity level, concluding that the movable object is not in motion.

14. The method of claim 13, further comprising:

when the complexity of pixel values in the first ROI does not exceed the first complexity level, determining whether a brightness in the first ROI exceeds a first brightness level;

when the brightness in the first ROI does not exceed the first brightness level, determining whether a brightness in the second ROI exceeds a second brightness level; and

when the brightness of the second ROI exceeds the second brightness level, determining that the movable object is in motion.

15. The method of claim 14, further comprising:

when the brightness in the second ROI exceeds the second brightness level, concluding that the current time is nighttime.

16. The method of claim 14, further comprising:

when the brightness in the second ROI does not exceed the second brightness level, concluding that the movable object is not in motion.

17. The method of claim 14, further comprising:

when the brightness in the second ROI does not exceed the second brightness level, concluding that the movable object is in at least one of nighttime or a covered space.

18. The method of claim 14, further comprising:

when the brightness in the first ROI exceeds the first brightness level, concluding that the movable object is in motion.

19. The method of claim 14, further comprising:

when the brightness in the first ROI exceeds the first brightness level, concluding that the movable object is in daytime.

20. The method of claim 1, further comprising:

determining a distance between the movable object and a second object outside the movable object;

analyzing the distance over time; and

determining a current state of the movable object based on the analysis of the distance.

21. The method of claim 20, further comprising:

when the distance stays below a first threshold distance for longer than a first time duration, concluding that the movable object is not in motion.

22. A device, comprising:

a memory storing instructions; and

a processor in communication with an imaging device mounted on a movable object, the processor configured to execute the instructions to;

in a view of an environment outside the movable object, identify a first region of interest (ROI) corresponding to a static portion of the environment, the first ROI covering at least an area in which vehicle and pedestrian traffic cannot be observed;

in the view of the environment, identify a second region of interest (ROI) corresponding to an active portion of the environment, the second ROI covering at least an area in which vehicle or pedestrian traffic can be observed;

receive, from the imaging device, first and second image data respectively representing the first and second ROIs;

analyze the first image data over time, wherein analyzing the first image data comprises performing a first comparison of an image property of the first ROI with a first threshold;

analyze the second image data over time, wherein analyzing the second image data comprises performing a second comparison of an image property of the second ROI with a second threshold; and

determine whether the movable object is in motion based on the analyses of the first and second image data.

23. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor in communication with an imaging device mounted on a movable object, cause the processor to perform operations comprising:

in a view of an environment outside the movable object, identifying a first region of interest (ROI) corresponding to a static portion of the environment, the first ROI covering at least an area in which vehicle and pedestrian traffic cannot be observed;

in the view of the environment, identifying a second region of interest (ROI) corresponding to an active portion of the environment, the second ROI covering at least an area in which vehicle or pedestrian traffic can be observed;

receiving, from the imaging device, first and second image data respectively representing the first and second ROIs;

analyzing the first image data over time, wherein analyzing the first image data comprises performing a first comparison of an image property of the first ROI with a first threshold;

analyzing the second image data over time, wherein analyzing the second image data comprises performing a second comparison of an image property of the second ROI with a second threshold; and

determining whether the movable object is in motion based on the analyses of the first and second image data.