Method for displaying off-screen target indicators in motion video

• US 10,410,360 B2
• Filed: 12/15/2017
• Issued: 09/10/2019
• Est. Priority Date: 12/15/2017
• Status: Active Grant

• Current Assignee: the united states of america as represented by the secretary of the navy
• Original Assignee: The United States of America As Represented By The Secretary of Agriculture
• Inventors: Bagnall, Bryan D., Harguess, Joshua, Parameswaran, Shibin, Jaszewski, Martin T.
• Primary Examiner(s): Shin, Soo
• Application Number:
  US15/844,008
• Publication Number:
  US 20190188868A1
• Time in Patent Office: 634 Days
• Field of Search: None
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20092 Interactive image processing based on input by user
• CPC Class Codes: G06T 2207/20104 Interactive definition of region of interest [ROI]
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06T 7/75 involving models

Abstract

A method for displaying off-screen target indicators in motion video comprising the steps of receiving motion video containing a series of individual video frames, selecting a target object within a selected video frame by choosing selected target object pixel space coordinates, and determining whether the selected target object pixel space coordinates are within the selected video frame. Upon determining that the selected target object pixel space coordinates are within the selected video frame, the method updates a dynamical system model with the target object geographical coordinates, longitudinal target object speed, and latitudinal target object speed. Upon determining that the selected target object pixel space coordinates are not within the selected video frame, the method calculates estimated target object geographical coordinates at time t using the dynamical system model. The method then calculates final values in the video field of view at which to draw a target indicator.

Claims

1. A method for displaying target indicators in motion video comprising:

receiving a motion video containing a plurality of video frames;

initializing a dynamical system model;

selecting a target object within a selected video frame by choosing selected target object pixel space coordinates;

determining whether the selected target object pixel space coordinates are within the selected video frame;

upon determining the selected target object pixel space coordinates are within the selected video frame,

updating the dynamical system model with a target object geographical coordinates, longitudinal target object speed, and latitudinal target object speed;

upon determining the selected target object pixel space coordinates are not within the selected video frame,

calculating an estimated target object geographical coordinates at time t using the dynamical system model;

calculating a final value for the dynamical system model; and

drawing the target indicators at the final value for the dynamical system model.

2. The method of claim 1, wherein the step of upon determining the selected target object pixel space coordinates are within the selected video frame further comprises sub steps of

tracking the selected target object pixel space coordinates;

bilinearly transforming the selected target object pixel space coordinates into selected target object geographical coordinates; and

calculating the longitudinal target object speed and the latitudinal target object speed using the selected target object geographical coordinates.

3. The method of claim 1, wherein the longitudinal target object speed is calculated as a circle distance in a longitudinal direction between selected target object geographical coordinates in the selected video frame and the selected target object geographical coordinates in a previous video frame.

4. The method of claim 1, wherein the upon determining the selected target object pixel space coordinates are not within the selected video frame step further comprises substeps of

stopping the updating of the dynamical system model;

calculating an estimated target object geographical coordinates at the time t using the dynamical system model; and

transforming the estimated target object geographical coordinates into estimated target object pixel space coordinates.

5. The method of claim 4, further comprising a step of calculating a field of view midpoint.

6. The method of claim 4, further comprising steps of determining whether the estimated target pixel space coordinates are behind a frame of view;

upon determining the target object pixel space coordinates are behind the frame of view,

updating the target object pixel space coordinates by multiplying the target object pixel space coordinates by negative one; and

upon determining the target object pixel space coordinates are not behind the frame of view,

leaving the target object pixel space coordinates unchanged.

7. The method of claim 4, further comprising steps of

calculating adjusted values by subtracting a field of view midpoint from the estimated target object pixel space coordinates;

calculating a first slope between the estimated target object pixel space coordinates and the field of view midpoint;

calculating screen bounds values by multiplying the field of view midpoint by 0.9;

updating the adjusted values using the screen bounds values and the first slope; and

calculating the final value for the dynamical system model by adding the field of view midpoint to the adjusted values.

8. The method of claim 4, further comprising a step of calculating a first distance between a field of view midpoint and the estimated target object geographic coordinates.

9. The method of claim 5, wherein the field of view midpoint is calculated by multiplying a frame of view dimension by one-half.

10. The method of claim 8, wherein the first distance is calculated by transforming the field of view midpoint into screen center geographic coordinates, and calculating a second distance between the screen center geographic coordinates and the estimated target object geographic coordinates.

11. The method of claim 8, wherein the target indicator displays the first distance.

12. The method of claim 2, wherein the dynamical system model is a Kalman filter, and wherein the target object is tracked by a key point tracking method.

13. A method for displaying target indicators in motion video comprising:

receiving a motion video containing a plurality of video frames;

selecting a target object within a selected video frame by choosing selected target object pixel space coordinates;

initializing a dynamical system model;

determining whether the selected target object pixel space coordinates are within the selected video frame;

upon determining the selected target object pixel space coordinates are within the selected video frame,

tracking the selected target object pixel space coordinates;

bilinearly transforming the selected target object pixel space coordinates into selected target object geographical coordinates;

calculating a longitudinal target object speed and a latitudinal target object speed using the selected target object geographical coordinates;

updating the dynamical system model with the selected target object geographical coordinates, the longitudinal target object speed, and the latitudinal target object speed;

upon determining the selected target object pixel space coordinates are not within the selected video frame,

stopping the updating of the dynamical system model;

calculating an estimated target object geographical coordinates at time t using the dynamical system model;

transforming the estimated target object geographical coordinates into estimated target object pixel space coordinates;

calculating a field of view midpoint;

determining whether the estimated target object pixel space coordinates are behind a frame of view;

upon determining the target object pixel space coordinates are behind the frame of view,

updating the target object pixel space coordinates by multiplying the target object pixel space coordinates by negative one;

upon determining the target object pixel space coordinates are not behind the frame of view,

leaving the target object pixel space coordinates unchanged;

calculating adjusted values by subtracting the field of view midpoint from the estimated target object pixel space coordinates;

calculating a first slope between the estimated target object pixel space coordinates and the field of view midpoint;

calculating screen bounds values by multiplying the field of view midpoint by 0.9;

updating the adjusted values using the screen bounds value and the first slope;

calculating a final value for the dynamical system model by adding the field of view midpoint to the adjusted values;

calculating a first distance between the field of view midpoint and the estimated target object geographic coordinates; and

drawing the target indicators at the final value for the dynamical system model.

14. The method of claim 13, wherein the longitudinal target object speed is calculated as a circle distance in a longitudinal direction between the selected target object geographical coordinates in the selected video frame and selected target object geographical coordinates in a previous video frame.

15. The method of claim 13, wherein the field of view midpoint is calculated by multiplying a frame of view dimension by one-half.

16. The method of claim 13, wherein the first distance is calculated by transforming the field of view midpoint into screen center geographic coordinates, and calculating a second distance between the screen center geographic coordinates and the estimated target object geographical coordinates.

17. The method of claim 13, wherein the target indicator displays the first distance.

18. The method of claim 13, wherein the dynamical system model is a Kalman filter.

19. The method of claim 13, wherein the target object is tracked by a key point tracking method.

20. A method for displaying target indicators in motion video comprising:

receiving a motion video containing a plurality of video frames;

selecting a target object within a selected video frame by choosing selected target object pixel space coordinates;

initializing a dynamical system model;

determining whether the selected target object pixel space coordinates are within the selected video frame;

upon determining the selected target object pixel space coordinates are within the selected video frame,

tracking the selected target object pixel space coordinates;

bilinearly transforming the selected target object pixel space coordinates into selected target object geographical coordinates;

calculating a longitudinal target object speed and a latitudinal target object speed using the selected target object geographical coordinates;

updating the dynamical system model with the selected target object geographical coordinates, longitudinal target object speed, and latitudinal target object speed;

wherein the longitudinal target object speed is calculated as a great circle distance in a longitudinal direction between the selected target object geographical coordinates in the selected video frame and the selected target object geographical coordinates in a previous video frame;

upon determining the selected target object pixel space coordinates are not within the selected video frame,

stopping the updating of the dynamical system model;

calculating an estimated target object geographical coordinates at time t using the dynamical system model;

transforming the estimated target object geographical coordinates into estimated target object pixel space coordinates;

calculating a field of view midpoint;

determining whether the estimated target object pixel space coordinates are behind a frame of view;

upon determining the target object pixel space coordinates are behind the frame of view,

updating the target object pixel space coordinates by multiplying the target object pixel space coordinates by negative one;

upon determining the target object pixel space coordinates are not behind the frame of view,

leaving the target object pixel space coordinates unchanged;

calculating adjusted values by subtracting the field of view midpoint from the estimated target object pixel space coordinates;

calculating a first slope between the estimated target object pixel space coordinates and the field of view midpoint;

calculating screen bounds values by multiplying the field of view midpoint by 0.9;

updating the adjusted values using the screen bounds value and the first slope;

calculating final value by adding the field of view midpoint to the adjusted values;

calculating a first distance between the field of view midpoint and the estimated target object geographic coordinates;

drawing a target indicator at the final value;

wherein the field of view midpoint is calculated by multiplying a frame of view dimension by one-half;

wherein the first distance is calculated by transforming the field of view midpoint into screen center geographic coordinates, and calculating the distance between the screen center geographic coordinates and the estimated target object geographical coordinates;

wherein the target indicator displays the first distance;

wherein the dynamical system model is a Kalman filter, and wherein the target object is tracked by a key point tracking method.