Apparatus and method for estimating and using a predicted vehicle speed in an indirect vision driving task

• US 8,988,524 B2
• Filed: 03/11/2013
• Issued: 03/24/2015
• Est. Priority Date: 03/11/2013
• Status: Expired due to Fees

• Current Assignee: United States Of America As Represented By The Secretary Of The Army (Government of the United States of America)
• Original Assignee: The United States Of America As Represented By The Secretary Of The Army
• Inventors: Smyth, Christopher C.
• Primary Examiner(s): Czekaj, Dave
• Assistant Examiner(s): Beck, Leron
• Application Number:
  US13/792,585
• Publication Number:
  US 20140253722A1
• Time in Patent Office: 743 Days
• US Class Current: 348/121
• CPC Class Codes: B60R 1/24 in front of the vehicle
• CPC Class Codes: B60R 1/28 with an adjustable field of view
• CPC Class Codes: B60R 2300/207 using multi-purpose displays, e.g. camera image and navigation or video on same display
• CPC Class Codes: B60R 2300/302 combining image information with GPS information or vehicle data, e.g. vehicle speed, gyro, steering angle data
• CPC Class Codes: B60W 30/06 Automatic manoeuvring for parking
• CPC Class Codes: B60W 40/072 Curvature of the road
• CPC Class Codes: G01P 3/38 using photographic means
• CPC Class Codes: G06T 2207/20112 Image segmentation details
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/10 Segmentation; Edge detection motion-based segmentation G06T7/215
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: H04N 23/69 Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

A method and apparatus for predicting vehicle speed during an indirect vision driving task. A further method and apparatus for optimizing the display of a camera return during an indirect vision driving task based on operator perceived vehicle speed as set by the display characteristics and the field-of-view of the camera. A further method and apparatus for using the perceived speed as a driving task aid, in particular, as an electronic aider for optimizing the driving scene display characteristics of scene compression and camera field-of view. In this manner, the invention adjusts the perceived speed in order to match the operator'"'"'s cognitive flow to the control dynamics needed from the operator for the task. The invention has application to autonomous driving where manual intervention is incorporated during critical events for particular tasks; and with limited display space within the vehicle, the display format is adjusted by the invention according to the operator'"'"'s task needs.

Claims

1. A method for aiding an operator during performance of an indirect vision driving task while viewing a display of a video camera return of a driving scene via a display device, comprising:

determining parameters for both of a display being viewed during an indirect vision driving task and of a camera providing a camera return that is being displayed;

determining characteristics of a plurality of sections of a course in the driving scene being displayed to the operator via the display device;

determining an actual speed of the vehicle during the indirect vision driving task;

calculating an estimated operator-perceived vehicle speed as would be expected to be visually perceived and interpreted as being the actual vehicle speed by the operator during performance of the indirect vision driving task while viewing the display device for each of a plurality of sections of the course in the driving scene based on the above determinations; and

adaptively controlling, via a processor, the display of the driving scene on the display device to the operator in a manner so as to mitigate the difference between the estimated operator-perceived vehicle speed for the operator and the actual vehicle speed during the performance of the indirect vision driving task.

2. The method of claim 1, wherein calculating the estimated operator-perceived vehicle speed is based on determinations that include parameters of an optic flow locus point seen on the display being viewed by the operator, at a camera viewing distance and look-down angle as determined by the display and camera parameters, and geometric characteristics of the plurality of sections of driving course.

3. The method of claim 2, wherein the camera viewing distance and look-down angle are determined by a display scene compression ratio (α

) that is the ratio of the display field-of-view as seen by the operator to the camera field-of-view (FOV), and by the determined driving course characteristics.

4. The method of claim 3, in which the estimated operator-perceived vehicle speed is:

determined for a straight section of course separately by a linear speed seen generated at the optic flow locus point by forward motion the vehicle;

determined for a circular section of course with unlimited camera FOV separately by a composite of the linear speed and a rotational speed seen generated at the optic flow locus point by the vehicle forward motion and rotational motion; and

/or

determined for a circular course with a limited camera FOV separately by a composite of the linear speed and rotational speed seen generated at a camera-viewing limit by the vehicle forward motion and the rotational motion, where the locus point is outside of the camera FOV.

5. The method of claim 4, in which the estimated operator-perceived vehicle speed (V_p) is generated from the vehicle speed (V_M) and comprises:

an expression for a straight course, that is given by;

V_p=V_M*α

^+1/3, a function of the display scene compression ratio (α

);

an expression for a circular course with unlimited camera FOV, that is given by;

V_P=V_M*sqrt(1+(η

/(R*sin θ

′

_c))²)*α

^+1/3, a function of the radius of curvature (R), where θ

′

_c=a sin(η

*α

^+2/3/ρ

), and where ρ

=η

/sin θ

_c, is the camera viewing distance to the locus point, where θ

_c is the camera viewing angle to the locus point, and η

is the camera height above ground level; and

an expression for a circular course with a limited camera horizontal field-of view (FOV_L), that is given by;

V_P=V_M*sqrt(1+(η

/(R*sin θ

_L))²)*sin²(FOV_c/2)*α

^+1/3/sin²(FOV_L), where FOV_L<

FOV_c=2a sin(η

/(2R*tan θ

′

_c)), twice the horizontal viewing angle at the camera position to the locus point, and θ

_L=a tan(η

/(2R*sin(FOV_L/2))), the camera look-down angle to the ground as seen at the camera-viewing limit.

6. The method of claim 1, wherein adaptively controlling the display of the driving scene on the display device comprises alerting the operator that there is a difference between the estimated operator perceived vehicle speed for the operator and the actual vehicle speed.

7. The method of claim 1, wherein adaptively controlling the display of the driving scene on the display device comprises adjusting the format of the driving scene that is displayed to the operator such that the operator will visually perceive and interpret the vehicle speed during performance of the indirect vision driving task while viewing the display device as substantially being the actual speed of the vehicle.

8. The method of claim 7, wherein the format of the driving scene that is displayed is controlled in accordance with a control strategy that includes adjustments of one or more of:

a. parameters of the camera;

b. parameters of the display of the camera return;

c. the vehicle speed.

9. The method of claim 8, wherein the adjustments are made in a manner so as to generate a cognitive flow rate in the operator that is optimal for control dynamics required for the task.

10. The method of claim 8, wherein the control strategy comprises:

a. specifying task events composed of time periods and sub-tasks to be enabled during the time periods, and

b. specifying a reference path to be executed,

where the sub-tasks may comprise at least one of;

i. sub-task of orientation;

ii. task recall;

iii. enabling option review and decision;

iv. selection activation; and

v. continual control activation.

11. The method of claim 8, wherein the control strategy comprises modeling of the operator for the control strategy events, as one or more of:

a. an executor specifying a representative model of a knowledge base consisting of a feature set and rule scripts;

b. a state-space variable frame for matching features to the variables;

c. a rules processor comprising a state-space variable controller for specifying a control set point from the variable frame and the rule scripts; and

d. a skills processor based on a feed forward controller with feedback correction for reaching the control set point.

12. The method of claim 11, wherein modeling uses a micro-model, comprising:

a. a table of subtask elements composed of orientation, task recall, enabling option review and decision, and selection activation; and

b. a data base of corresponding subtask times at the millisecond level and workload as cost variables.

13. The method of claim 8 wherein the control strategy includes determining an attention state of the operator.

14. The method of claim 12 wherein the control strategy includes using the micro-model with the use of subtask times and workloads as task cost elements of an optimal cognitive flow rate in the operator.

15. The method of claim 10 wherein the control strategy includes modeling of the continual control activation as a feed-forward control loop based on a path prediction derived from a curvature of the reference path, and as a feedback control loop with at least heading as input based on evaluating path performance.

16. Apparatus for aiding an operator during performance of an indirect vision driving task performed by a vehicle operator while viewing a display of a video camera return of a driving scene via a display device, comprising:

a camera for generating a video camera return signal representative of an driving scene;

a video signal processor for applying the video camera return signal to a display for viewing of the driving scene by the vehicle operator;

a speed prediction modeler for predicting a driving speed, where the speed prediction modeler determines parameters for both of the display and of the video camera, determines characteristics of a plurality of sections of a course in the driving scene being displayed, determines an actual speed of the vehicle during the indirect vision driving task, and then calculates an estimated operator-perceived vehicle speed as would be expected to be visually perceived and interpreted as being the actual vehicle speed by the operator during performance of the indirect vision driving task while viewing the display device for each of a plurality of sections of the course in the driving scene based on the above determinations; and

an adapter aider for adaptively controlling the display of the driving scene on the display device to the operator in a manner so as to mitigate the difference between the estimated operator-perceived vehicle speed for the operator and the actual vehicle speed during the performance of the indirect vision driving task.

17. The apparatus of claim 16, wherein the adapter aider adaptively controls the display of the driving scene on the display device by alerting the operator that there is a difference between the estimated operator-perceived vehicle speed for the operator and the actual vehicle speed.

18. The apparatus of claim 16, wherein the adapter aider adaptively adjusts the format of the display of the driving scene to the operator such that the operator will visually perceive the vehicle speed during performance of the indirect vision driving task while viewing the display device as substantially being the actual speed of the vehicle.

19. The apparatus of claim 18, wherein the adapter aider controls the format of the display of the driving scene in accordance with a control strategy that includes adjustments of one or more of:

a. parameters of the camera;

b. parameters of the display of the camera return;

c. the vehicle speed.

20. The apparatus of claim 16 wherein the speed prediction modeler determines an expression for a straight section of course separately, an expression for a circular section of course with an unlimited field-of-view (FOV) separately, and/or an expression for a circular section of course with a limited FOV in calculating the estimated operator perceived vehicle speed separately.