Mobility assist device

• US 20040066376A1
• Filed: 07/25/2003
• Published: 04/08/2004
• Est. Priority Date: 07/18/2000
• Status: Abandoned Application

• Current Assignee: University of Minnesota
• Original Assignee: University of Minnesota
• Inventors: Pardhy, Sameer, Lim, Heon Min, Cheng, Pi-Ming, Alexander, Lee, Donath, Max, Gorjestani, Alec, Newstrom, Bryan, Shankwitz, Craig R.
• Application Number:
  US10/626,953
• Publication Number:
  US 20040066376A1
• Time in Patent Office: Days
• US Class Current: 345/169
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 2300/205 using a head-up display
• CPC Class Codes: B60R 2300/301 combining image information with other obstacle sensor information, e.g. using RADAR/LIDAR/SONAR sensors for estimating risk of collision
• CPC Class Codes: B60R 2300/302 combining image information with GPS information or vehicle data, e.g. vehicle speed, gyro, steering angle data
• CPC Class Codes: B60R 2300/305 merging camera image with lines or icons
• CPC Class Codes: B60R 2300/307 virtually distinguishing relevant parts of a scene from the background of the scene
• CPC Class Codes: B60R 2300/60 characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective
• CPC Class Codes: B60R 2300/804 for lane monitoring
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: B60T 2201/08 Lane monitoring; Lane Keeping Systems
• CPC Class Codes: B60T 2201/086 using driver related features
• CPC Class Codes: G01C 21/365 Guidance using head up displays or projectors, e.g. virtual vehicles or arrows projected on the windscreen or on the road itself

Abstract

The present invention is directed to a visual mobility assist device which provides a conformal, augmented display to assist a moving body. When the moving body is a motor vehicle, for instance (although it can be substantially any other body), the present invention assists the driver in either lane keeping or collision avoidance, or both. The system can display objects such as lane boundaries, targets, other navigational and guidance elements or objects, or a variety of other indicators, in proper perspective, to assist the driver.

Claims

1. A display on a mobile body, comprising:

a conformal, augmented display.

2. The display of claim 1 wherein the conformal, augmented display, comprises:

displayed objects, displayed at a perspective approximately equal to a perspective that would be perceived from an operator position at a location of the mobile body by an operator who has visual contact with actual objects corresponding to the displayed objects.

3. The display of claim 2 wherein the displayed objects include blocking templates displayed in a position to reduce glare.

4. The display of claim 2 wherein the displayed objects include enhanced text of signage located proximate to the mobile body.

5. The display of claim 1 wherein the conformal, augmented display comprises:

a guidance indicator guiding the mobile body in a desired direction.

6. The display of claim 2 wherein the displayed objects are positioned within a field of view of the operator in the operator position, at a location which approximately overlies the actual objects in the field of view.

7. The display of claim 6 wherein the displayed objects are see through.

8. The display of claim 6 wherein the displayed objects are displayed in a forward-looking field of view of the operator.

9. The display of claim 6 wherein the displayed objects are displayed in a rear or side view of the operator.

10. The display of claim 9 wherein the mobile body is a vehicle and wherein the displayed objects are displayed in a location simulating a perspective from the operator through a rearview mirror.

11. The display of claim 6 wherein the displayed objects are displayed in a side view of the operator.

12. The display of claim 11 wherein the mobile body is a vehicle and wherein the displayed objects are displayed in a location simulating a perspective from the operator through a side view mirror.

13. The display of claim 6 wherein the displayed objects comprise:

at least one of traffic lane markings or virtual path boundaries.

14. The display of claim 13 wherein the displayed objects comprise:

at least one of traffic lights, traffic signals and traffic signs.

15. The display of claim 13 wherein the displayed objects comprise:

landmarks.

16. The display of claim 1 wherein the conformal, augmented display, comprises:

displayed target objects, displayed at a perspective approximately equal to a perspective that would be perceived from an operator position at a location of the mobile body by an operator who has visual contact with actual targets corresponding to the displayed target objects.

17. The display of claim 16 wherein the displayed target objects are positioned within a field of view of the operator in the operator position, at a location which approximately overlies the actual target objects in the field of view.

18. The display of claim 17 wherein the displayed target elements are displayed in a forward-looking view of the operator.

19. The display of claim 18 wherein the mobile body comprises a vehicle and wherein the vehicle travels over a roadway and wherein the displayed target elements correspond to transitory targets, not fixed in place during normal operating circumstances of the roadway.

20. The display of claim 19 wherein the transitory targets comprise:

other vehicles proximate to the roadway.

21. The display of claim 19 wherein the transitory targets comprise:

pedestrians or animals proximate to the roadway.

22. The display of claim 6 and further comprising:

an object display indicative of objects outside the field of view of the driver.

23. The display of claim 22 wherein the object display is indicative of service or goods available in a vicinity of the mobile body.

24. The display of claim 1 and further comprising a warning display, warning of an object which the mobile body is approaching.

25. A mobility assist device, comprising:

a location system providing a location signal indicative of a location of a mobile body;

a data storage system storing object information indicative of objects located in a plurality of locations;

a display system; and

a controller coupled to the location system, the data storage system and the display system, and configured to receive the location signal and retrieve object information based on the location signal and provide a display signal to the display system such that the display system displays objects in substantially a correct perspective of an observer located at the location of the mobile body.

26. The mobility assist device of claim 25 wherein the display system is configured to provide a conformal augmented display of the objects based on the display signal.

27. The mobility assist device of claim 25 wherein the controller provides the display signal such that the objects are displayed at a position in a field of view of the observer at a location which substantially overlies the actual objects in the field of view.

28. The mobility assist device of claim 26 wherein the display system comprises:

a projection system providing a projection of an image of the objects; and

a partially reflective, partially transmissive screen, positioned in the field of view of the observer and positioned to receive the projection to allow the observer to see through the screen and to see the image of the objects projected thereon.

29. The mobility assist device of claim 25 and further comprising:

a ranging system, coupled to the controller and configured to detect transitory objects and provide a detection signal to the controller indicative of the location of the transitory object relative to the mobile body.

30. The mobility assist device of claim 29 wherein the controller is further configured to provide the display signal, based at least in part on the detection signal, such that the display system displays the transitory objects in substantially a correct perspective of an observer located at the location of the mobile body.

31. The mobility assist device of claim 25 wherein the controller is configured to filter the display signal such that the display system displays only transitory objects based on operator-selected criteria.

32. The mobility assist device of claim 25 wherein the controller is configured to filter the display signal such that the display system displays only transitory objects and selected objects indicated by the object information that have been selected for display.

33. The mobility assist device of claim 25 and further comprising:

a mobile body orientation detection system, coupled to the controller and the mobile body, detecting an orientation of the mobile body and providing an orientation signal to the controller.

34. The mobility assist device of claim 25 wherein the observer comprises a human with a head and further comprising:

a head orientation tracking system, coupled to the controller, detecting an orientation of the observer'"'"'s head and providing a head orientation signal to the controller.

35. The mobility assist device of claim 25 wherein the object information is intermittently updated.

36. The mobility assist device of claim 25 wherein the display system comprises a helmet-mounted display system.

37. The mobility assist device of claim 25 wherein the display system comprises a visor-mounted display system.

38. The mobility assist device of claim 25 wherein the display system comprises an eyeglass-mounted display system.

39. A method of monitoring operation of a mobility assist device having a location system providing a location signal indicative of a location of a mobile body, a data storage system storing object information indicative of objects located in a plurality of locations, a display system, a ranging system detecting a location of objects and transitory objects relative to the mobile body and providing an object detection signal based thereon, and a controller coupled to the location system, the data storage system, the ranging system and the display system, and configured to receive the location signal and the object detection signal and retrieve object information based on the location signal and provide a display signal to the display system such that the display system displays objects and transitory objects in substantially a correct perspective of an observer located at the location of the mobile body, the method comprising:

receiving the object detection signal;

determining whether the object detection signal correlates to the object information in the data storage system; and

providing an output at least indicative of a system problem when the object detection signal and the object information are determined not to correlate.

40. The method of claim 39 wherein determining whether the object detection signal correlates to the object information in the data storage system comprises:

accessing the data storage system based on the location signal; and

determining whether the object detection signal indicates the presence of objects indicated by the object information for the location of the mobile body.

41. The method of claim 39 wherein providing an output comprises:

when the object detection signal does not indicate the presence of objects indicated by the object information for the location of the mobile body, providing a user observable indication of a possible malfunction.

42. The method of claim 40 wherein providing an output comprises:

when the object detection signal indicates the presence of objects indicated by the object information for the location of the mobile body, providing a user observable indication of proper operation.

43. The method of claim 39 wherein providing an output comprises:

providing a visual display.

44. A method of controlling a mobility assist device having a location system providing a location signal indicative of a location of a mobile body, a data storage system storing object information indicative of objects located in a plurality of locations, a display system, a ranging system detecting a location of objects and transitory objects relative to the mobile body and providing an object detection signal based thereon, and a controller coupled to the location system, the data storage system, the ranging system and the display system, and comprising:

receiving the location signal and the object detection signal;

retrieving object information based on the location signal; and

providing a filtered display signal to the display system, the display signal being filtered such that the display system displays objects and transitory objects, based on operator selected filtering criteria, in substantially a correct perspective of an observer located at the location of the mobile body.

45. A mobility assist device, comprising:

a location system providing a location signal indicative of a location of a mobile body;

a data storage system storing object information indicative of objects located in a plurality of locations;

a neurostimulation system; and

a controller coupled to the location system, the data storage system and the neurostimulation system, and configured to receive the location signal and retrieve object information based on the location signal and provide a stimulation signal to the neurostimulation system.

46. The mobility assist device of claim 45 and further comprising:

a ranging system, coupled to the controller and configured to detect transitory objects and provide a detection signal to the controller indicative of the location of the transitory object relative to the mobile body.

47. The mobility assist device of claim 46 wherein the controller is further configured to provide the display signal, based at least in part on the detection signal.