Virtual mirror

• US 7,375,728 B2
• Filed: 10/01/2001
• Issued: 05/20/2008
• Est. Priority Date: 10/01/2001
• Status: Active Grant

• Current Assignee: University of Minnesota
• Original Assignee: University of Minnesota
• Inventors: Pardhy, Sameer, Cheng, Pi-Ming, Donath, Max, Shankwitz, Craig R.
• Primary Examiner(s): Nguyen; Phu K.
• Application Number:
  US09/968,724
• Publication Number:
  US 20030128182A1
• Time in Patent Office: 2,423 Days
• Field of Search: 345/419, 345/427, 345/660, 345/473, 345/973, 345/701, 345/707, 345/712, 345/771, 345/952, 345/702, 434/69, 348/118, 348/148, 340/435, 340/438, 359/13
• US Class Current: 345/427
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 2300/207 using multi-purpose displays, e.g. camera image and navigation or video on same display
• CPC Class Codes: B60R 2300/30 characterised by the type of image processing
• CPC Class Codes: B60R 2300/302 combining image information with GPS information or vehicle data, e.g. vehicle speed, gyro, steering angle data
• CPC Class Codes: B60R 2300/60 characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective
• CPC Class Codes: B60R 2300/8046 for replacing a rear-view mirror system
• CPC Class Codes: B60R 2300/8066 for monitoring rearward traffic
• CPC Class Codes: G01C 21/26 specially adapted for navigation in a road network
• CPC Class Codes: G06F 16/29 Geographical information databases

Abstract

A virtual mirror is rendered by a computer graphics display or screen. The display substantially replicates the optical properties of a physical mirror. The display is provided on a host vehicle.

Claims

1. A method of generating a display on a mobile body, comprising:

obtaining position information including coordinates that are indicative of a position of the mobile body on the surface of the Earth;

obtaining display data indicative of items in proximity to the mobile body based on the position information;

generating a virtual mirror display of the items that would be seen in a mirror with an unobstructed view based on the display data.

2. The method of claim 1 wherein generating a virtual mirror display comprises:

performing a mirror transformation on the display data, the mirror transformation transforming the display data into reflected data indicative of a reflection of the items.

3. The method of claim 2 wherein generating a virtual mirror display comprises:

performing a stencil transformation on the reflected data to obtain stencil data indicative of conformance of the reflected data to a simulated physical mirror boundary.

4. The method of claim 3 wherein generating a virtual mirror display comprises:

performing a forward transformation on the stencil data to obtain perspective data indicative of the stencil data in perspective on the virtual mirror display.

5. The method of claim 4 wherein generating a virtual mirror display comprises:

rendering the virtual mirror display based on the perspective data.

6. The method of claim 5 wherein generating a virtual mirror display comprises:

prior to performing the mirror transformation, determining whether the display data is indicative of an item that will be displayed.

7. The method of claim 6 wherein generating a virtual mirror display comprises:

if the display data is indicative of an item that will not be displayed, selecting other display data prior to performing the mirror transformation.

8. The method of claim 7 wherein generating a virtual mirror display comprises:

representing lines or object surface boundaries in the virtual mirror display by coordinate vectors; and

mathematically transforming the coordinate vectors to create visual representations of objects.

9. The method of claim 1 wherein obtaining position information comprises:

obtaining mobile body orientation data indicative of an orientation of the mobile body.

10. The method of claim 1 wherein obtaining display information comprises:

accessing a geospatial database based on the position information.

11. The method of claim 10 wherein obtaining display information comprises:

sensing objects proximate to the mobile body and not found in the geospatial database.

12. The method of claim 11 wherein the mobile body comprises a motor vehicle and wherein generating a virtual mirror display comprises:

generating the virtual mirror display as a rearview mirror on the motor vehicle displaying items obstructed from view by the motor vehicle.

13. The method of claim 1 and further comprising:

obtaining object information indicative of location of an object proximate the mobile body; and

generating the virtual mirror display based on the object information.

14. The method of claim 1 further comprising:

obtaining virtual mirror data indicative of spatial characteristics of the virtual mirror; and

generating the virtual mirror display based on the virtual mirror data.

15. The method of claim 14 wherein obtaining virtual mirror data comprises:

obtaining location, orientation, shape and size data indicative of a desired location, orientation, shape and size of the virtual mirror.

16. The method of claim 1 wherein obtaining virtual mirror data comprises:

obtaining virtual mirror data indicative of optical characteristics of the virtual mirror.

17. The method of claim 1 and further comprising:

obtaining user information indicative of an orientation of a user viewing the virtual mirror display; and

modifying the virtual mirror display based on changes in the user information.

18. The method of claim 17 wherein obtaining user information comprises:

tracking user eye orientation.

19. The method of claim 17 wherein obtaining user information comprises:

tracking user head orientation.

20. The method of claim 1 wherein generating a virtual mirror display comprises:

displaying the items differently based on a position of the items relative to the mobile body.

21. The method of claim 20 wherein displaying the items differently comprises:

changing a visual characteristic of the items based on a distance the items are from the mobile body.

22. The method of claim 20 wherein displaying the items differently comprises:

changing an attribute of the virtual mirror.

23. The method of claim 22 wherein changing an attribute of the virtual mirror comprises:

dynamically changing a computed radius of curvature of the virtual mirror in generating the virtual mirror display.

24. The method of claim 23 wherein changing an attribute of the mirror comprises:

changing the virtual mirror between, flat, concave and/or convex.

25. The method of claim 23 wherein dynamically changing a radius of curvature comprises:

changing the computed radius of curvature of different portions of the mirror such that the different portions have different radii of curvature.

26. The method of claim 1 wherein generating a virtual mirror display comprises:

generating a rendering of the items.

27. The method of claim 1 wherein generating a virtual mirror display comprises:

generating a plurality of virtual mirror displays.

28. The method of claim 1 wherein generating a virtual mirror display comprises:

generating a display element representative of the items.

29. The method of claim 1 wherein generating a virtual mirror display comprises:

selectively suppressing light reflection that would otherwise be seen in the virtual mirror display.

30. The method of claim 1 wherein generating a virtual mirror display comprises:

oscillating display of the items in the virtual mirror display to create depth.

31. The method of claim 1 and further comprising:

obtaining item information, indicative of physical characteristics of the items in proximity to the mobile body, by wireless transmission from the items.

32. The method of claim 31 wherein the items in proximity to the mobile body are motor vehicles having vehicle identification numbers (VINs) and wherein the vehicles transmit the VIN using wireless transmissions.

33. The method of claim 32 wherein a portion of the VIN is used to recreate a shape of the vehicle for display.

34. The method of claim 33 and further comprising:

accessing a database based on the portion of the VIN, and returning data indicative of the shape of the vehicle.

35. The method of claim 34 wherein generating a virtual mirror display comprises:

displaying the shape of the vehicle based on the data indicative of the shape.

36. The method of claim 31 wherein the physical characteristics comprise position and orientation of other mobile bodies.

37. The method of claim 1 and further comprising:

generating an alarm based on a position of the items relative to the mobile body.

38. The method of claim 1 wherein generating a virtual mirror display comprises:

rendering the display on a flat panel display.

39. The method of claim 1 and further comprising:

providing a stimulation output to stimulate a sense modality of a user based on a position of an item relative to the mobile body.

40. The method of claim 1 wherein generating a virtual mirror display comprises:

invoking exposed three dimensional graphics programming interfaces.

41. The method of claim 40 wherein invoking comprises:

defining a stencil to be drawn.

42. The method of claim 41 wherein invoking exposed programming interfaces further comprises:

defining a mirror coordinate system using a model view matrix.

43. The method of claim 42 wherein invoking further comprises:

rendering the stencil; and

activating portions of the stencil.

44. The method of claim 43 wherein invoking further comprises:

defining the model view matrix as a mirror transformation; and

rendering a mirror view based on the model view matrix.

45. The method of claim 44 wherein invoking further comprises:

defining the model view matrix as a forward-looking view;

activating different portions of the stencil; and

rendering the forward-looking view behind the virtual mirror.

46. A display system for generating a display on a mobile body, the display system comprising:

a location system generating location data including coordinates that are indicative of a location of the mobile body on the surface of the Earth;

a geospatial database storing display item information indicative of items to be displayed;

a display device; and

a controller coupled to the location system, the display device, and the geospatial database and configured to access the geospatial database based on the location data and output display data to the display device indicative of a virtual mirror display of the items.

47. The display system of claim 46 wherein the display device comprises a flat panel display.

48. The display system of claim 46 wherein the display device comprises a head mounted display.

49. The display system of claim 46 and further comprising:

a projector receiving the display data and projecting the virtual mirror display on the display device.

50. The display system of claim 46 and further comprising:

a ranging system, coupled to the controller, generating object data indicative of objects proximate the mobile body.

51. The display system of claim 50 wherein movement of the display device changes the virtual mirror display and the ranging system to generate object data based on a new virtual mirror display determined by a new position or orientation of the display device.

52. The display system of claim 46 and further comprising:

an imaging system, coupled to the controller, generating object data indicative of physical characteristics of objects proximate the mobile body.

53. The display system of claim 52 wherein the physical characteristics comprise at least one of shape, size and color.

54. The display system of claim 46 wherein the controller is configured to output the display data indicative of objects in the virtual mirror display.

55. The display system of claim 54 wherein the controller is configured to perform a mirror transformation on the display data, the mirror transformation transforming the display data into reflected data indicative of a reflection of the items.

56. The display system of claim 55 wherein the controller is configured to perform a stencil transformation on the reflected data to obtain stencil data indicative of conformance of the reflected data to a boundary of a mirror positioned or oriented in a desired way.

57. The display system of claim 56 wherein the controller is configured to perform a forward transformation on the stencil data to obtain perspective data indicative of the stencil data in proper perspective on the virtual mirror display.

58. The display system of claim 57 wherein the controller is configured to render the virtual mirror display based on the perspective data.

59. The display system of claim 46 wherein the controller is configured to sense a change in orientation of the display device and modify the display data based on the change in orientation of the display device.

60. The display system of claim 46 wherein the controller is configured to provide the display data as being indicative of the items and outlined portions of the mobile body that would otherwise obstruct a mirror view of the items.

61. A computer generated virtual mirror display on a mobile body, comprising:

a vehicle location system configured to generate a location signal including coordinates of a position of the mobile body on the surface of the Earth;

a geospatial database including road data, the road data including road boundary information corresponding to a boundary of a road on which the mobile body is traveling;

a conformal virtual mirror display configured to display a virtual image representing the boundary of the road based on the road boundary information and the location signal;

wherein the virtual image is displayed in a manner that generates a visual effect of the virtual image substantially overlying the corresponding boundary of the road that would be seen if the display were a mirror having an unobstructed view of the road.

62. The computer generated display of claim 61 wherein the display comprises:

a conformal mirror display of items that would be seen in a mirror mounted on the vehicle if the view were unobstructed.

63. The computer generated display of claim 62 wherein the items comprise:

road lane boundaries.

64. The computer generated display of claim 63 wherein the items comprise:

sensed objects proximate to the mobile body.

65. The computer generated display of claim 64 wherein the sensed objects comprise:

other mobile bodies.

66. The computer generated display of claim 64 wherein the items comprise:

at least one of traffic signs, guard rails, other road furniture, and landmarks.

67. The computer generated display of claim 62 wherein the mirror display is superimposed on an actual mirror.

68. The computer generated display of claim 67 wherein the mirror display emphasizes desired portions of the display.