Projection type display device and operation assistance method

• US 10,336,192 B2
• Filed: 06/29/2017
• Issued: 07/02/2019
• Est. Priority Date: 03/12/2015
• Status: Active Grant

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Inventors: Fujita, Koudai
• Primary Examiner(s): Liang, Dong Hui
• Application Number:
  US15/638,344
• Publication Number:
  US 20170355265A1
• Time in Patent Office: 733 Days
• CPC Class Codes: B60K 2360/16 Type of output information
• CPC Class Codes: B60K 2360/18 Information management
• CPC Class Codes: B60K 2360/334 Projection means
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/20 Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 35/23 Head-up displays [HUD] optical aspects of head-up displays G02B27/01
• CPC Class Codes: B60K 35/28 characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B60K 35/29 Instruments characterised by the way in which information is handled, e.g. showing information on plural displays or prioritising information according to driving conditions
• CPC Class Codes: B60K 35/60 Instruments characterised by their location or relative disposition in or on vehicles arrangements of lighting devices on dashboards B60Q3/10
• CPC Class Codes: B60K 35/81 for controlling displays
• CPC Class Codes: B60Q 9/008 for anti-collision purposes
• CPC Class Codes: B60W 50/029 Adapting to failures or work around with other constraints, e.g. circumvention by avoiding use of failed parts
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0141 characterised by the informative content of the display
• CPC Class Codes: G02B 27/01 Head-up displays
• CPC Class Codes: G02B 27/0101 characterised by optical features G02B27/0172 takes precedence
• CPC Class Codes: G09G 2320/10 Special adaptations of display systems for operation with variable images
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 2380/10 : Automotive applications
• CPC Class Codes: G09G 3/002 : to project the image of a two-dimensional display, such as an array of light emitting or modulating elements or a CRT
• CPC Class Codes: G09G 5/00 : Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T
• CPC Class Codes: H04N 5/74 : Projection arrangements for image reproduction, e.g. using eidophor
• CPC Class Codes: H04N 9/3194 : including sensor feedback

Abstract

Provided are a projection type display device and an operation assistance method that enable an operator to enjoy an operation assistance function even in the case of occurrence of abnormality in an optical modulation unit or an image information input unit during operation of a vehicle. A system control unit 47 of a HUD, if sensing abnormality in an optical modulation device 44 and a drive unit 45, stops a light source unit 40 and, furthermore, in the case of image information generated by a projected image information generation unit 52 including image information that calls attention of the operator to the front field of view, displays an alert image based on the image information on a liquid crystal display device 61 in a meter cluster.

Claims

1. A projection type display device comprising:

a first optical modulation unit that spatially modulates light emitted from a first light source unit in accordance with image information;

a first transmitter that inputs the image information into the first optical modulation unit;

a first projection unit that projects the light spatially modulated by the first optical modulation unit to a projection surface disposed on a windshield of a vehicle;

a first processor configured to perform as;

a first image information generation unit that generates the image information; and

a first abnormality sensing unit that senses abnormality in the first optical modulation unit, wherein the abnormality in the first optical modulation unit is determined when a degree of match between a calculated projected image information of the first optical modulation unit and the imaqe information generated from the first image information generation unit is less than or equal to a threshold; and

a first control unit that

stops at least one of the first optical modulation unit or the first light source unit when the first abnormality sensing unit senses abnormality in the first optical modulation unit,

determines whether or not the image information generated by the first image information generation unit includes image information which calls attention of an operator of the vehicle to a field of view in a travelling direction of the vehicle, wherein the image information which calls attention of the operator of the vehicle to the field of view in the travelling direction of the vehicle is generated due to at least one of an obstacle and a dangerous object in the travelling direction of the vehicle,

produces a first output for calling attention of the operator to the field of view when the image information includes the image information calling attention, and

produces no first output for calling attention of the operator to the field of view when the image information does not include the image information calling attention,

wherein the first control unit displays, on a display device mounted in the vehicle, an icon calling attention and a message calling attention as part of the first output for calling attention,

wherein the display device is a projection type display device comprising;

a second optical modulation unit that spatially modulates light emitted from a second light source unit in accordance with image information;

a second transmitter that inputs the image information into the second optical modulation unit;

a second projection unit that projects the light spatially modulated by the second optical modulation unit to the projection surface disposed on the windshield of the vehicle;

a second processor configured to perform as;

a second image information generation unit that generates the image information; and

a second abnormality sensing unit that senses abnormality in at least one of the second optical modulation unit or the second transmitter; and

a second control unit that stops at least one of the second optical modulation unit or the second light source unit when the second abnormality sensing unit senses abnormality in at least one of the second optical modulation unit or second the transmitter, determines whether or not the image information generated by the second image information generation unit includes image information which calls attention of the operator of the vehicle to the field of view in a travelling direction of the vehicle, produces a second output for calling attention of the operator to the field of view when the image information includes the image information calling attention, and produces no second output for calling attention of the operator to the field of view when the image information does not include the image information calling attention.

2. An operation assistance method that uses a projection type display device mounted in the vehicle according to claim 1,

the projection type display device including the first processor configured to perform as the first image information generation unit that generates image information, the first optical modulation unit that spatially modulates light emitted from the first light source unit in accordance with the image information, the first transmitter that inputs the image information into the first optical modulation unit, and the first projection unit that projects the light spatially modulated by the first optical modulation unit to the projection surface disposed on the windshield of the vehicle, and

the method comprising:

sensing abnormality in at least one of the first optical modulation unit, wherein the abnormality in the first optical modulation unit is determined when the degree of match between the calculated projected image information of the first optical modulation unit and the image information generated from the first image information generation unit is less than or equal to the threshold; and

stopping at least one of the first optical modulation unit or the first light source unit when sensing abnormality in at least one of the first optical modulation unit,

determining whether or not the image information generated by the first image information generation unit includes image information which calls attention of the operator of the vehicle to the field of view in the travelling direction of the vehicle, wherein the image information which calls attention of the operator of the vehicle to the field of view in the travelling direction of the vehicle is generated due to at least one of the obstacle and the dangerous object in the travelling direction of the vehicle,

producing the first output for calling attention of the operator to the field of view when the image information includes the image information calling attention, and

producing no first output for calling attention of the operator to the field of view when the image information does not include the image information calling attention,

wherein the icon calling attention and the message calling attention are displayed, on the display device mounted in the vehicle, as part of the first output for calling attention,

wherein the display device is a projection type display device comprising;

the second optical modulation unit that spatially modulates light emitted from the second light source unit in accordance with image information;

the second transmitter that inputs the image information into the second optical modulation unit;

the second projection unit that projects the light spatially modulated by the second optical modulation unit to the projection surface disposed on the windshield of the vehicle;

the second processor configured to perform as;

the second image information generation unit that generates the image information; and

the second abnormality sensing unit that senses abnormality in at least one of the second optical modulation unit or the second transmitter; and

the second control unit that stops at least one of the second optical modulation unit or the second light source unit when the second abnormality sensing unit senses abnormality in at least one of the second optical modulation unit or the second transmitter, determines whether or not the image information generated by the second image information generation unit includes image information which calls attention of the operator of the vehicle to the field of view in the travelling direction of the vehicle, produces the second output for calling attention of the operator to the field of view when the image information includes the image information calling attention, and produces no second output for calling attention of the operator to the field of view when the image information does not include the image information calling attention.

3. The operation assistance method according to claim 2, wherein, the part of the first output for calling attention is displayed on the display device in a position that corresponds to a projection position, on the projection surface, of light corresponding to the image information calling attention.

4. The operation assistance method according to claim 2, wherein the display device further displays an image other than the part of the first output for calling attention, and

a part of the image other than the part of the first output for calling attention, displayed on the display device is replaced with the part of the first output for calling attention.

5. The operation assistance method according to claim 2,

wherein an image for notifying the sensed abnormality is further displayed on the display device.

6. The operation assistance method according to claim 2,

wherein audio for calling attention of the operator to the field of view is output.

7. The operation assistance method according to claim 2,

wherein the vehicle is an automobile.

8. The projection type display device according to claim 1, wherein the first control unit displays, on the display device, the part of the first output for calling attention in a position that corresponds to a projection position, on the projection surface, of light corresponding to the image information calling attention.

9. The projection type display device according to claim 1, wherein the display device further displays an image other than the part of the first output for calling attention, and

the first control unit replaces a part of the image other than the part of the first output for calling attention, displayed on the display device with the part of the first output for calling attention.

10. The projection type display device according to claim 1,

wherein the first control unit further displays an image for notifying the abnormality sensed by the first abnormality sensing unit on the display device.

11. The projection type display device according to claim 1,

wherein the first control unit outputs audio for calling attention of the operator to the field of view.

12. The projection type display device according to claim 1,

wherein the vehicle is an automobile.