Cognitive displays

• US 9,771,083 B2
• Filed: 12/31/2015
• Issued: 09/26/2017
• Est. Priority Date: 02/02/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Gordon, Michael S., Kozloski, James R., Malkin, Peter K., Pickover, Clifford A.
• Primary Examiner(s): Ghulamali, Qutbuddin
• Application Number:
  US14/986,377
• Publication Number:
  US 20160221502A1
• Time in Patent Office: 635 Days
• Field of Search: 340576, 3405737
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/25 to the sides of the vehicle
• CPC Class Codes: B60R 1/26 to the rear of the vehicle
• CPC Class Codes: B60R 1/28 with an adjustable field of view
• CPC Class Codes: B60R 2001/1253 with cameras, video cameras or video screens
• CPC Class Codes: B60R 2300/8026 in addition to a rear-view mirror system
• CPC Class Codes: B60W 2040/0818 Inactivity or incapacity of driver
• CPC Class Codes: B60W 40/08 related to drivers or passengers
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness

Abstract

A cognitive mirror apparatus includes an imaging device configured to capture a plurality of images, a controller configured to determine a cognitive state of a user; and a cognitive mirror configured to display a virtual reflection comprising one or more of the plurality of images, wherein the virtual reflection of the cognitive mirror is determined using the cognitive state of a user.

Claims

1. A cognitive mirror apparatus comprising:

an imaging device configured to capture a plurality of images;

a sensor capturing one or more parameters indicative of a cognitive state of a user including at least one of eye movement, facial expression, posture, and pulse;

a cognitive mirror configured to display a virtual reflection comprising one or more of the plurality of images adjusted by an application of adjustments in a controllable parameter of the cognitive mirror; and

a controller configured to determine the cognitive state of the user using the one or more parameters, wherein the sensor communicates the one or more parameters indicative of the cognitive state of the user to the controller, and wherein the controller includes a mapping relating the one or more parameters indicative of the cognitive state of the user to the controllable parameter of the cognitive mirror, and wherein the controller is further configured to sample the one or more parameters indicative of the cognitive state of the user to convergence on a threshold for confidence, while iteratively applying the adjustments in the controllable parameter of the cognitive mirror.

2. The cognitive mirror apparatus of claim 1, wherein the controllable parameter of the cognitive mirror is one of a shape of the cognitive mirror, an angle of the cognitive mirror relative to the user, and wherein the controller uses the mapping to select a value of the controllable parameter.

3. The cognitive mirror apparatus of claim 1, wherein the controllable parameter of the cognitive mirror is one of a modification and augmentation of the virtual reflection, and wherein the controller uses the mapping to select a value of the controllable parameter.

4. The cognitive mirror apparatus of claim 3, wherein the augmentation of the virtual reflection includes an alert.

5. The cognitive mirror apparatus of claim 3, wherein the modification of the virtual reflection includes a zoomed portion of the virtual reflection, a brightness of the virtual reflection, contrast of the virtual reflection, headlight dazzle control for a vehicle imaged in the virtual reflection.

6. The cognitive mirror apparatus of claim 1, further comprising a communication device receiving data indicative of a cognitive state of an operator in a vicinity of the user.

7. The cognitive mirror apparatus of claim 1, wherein the controller is disposed in a vehicle and is configured to determine a state of the vehicle, and wherein the controller includes a mapping relating the state of the vehicle to a controllable parameter of the cognitive mirror.

8. A cognitive mirror apparatus comprising:

an imaging device capturing a plurality of images;

a sensor capturing data indicative of a cognitive state of a user;

a controller, receiving the data indicative of the cognitive state of the user from the sensor, determining the cognitive state of the user using the data indicative of the cognitive state of the user, estimating a confidence C for the cognitive state of the user, and mapping the data indicative of the cognitive state of the user to a controllable parameter; and

a cognitive mirror displaying a virtual reflection, the virtual reflection comprising one or more of the plurality of images adjusted using the controllable parameter mapped by the controller to the data indicative of the cognitive state of the user,

wherein the controller iteratively receives the data indicative of the cognitive state of the user from the sensor, determines the cognitive state of the user using the data indicative of the cognitive state of the user, estimates the confidence C for the cognitive state of the user, and maps the data indicative of the cognitive state of the user to the controllable parameter adjusting the virtual reflection, until the confidence C for the cognitive state of the user converges with a threshold.

9. The cognitive mirror apparatus of claim 8, wherein the controllable parameter is one of a shape of the cognitive mirror, an angle of the cognitive mirror relative to the user.

10. The cognitive mirror apparatus of claim 8, wherein the controllable parameter is one of a modification and augmentation of the virtual reflection.

11. The cognitive mirror apparatus of claim 10, wherein the augmentation of the virtual reflection includes an alert.

12. The cognitive mirror apparatus of claim 10, wherein the modification of the virtual reflection includes a zoomed portion of the virtual reflection, a brightness of the virtual reflection, contrast of the virtual reflection, headlight dazzle control for a vehicle imaged in the virtual reflection.

13. The cognitive mirror apparatus of claim 8, further comprising a communication device connecting the controller to a network comprising at least one other communication device and receiving data indicative of a cognitive state of an operator in a vicinity of the user from the least one other communication device.

14. The cognitive mirror apparatus of claim 8, wherein the controller is disposed in a vehicle, the controller determining a state of the vehicle, and wherein the controller includes a mapping relating the state of the vehicle to a controllable parameter of the cognitive mirror.