User-adjusted display devices and methods of operating the same

• US 10,699,326 B2
• Filed: 12/28/2016
• Issued: 06/30/2020
• Est. Priority Date: 07/07/2016
• Status: Active Grant

• Current Assignee: NIO Technology Company Limited (Shanghai Anbin Technology Co., Ltd.)
• Original Assignee: Nio Usa Inc.
• Inventors: Ricci, Christopher P.
• Primary Examiner(s): Rudolph, Vincent
• Assistant Examiner(s): Schwartz, Raphael
• Application Number:
  US15/393,097
• Publication Number:
  US 20180012089A1
• Time in Patent Office: 1,280 Days
• CPC Class Codes: A61B 2503/22 Motor vehicles operators, e.g. drivers, pilots, captains
• CPC Class Codes: A61B 5/1171 based on the shapes or appearances of their bodies or parts thereof
• CPC Class Codes: A61B 5/1172 using fingerprinting
• CPC Class Codes: A61B 5/1176 Recognition of faces
• CPC Class Codes: B60K 2360/334 Projection means
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/212 displaying on manual operation elements, e.g. on a knob
• CPC Class Codes: B60K 6/20 the prime-movers consisting of electric motors and internal combustion engines, e.g. HEVs
• CPC Class Codes: B60L 2240/549 Current
• CPC Class Codes: B60L 2240/70 Interactions with external data bases, e.g. traffic centres
• CPC Class Codes: B60L 2240/72 Charging station selection relying on external data
• CPC Class Codes: B60L 2270/32 of electricity
• CPC Class Codes: B60L 5/24 Pantographs
• CPC Class Codes: B60L 50/53 in combination with an external power supply, e.g. from overhead contact lines
• CPC Class Codes: B60L 53/00 Methods of charging batteries, specially adapted for electric vehicles; Charging stations or on-board charging equipment therefor; Exchange of energy storage elements in electric vehicles
• CPC Class Codes: B60L 53/12 Inductive energy transfer
• CPC Class Codes: B60L 53/14 Conductive energy transfer
• CPC Class Codes: B60L 53/65 involving identification of vehicles or their battery types
• CPC Class Codes: B60L 53/665 Methods related to measuring, billing or payment
• CPC Class Codes: B60L 53/80 Exchanging energy storage elements, e.g. removable batteries
• CPC Class Codes: B60L 7/10 : Dynamic electric regenerative braking B60L7/22 takes precedence
• CPC Class Codes: B60L 8/003 : Converting light into electric energy, e.g. by using photo-voltaic systems
• CPC Class Codes: B60L 8/006 : Converting flow of air into electric energy, e.g. by using wind turbines
• CPC Class Codes: B60L 9/00 : Electric propulsion with power supply external to the vehicle electric propulsion for monorail vehicles, suspension vehicles or rack railways B60L13/00; in combination with batteries or fuel cells within the vehicle B60L50/53
• CPC Class Codes: B60M 1/00 : Power supply lines for contact with collector on vehicle collectors therefor B60L5/00
• CPC Class Codes: B60M 7/00 : Power lines or rails specially adapted for electrically-propelled vehicles of special types, e.g. suspension tramway, ropeway, underground railway
• CPC Class Codes: B60R 11/04 : Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60R 2011/0003 : inside the vehicle
• CPC Class Codes: B60R 2011/004 : outside the vehicle
• CPC Class Codes: B60R 2300/30 : characterised by the type of image processing
• CPC Class Codes: B60R 2300/804 : for lane monitoring
• CPC Class Codes: B60R 2325/105 : Radio frequency identification data [RFID]
• CPC Class Codes: B60R 25/102 : a signal being sent to a remote location, e.g. a radio signal being transmitted to a police station, a security company or the owner
• CPC Class Codes: B60R 25/2081 : combined with personal settings of other vehicle devices, e.g. mirrors, seats, steering wheels
• CPC Class Codes: B60W 2040/0809 : Driver authorisation; Driver identity check
• CPC Class Codes: B60W 2050/143 : Alarm means
• CPC Class Codes: B60W 2050/146 : Display means
• CPC Class Codes: B60W 2300/34 : Compact city vehicles, e.g., microcars or kei cars
• CPC Class Codes: B60W 2540/00 : Input parameters relating to occupants
• CPC Class Codes: B60W 2540/21 : Voice
• CPC Class Codes: B60W 2540/215 : Selection or confirmation of options
• CPC Class Codes: B60W 40/08 : related to drivers or passengers
• CPC Class Codes: B60W 50/08 : Interaction between the driver and the control system
• CPC Class Codes: B60Y 2200/91 : Electric vehicles
• CPC Class Codes: B60Y 2200/912 : Electric vehicles with power supply external to vehicle, e.g. trolley buses or trams
• CPC Class Codes: B60Y 2200/92 : Hybrid vehicles
• CPC Class Codes: B60Y 2300/60 : Control of electric machines, e.g. problems related to electric motors or generators
• CPC Class Codes: B60Y 2302/07 : Disabling particular vehicle functions, e.g. to affect the driving style
• CPC Class Codes: B60Y 2400/92 : Driver displays
• CPC Class Codes: G01C 21/36 : Input/output arrangements for on-board computers
• CPC Class Codes: G01C 21/3617 : using user history, behaviour, conditions or preferences, e.g. predicted or inferred from previous use or current movement
• CPC Class Codes: G01C 21/3697 : Output of additional, non-guidance related information, e.g. low fuel level G01C21/3679 takes precedence
• CPC Class Codes: G01S 2013/9316 : combined with communication equipment with other vehicles or with base stations
• CPC Class Codes: G05B 15/02 : electric
• CPC Class Codes: G05D 1/0011 : associated with a remote control arrangement
• CPC Class Codes: G05D 1/0088 : characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G06F 16/248 : Presentation of query results
• CPC Class Codes: G06F 16/951 : Indexing; Web crawling techniques
• CPC Class Codes: G06F 21/31 : User authentication
• CPC Class Codes: G06F 21/32 : using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06F 21/6245 : Protecting personal data, e.g. for financial or medical purposes
• CPC Class Codes: G06F 3/011 : Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06K 19/0708 : the source being electromagnetic or magnetic
• CPC Class Codes: G06K 7/10257 : arrangements for protecting the interrogation against piracy attacks computer security in general G06F21/00; jamming of communication, countermeasures H04K3/00; secret communication H04K1/00
• CPC Class Codes: G06K 7/10316 : using at least one antenna particularly designed for interrogating the wireless record carriers antennas in general H01Q1/22
• CPC Class Codes: G06K 7/10425 : the interrogation device being arranged for interrogation of record carriers passing by the interrogation device
• CPC Class Codes: G06Q 10/20 : Administration of product repair or maintenance
• CPC Class Codes: G06Q 20/105 : involving programming of a portable memory device, e.g. IC cards, "electronic purses"
• CPC Class Codes: G06Q 20/108 : Remote banking, e.g. home banking
• CPC Class Codes: G06Q 20/14 : specially adapted for billing systems
• CPC Class Codes: G06Q 20/308 : using the Internet of Things
• CPC Class Codes: G06Q 20/32 : using wireless devices
• CPC Class Codes: G06Q 20/3224 : Transactions dependent on location of M-devices
• CPC Class Codes: G06Q 20/401 : Transaction verification
• CPC Class Codes: G06Q 20/4012 : Verifying personal identification numbers [PIN]
• CPC Class Codes: G06Q 20/405 : Establishing or using transaction specific rules
• CPC Class Codes: G06Q 30/012 : Providing warranty services
• CPC Class Codes: G06Q 30/0206 : Price or cost determination based on market factors
• CPC Class Codes: G06Q 30/0208 : Trade or exchange of goods or services in exchange for incentives or rewards
• CPC Class Codes: G06Q 30/0601 : Electronic shopping [e-shopping]
• CPC Class Codes: G06Q 30/0609 : Buyer or seller confidence or verification
• CPC Class Codes: G06Q 30/0613 : Third-party assisted
• CPC Class Codes: G06Q 30/0625 : Directed, with specific intent or strategy
• CPC Class Codes: G06Q 30/0635 : Processing of requisition or of purchase orders
• CPC Class Codes: G06Q 30/0637 : Approvals
• CPC Class Codes: G06V 20/59 : inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G06V 20/597 : Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/10 : Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/1365 : Matching; Classification
• CPC Class Codes: G06V 40/172 : Classification, e.g. identification
• CPC Class Codes: G07B 15/063 : using wireless information transmission between the vehicle and a fixed station identification of vehicles G08G
• CPC Class Codes: G07C 5/008 : communicating information to a remotely located station transmission systems for measured values G08C
• CPC Class Codes: G07C 5/02 : Registering or indicating driving, working, idle, or waiting time only apparatus forming part of taximeters G07B13/00
• CPC Class Codes: G07C 5/0808 : Diagnosing performance data testing of vehicles G01M17/00; testing of electrical installation on vehicles G01R31/005
• CPC Class Codes: G07C 5/0816 : Indicating performance data, e.g. occurrence of a malfunction
• CPC Class Codes: G07C 5/0858 : wherein the data carrier is removable
• CPC Class Codes: G07C 9/00563 : using personal physical data of the operator, e.g. finger prints, retinal images, voicepatterns
• CPC Class Codes: G08G 1/017 : identifying vehicles G08G1/015, G08G1/054 take precedence
• CPC Class Codes: G08G 1/0962 : having an indicator mounted inside the vehicle, e.g. giving voice messages
• CPC Class Codes: G08G 1/09626 : where the origin of the information is within the own vehicle, e.g. a local storage device, digital map
• CPC Class Codes: G08G 1/096775 : where the origin of the information is a central station
• CPC Class Codes: G08G 1/096827 : where the route is computed onboard
• CPC Class Codes: G08G 1/096838 : where the user preferences are taken into account or the user selects one route out of a plurality
• CPC Class Codes: G08G 1/16 : Anti-collision systems road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08
• CPC Class Codes: G08G 1/20 : Monitoring the location of vehicles belonging to a group, e.g. fleet of vehicles, countable or determined number of vehicles
• CPC Class Codes: H01Q 1/325 : characterised by the location of the antenna on the vehicle
• CPC Class Codes: H01Q 1/3266 : using the mirror of the vehicle
• CPC Class Codes: H01Q 1/3275 : mounted on a horizontal surface of the vehicle, e.g. on roof, hood, trunk
• CPC Class Codes: H01Q 1/3283 : side-mounted antennas, e.g. bumper-mounted, door-mounted mounted on windscreens H01Q1/1271
• CPC Class Codes: H01Q 1/3291 : mounted in or on other locations inside the vehicle or vehicle body
• CPC Class Codes: H01Q 21/30 : Combinations of separate antenna units operating in different wavebands and connected to a common feeder system
• CPC Class Codes: H02J 7/0068 : Battery or charger load switching, e.g. concurrent charging and load supply H02J7/0013 takes precedence
• CPC Class Codes: H04B 5/79 : for data transfer in combination with power transfer
• CPC Class Codes: H04L 2209/80 : Wireless network architectures or network communication protocols for wireless network security H04W12/00
• CPC Class Codes: H04L 2209/805 : Lightweight hardware, e.g. radio-frequency identification [RFID] or sensor
• CPC Class Codes: H04L 2209/84 : Vehicles
• CPC Class Codes: H04L 63/0428 : wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 9/321 : involving a third party or a trusted authority
• CPC Class Codes: H04L 9/3226 : using a predetermined code, e.g. password, passphrase or PIN network architectures or network communication protocols for supporting authentication of entities using passwords in a packet data network H04L63/083
• CPC Class Codes: H04W 12/02 : Protecting privacy or anonymity, e.g. protecting personally identifiable information [PII]
• CPC Class Codes: H04W 12/03 : Protecting confidentiality, e.g. by encryption
• CPC Class Codes: H04W 12/04 : Key management, e.g. using generic bootstrapping architecture [GBA]
• CPC Class Codes: H04W 12/06 : Authentication
• CPC Class Codes: H04W 4/40 : for vehicles, e.g. vehicle-to-pedestrians [V2P]
• CPC Class Codes: H04W 4/44 : for communication between vehicles and infrastructures, e.g. vehicle-to-cloud [V2C] or vehicle-to-home [V2H]
• CPC Class Codes: H04W 4/46 : for vehicle-to-vehicle communication [V2V]
• CPC Class Codes: H04W 4/80 : Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication
• CPC Class Codes: Y02T 10/70 : Energy storage systems for electromobility, e.g. batteries
• CPC Class Codes: Y02T 10/7072 : Electromobility specific charging systems or methods for batteries, ultracapacitors, supercapacitors or double-layer capacitors
• CPC Class Codes: Y02T 10/72 : Electric energy management in electromobility
• CPC Class Codes: Y02T 90/12 : Electric charging stations
• CPC Class Codes: Y02T 90/14 : Plug-in electric vehicles
• CPC Class Codes: Y02T 90/16 : Information or communication technologies improving the operation of electric vehicles
• CPC Class Codes: Y02T 90/167 : Systems integrating technologies related to power network operation and communication or information technologies for supporting the interoperability of electric or hybrid vehicles, i.e. smartgrids as interface for battery charging of electric vehicles [EV] or hybrid vehicles [HEV]
• CPC Class Codes: Y04S 30/14 : Details associated with the interoperability, e.g. vehicle recognition, authentication, identification or billing

Abstract

A device includes a microprocessor and a computer readable medium coupled to the microprocessor. The computer readable medium includes instructions stored thereon that cause the microprocessor to receive output from at least one sensor monitoring an interior space of a vehicle and determine, based on the received output, a condition of an occupant in the interior space of the vehicle. The instructions cause the microprocessor to determine, based on the determined condition of the occupant, to alter a first presentation of information displayed to a display device of the vehicle to a second presentation of the information displayed to the display device. The instructions case the microprocessor to render the second presentation of the information to the display device of the vehicle to replace the first presentation of the information displayed to the display device of the vehicle.

Claims

1. A vehicle, comprising:

a camera to monitor an interior of the vehicle;

a microprocessor; and

a computer readable medium coupled to the microprocessor and comprising instructions stored thereon that cause the microprocessor to;

render a first presentation of selected information on a display device of the vehicle, the selected information in the first presentation including a certain item in a first location on the display device;

identify, from output of the camera, a temporary condition including a counted number of hand motions of an operator of the vehicle toward the certain item in the first presentation;

determine a priority level associated with the certain item, a magnitude of the priority level being related to a level of occupant safety;

automatically determine, when the priority level exceeds a threshold priority level and when the counted number of hand motions exceeds a threshold count of hand motions, to alter the first presentation of the selected information currently displayed by the display device of the vehicle to a different second presentation of the selected information to be displayed by the display device, the second presentation being associated with a higher degree of occupant safety than the first presentation and including at least the certain item of the selected information in a second location on the display device that is closer to the operator compared to the first location;

automatically render, in response to the determination to alter the first presentation to the second presentation, the second presentation to the display device of the vehicle to replace the display of the first presentation; and

upon occurrence of a selected triggering event associated with the identified temporary condition, automatically re-render the first presentation to the display device of the vehicle to replace the second presentation so that the certain item of information returns to the first location on the display device.

2. The vehicle of claim 1, wherein the microprocessor generates, from the camera output, a three-dimensional representation of the interior of the vehicle including one or more vehicle occupants, the one or more vehicle occupants comprising the operator, and identifies the temporary condition from the three-dimensional representation and wherein the microprocessor assigns each pixel of the three-dimensional representation a depth value based on an elapsed time between a light source emitting light and the camera receiving the light after reflection by the operator.

3. The vehicle of claim 2, wherein the microprocessor analyzes the light reflected from a pupil or cornea or of an occupant to detect a rotation of the eye of the operator.

4. The vehicle of claim 1, wherein the microprocessor determines a safety score for the identified temporary condition, the priority level being based on at least the safety score associated with the identified temporary condition, and a weighting factor associated with the safety score and determines the priority level based on the safety score and associated weighting factor, and

wherein the microprocessor temporarily increases the safety score in response to a change in the identified temporary condition, the change being related to reduced safety of the one or more vehicle occupants.

5. The vehicle of claim 3, wherein the identified temporary condition includes a skin temperature of the one or more occupants, wherein the change relates to the skin temperature being above a first temperature or below a second temperature, wherein the microprocessor determines the first temperature and the second temperature based on a current ambient temperature.

6. The vehicle of claim 1, wherein the selected triggering event is a determination by the microprocessor that the identified temporary condition has remained unchanged over a selected threshold time interval or the operator is no longer operating the vehicle, and wherein the first and second presentations differ in one or more of a size of the selected information displayed, a color of the selected information displayed, and a brightness of the selected information displayed.

7. The vehicle of claim 1, wherein a third presentation of the selected information has an associated third priority level and second temporary condition, wherein the second presentation has an associated second priority level and is associated with the identified temporary condition, wherein the second priority level is higher than the third priority level, wherein the microprocessor identifies that the second temporary condition is occurring concurrently with the identified temporary condition, and wherein the microprocessor uses the higher second priority level of the identified temporary condition rather than the lower third priority level of the third temporary condition in selecting the second presentation.

8. The vehicle of claim 1, wherein the display device includes a plurality of vehicle operational display portions that are physically separated from one another, and wherein the selected information in the second presentation appears in a different vehicle operational display portion than in the first presentation.

9. A method for operating a vehicle, comprising:

providing a vehicle comprising a camera to monitor an interior of the vehicle, a microprocessor, and a computer readable medium coupled to the microprocessor;

rendering, by the microprocessor, a first presentation of selected information on a display device of the vehicle, the selected information in the first presentation including a certain item in a first location on the display device;

identifying, by the microprocessor and from output of the camera, a temporary condition including a counted number of hand motions of an operator of the vehicle toward the certain item in the first presentation;

determining, by the microprocessor, a priority level associated with the certain item, a magnitude of the priority level being related to a level of occupant safety;

automatically determining, by the microprocessor and when the priority level exceeds a threshold priority level and when the counted number of hand motions exceeds a threshold count of hand motions, to alter the first presentation of the selected information currently displayed by the display device of the vehicle to a different second presentation of the selected information to be displayed by the display device, the second presentation, for the identified temporary condition, being associated with a higher degree of occupant safety than the first presentation and including at least the certain item of the selected information in a second location on the display device that is closer to the operator compared to the first location;

automatically rendering, by the microprocessor, in response to the determination to alter the first presentation to the second presentation, the second presentation to the display device of the vehicle to replace the display of the first presentation;

detecting, by the microprocessor, that a selected triggering event associated with the identified temporary condition has occurred; and

in response to the detecting, automatically re-rendering, by the microprocessor, the first presentation to the display device of the vehicle to replace the second presentation so that the certain item of information returns to the first location on the display device.

10. The method of claim 9, wherein the microprocessor generates, from the camera output, a three-dimensional representation of the interior of the vehicle including one or more vehicle occupants, the one or more vehicle occupants comprising the operator, and identifies the temporary condition from the three-dimensional representation and wherein the microprocessor assigns each pixel of the three-dimensional representation a depth value based on an elapsed time between a light source emitting light and the camera receiving the light after reflection by the operator.

11. The method of claim 10, wherein the microprocessor analyzes the light reflected from a pupil or cornea or of an occupant to detect a rotation of the eye of the operator.

12. The method of claim 9, wherein the microprocessor determines a safety score for the identified temporary condition, the priority level being based on at least the safety score associated with the identified temporary condition, and a weighting factor associated with the safety score and determines the priority level based on the safety score and associated weighting factor.

13. The method of claim 12, wherein the microprocessor assigns the priority level to the identified temporary condition by evaluating the priority score against one or more thresholds.

14. The method of claim 9, wherein the selected triggering event is a determination by the microprocessor that the identified temporary condition has remained unchanged over a selected threshold time interval or the operator is no longer operating the vehicle, and wherein the first and second presentations differ in one or more of a size of the selected information displayed, a color of the selected information displayed, and a brightness of the selected information displayed.

15. The method of claim 9, wherein a third presentation of the selected information has an associated third priority level and second temporary condition, wherein the second presentation has an associated second priority level and is associated with the identified temporary condition, wherein the second priority level is higher than the third priority level, wherein the microprocessor identifies that the second temporary condition is occurring concurrently with the identified temporary condition, and wherein the microprocessor uses the higher second priority level of the identified temporary condition rather than the lower third priority level of the third temporary condition in selecting the second presentation.

16. A method for operating a vehicle, comprising:

providing a vehicle comprising a camera to monitor an interior of the vehicle, a microprocessor, and a computer readable medium coupled to the microprocessor;

rendering, by the microprocessor, a first presentation of selected information on a display device of the vehicle, the selected information in the first presentation including a certain item in a first location on the display device;

identifying, by the microprocessor and from output of the camera, a temporary condition including a counted number of hand motions of an operator of the vehicle toward the certain item in the first presentation;

determining, by the microprocessor, a priority level associated with the certain item, a magnitude of the priority level being related to a level of occupant safety;

automatically determining, by the microprocessor and when the priority level exceeds a threshold priority level and when the counted number of hand motions exceeds a threshold count of hand motions within a threshold amount of time, to alter the first presentation of the selected information currently displayed by the display device of the vehicle to a different second presentation of the selected information to be displayed by the display device, the second presentation, for the identified temporary condition, having a higher degree of occupant safety than the first presentation, wherein the certain item of the selected information on the display device is in a different location in the second presentation such that the selected information is closer to the operator in the second presentation than in the first presentation;

automatically rendering, by the microprocessor, in response to the determination to alter the first presentation to the second presentation, the second presentation to the display device of the vehicle to replace the display of the first presentation;

detecting, by the microprocessor, that a selected triggering event associated with the identified temporary condition has occurred; and

in response to the detecting, automatically re-rendering, by the microprocessor, the first presentation to the display device of the vehicle to replace the second presentation so that the certain item of information returns to the first location on the display device, wherein the selected triggering event is a determination by the microprocessor that the operator is no longer operating the vehicle.

17. The method of claim 16, wherein a third presentation of the selected information has an associated third priority level and second temporary condition, wherein the second presentation has an associated second priority level and is associated with the identified temporary condition, wherein the second priority level is higher than the third priority level, wherein the microprocessor identifies that the second temporary condition is occurring concurrently with the identified temporary condition, and wherein the microprocessor uses the higher second priority level of the identified temporary condition rather than the lower third priority level of the third temporary condition in selecting the second presentation.