System and method to assist users having reduced visual capability utilizing lighting device provided information

• US 10,130,548 B2
• Filed: 05/25/2018
• Issued: 11/20/2018
• Est. Priority Date: 05/13/2015
• Status: Active Grant

• Current Assignee: ABL IP Holding LLC (Acuity Brands, Inc.)
• Original Assignee: ABL IP Holding LLC (Acuity Brands, Inc.)
• Inventors: Lydecker, Stephen H., Ramer, David P., Rains, Jr., Jack C., Aggarwal, Januk
• Primary Examiner(s): Gray, Ryan M
• Application Number:
  US15/989,769
• Publication Number:
  US 20180271740A1
• Time in Patent Office: 179 Days
• Field of Search: None
• CPC Class Codes: A61H 2003/063 with tactile perception
• CPC Class Codes: A61H 2201/0188 Illumination related features
• CPC Class Codes: A61H 2201/1604 Head
• CPC Class Codes: A61H 2201/165 Wearable interfaces
• CPC Class Codes: A61H 2201/5007 computer controlled
• CPC Class Codes: A61H 2201/5043 Displays
• CPC Class Codes: A61H 2201/5048 Audio interfaces, e.g. voice or music controlled
• CPC Class Codes: A61H 2201/5082 Temperature sensors
• CPC Class Codes: A61H 2201/5092 Optical sensor
• CPC Class Codes: A61H 2201/5097 wireless
• CPC Class Codes: A61H 3/061 with electronic detecting or guiding means
• CPC Class Codes: G01C 21/206 specially adapted for indoor navigation
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0141 characterised by the informative content of the display
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06T 19/006 : Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09B 21/007 : using both tactile and audible presentation of the information
• CPC Class Codes: G09B 21/008 : using visual presentation of the information for the partially sighted
• CPC Class Codes: G09G 2354/00 : Aspects of interface with display user
• CPC Class Codes: G09G 5/12 : Synchronisation between the display unit and other units, e.g. other display units, video-disc players
• CPC Class Codes: G10L 15/22 : Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 : Execution procedure of a spoken command

Abstract

A lighting device obtains data related to objects and boundaries in an area in the vicinity of the lighting device, and a user wearable device provides a display (e.g. an augmented reality display based on the data related to the objects and the area boundaries) for a user/wearer. The lighting device includes a mapping sensor that collects data related to the objects and boundaries in the area. The user wearable device includes a camera or other optical sensor and wireless communication capability. The user wearable device is provided with mapping data that is presented on a display of the user wearable device. The communications and display capabilities allow the user wearable device to obtain room mapping information related to area in the vicinity of the lighting device in order to provide navigational assistance to a visually impaired person in the area.

Claims

1. A system, comprising:

a data storage containing mapping data of an area of a premises;

a lighting device configured to be installed in the area of the premises, the lighting device including;

a lighting device processor,

a wireless communication transceiver,

a light source, and

a memory that stores programming instructions, wherein;

the lighting device processor is coupled to the wireless communication transceiver, the memory, and the light source,

execution of the programming instructions stored in the memory configures the lighting device processor to control operations of the lighting device and perform functions, including a function to;

in response to a communication from a user device, deliver mapping data for the area of the premises in which the lighting device is located to the user device;

the system further includes the user device,

the user device includes a display, and

the user device is configured to use the mapping data to overlay a visual indicator on an unobstructed pathway, on a view of at least a portion of the area presented on the display, as the user device moves within the area.

2. The system of claim 1, further comprising:

a mapping sensor included in the lighting device configured to collect mapping data related to the area of the premises in which the lighting device is located, wherein the mapping sensor is coupled to the lighting device processor; and

a server, coupled to the lighting device processor and the database, the server configured to provide image processing of collected mapping data from the mapping sensor, the image processing including identifying objects, object boundaries, and inserting navigation instructions to form the mapping data contained in the data storage;

wherein the execution of the programming by the lighting device processor configures the lighting device processor to perform further functions, including functions to;

instruct the mapping sensor to collect mapping data of the area of the premises in which the lighting device is located; and

send the collected mapping data to the server for the image processing.

3. The system of claim 2, wherein:

the execution of the programming by the lighting device processor further configures the lighting device to send a lighting device identifier to the server with the collected mapping data, and

the lighting device identifier uniquely identifies location of the lighting device in the premises.

4. The system of claim 1, further comprising:

a mapping sensor included in the lighting device configured to collect mapping data related to the area of the premises in which the lighting device is located, wherein the mapping sensor is coupled to the lighting device processor;

wherein the execution of the programming by the lighting device processor configures the lighting device processor to perform further functions, including functions to;

instruct the mapping sensor to collect mapping data of the area of the premises in which the lighting device is located;

perform processing on the collected mapping data; and

store the data in the lighting device memory.

5. The system of claim 4, wherein the execution of the programming by the lighting device processor further configures the lighting device to forward a copy of the processed mapping data to the data storage for inclusion in the mapping data contained in the data storage.

6. The system of claim 1, wherein:

the user device further comprises a user device processor, a user device wireless communication transceiver, a user device memory, and user device programming instructions stored in the user device memory,

the user device processor is coupled to the user device wireless communication transceiver and the user device memory to control operations of the user device, and

execution of the user programming instructions by the user device processor configures the user device processor to perform functions, including functions to;

send a request for mapping data, via the user device wireless communication transceiver, related to a premises location of the user wearable device;

receive, from the lighting device, the mapping data for the area of the premises in which the lighting device is located; and

based on the received mapping data, cause the display to present the overlay of the visual indicator on the unobstructed pathway on the view presented on the display.

7. The system of claim 1, wherein the user device is further configured to use the mapping data to overlay another visual indicator on an object on the view presented on the display.

8. The system of claim 7, wherein the user device is further configured to overlay different visual indicators on the object and the unobstructed path.

9. The system of claim 8, wherein the different visual indicators include different colors of edge highlighting.

10. The system of claim 8, wherein the delivered mapping data includes one or more of an object identification, an object boundary indication, a navigation instruction or a navigation indicator.

11. The system of claim 1, wherein the user device comprises an eyewear frame having a structure to configure the display to be worn as goggles or eyeglasses.

12. A user wearable device, comprising:

a framework configured to enable a visually impaired user to wear the user wearable device;

a display supported by the framework so as to be viewable by at least one eye of the visually impaired user when wearing the user wearable device;

a wireless transceiver configured for communication with at least one element of a lighting system;

a processor, supported by the framework, coupled to the display, and the wireless transceiver, the processor being configured to control operations of the user wearable device;

a memory accessible to the processor and supported by the framework; and

programming in the memory, wherein execution of the programming by the processor configures the user wearable device to perform functions, including functions to;

obtain mapping data of an area of a premises served by the lighting system, via wireless communication with the at least one element of the lighting system; and

based on the mapping data, cause the display to present an overlay of a visual indicator an unobstructed pathway on a view of at least a portion of an area the premises.

13. The user wearable device of claim 12, wherein the execution of the programming by the processor further configures the user wearable device to use the mapping data to overlay another visual indicator on an object on the view presented on the display.

14. The user wearable device of claim 13, wherein the execution of the programming by the processor further configures the user wearable device to overlay different visual indicators on the object and the unobstructed path.

15. The user wearable device of claim 14, wherein the different visual indicators include different colors of edge highlighting.

16. The user wearable device of claim 14, wherein the mapping data includes one or more of an object identification, an object boundary indication, a navigation instruction or a navigation indicator.

17. The user wearable device of claim 12, wherein the framework comprises an eyewear frame having a structure to configure the user wearable device to be worn as goggles or eyeglasses.

18. The user wearable device of claim 12, further comprising:

a camera supported by the framework, wherein;

the camera is configured to capture an image of an area in which the user wearable device is located, and

the wireless transceiver is configured to receive the mapping from a lighting device of the lighting system in the area in which the user wearable device is located.

19. An article of manufacture, comprising:

a non-transitory machine readable medium; and

a navigational application program embodied in the medium for execution by a processor of a user wearable device having a display and a wireless transceiver, to configure the user wearable device to;

obtain mapping data of an area of a premises served by a lighting system, via wireless communication with at least one element of the lighting system in an area in which the user wearable device is located; and

based on the received mapping data, cause the display to present an overlay of different visual indicators on an object and on an unobstructed pathway on a view of at least a portion of the area in which the user wearable device is located.

20. The article of manufacture of claim 19, wherein the execution of the navigational application program further configures the user wearable device to use the mapping data to overlay another visual indicator on an object on the view presented on the display.

21. The article of manufacture of claim 20, wherein the execution of the navigational application program further configures the user wearable device to overlay different visual indicators on the object and the unobstructed path on the view presented on the display.

22. The user wearable device of claim 21, wherein the different visual indicators include different colors of edge highlighting.