System and method for alerting visually impaired users of nearby objects

• US 9,801,778 B2
• Filed: 06/18/2010
• Issued: 10/31/2017
• Est. Priority Date: 06/19/2009
• Status: Active Grant

• Current Assignee: Andrew Mahoney
• Original Assignee: Andrew Mahoney
• Inventors: Mahoney, Andrew
• Primary Examiner(s): Ustaris, Joseph
• Assistant Examiner(s): SECHSER, JILL D
• Application Number:
  US13/375,316
• Publication Number:
  US 20120127291A1
• Time in Patent Office: 2,692 Days
• CPC Class Codes: A61F 9/08 Devices or methods enabling eye-patients to replace direct visual perception by another kind of perception
• CPC Class Codes: A61H 2003/063 with tactile perception
• CPC Class Codes: A61H 2201/5084 Acceleration sensors
• CPC Class Codes: A61H 3/061 with electronic detecting or guiding means
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G09B 21/007 using both tactile and audible presentation of the information

Abstract

A system and method for assisting a visually impaired user including an imaging device, a processing unit for receiving images from the imaging device and converting the images into signals for use by one or more controllers, and one or more vibro-tactile devices, wherein the one or more controllers activates one or more of the vibro-tactile devices in response to said signals received from the processing unit. The system preferably includes a lanyard to be worn around the neck of the user such that a first vibro-tactile device is arranged on the right side of the user'"'"'s neck, a second vibro-tactile device on a left side of the user'"'"'s neck, and a third vibro-tactile device at the back portion of the user'"'"'s neck. The vibro-tactile devices are activated depending on a determined position in front of the user of an object and the distance from the user to the object.

Claims

1. A method for alerting a visually impaired user of collision or falling hazards, the method comprising:

acquiring a first image and a second image, of a space in front of said user, by two spaced-apart imagers;

deriving, from said first image and said second image, a three-dimensional reconstruction comprising a plurality of three-dimensional points;

dividing said space in front of said user into a plurality of collision frusta each having a three-dimensional location and an orientation relative to said user, for detecting one of;

a presence of objects proximate a head of said user and an absence of a floor ahead of said user;

determining one of;

said presence of objects proximate said head of said user and said absence of said floor ahead of said user, based on counting a number of said plurality of three-dimensional points in each one of said plurality of collision frusta; and

activating one or more vibro-tactile devices, in a particular way, in response to said determining;

wherein said plurality of collision frusta comprises;

a right collision frustum, a center collision frustum, and a left collision frustum, each representing a corresponding right, center, and left portion of said space in front of said user, respectively; and

wherein said one or more vibro-tactile devices, comprise a right, a center, and a left vibro-tactile devices, and wherein said activating one or more vibro-tactile devices comprises one of;

activating said right vibro-tactile device in response to said determining in said right collision frustum, activating said center vibro-tactile device in response to said determining in said center collision frustum, and activating said left vibro-tactile device in response to said determining in said left collision frustum.

2. A method for alerting a visually impaired user of collision, the method comprising:

acquiring a first image and a second image, of a space in front of said user, by two spaced-apart imagers;

deriving, from said first image and said second image, a three-dimensional reconstruction comprising a plurality of three-dimensional points;

dividing said space in front of said user into a right collision frustum, a center collision frustum, and a left collision frustum, each having a three-dimensional location and orientation, relative to said user;

further dividing each of said right collision frustum, said center collision frustum, and said left collision frustum into three collision frusta to provide a near right collision frustum, a near center collision frustum, a near left collision frustum, a medium right collision frustum, a medium center collision frustum, a medium left collision frustum, a far right collision frustum, a far center collision frustum and a far left collision frustum, said frusta being relative to said user;

determining a presence of one or more objects proximate said user, based on counting a number of said plurality of three-dimensional points in each one of;

said near right collision frustum, said near center collision frustum, said near left collision frustum, said medium right collision frustum, said medium center collision frustum, said medium left collision frustum, said far right collision frustum, said far center collision frustum and said far left collision frustum;

when an object is determined to be present in adjacent collision frusta, a priority decision is made; and

activating one of;

a left, a center, and a right vibro-tactile device, in a particular way, based on said priority decision.

3. A method according to claim 2, wherein said priority decision comprises prioritizing a near collision frustum over a medium collision frustum, and prioritizing a medium collision frustum over a far collision frustum.

4. A method according to claim 2, wherein said priority decision comprises prioritizing a center collision frustum over both a left collision frustum and a right collision frustum.

5. A method according to claim 3 wherein activating one of:

said left, center, and right vibro-tactile devices, in a particular way, comprises activating one of;

said left, center, and right vibro-tactile devices at a low speed, a medium speed, or a high speed based on said priority decision.

6. A method according to claim 3 wherein activating one of:

said left, center, and right vibro-tactile devices, in a particular way, comprises activating one of;

said left, center, and right vibro-tactile devices for a short duration, a medium duration, or a long duration based on said priority decision.

7. A method according to claim 5 wherein activating one of said left, center, and right vibro-tactile devices, at a particular speed comprises one of:

activating one of;

said left vibro-tactile device, said right-vibro-tactile device, and said center vibro-tactile device, at a high speed, if said priority decision indicates said presence of said one or more objects in a corresponding one of;

said near left collision frustum, said near center collision frustum, and said near right collision frustum;

activating one of;

said left vibro-tactile device, said center vibro-tactile device, and said right vibro-tactile device, at a medium speed, if said priority decision indicates said presence of said one or more objects in a corresponding one of;

said medium left collision frustum, said medium center collision frustum, and said medium right collision frustum; and

activating one of;

said left vibro-tactile device, said right-vibro-tactile device, and said center vibro-tactile device, at a low speed, if said priority decision indicates said presence of said one or more objects in a corresponding one of;

said far left collision frustum, said far center collision frustum, and said far right collision frustum.

8. A method according to claim 2 wherein said activating said left, center, or right vibro-tactile devices, comprises:

activating said left vibro-tactile device if said priority decision indicates said presence of said one or more objects in one of;

said near left collision frustum, said medium left collision frustum, and said far left collision frustum;

activating said center vibro-tactile device if said priority decision indicates said presence of said one or more objects in one of;

said near center collision frustum, said medium center collision frustum, and said far center collision frustum; and

activating said right vibro-tactile device if said priority decision indicates said presence of said one or more objects in one of;

said near right collision frustum, said medium right collision frustum, and said far right collision frustum.

9. A system for assisting a visually impaired user comprising:

a first imager and a second imager spaced apart producing a first image and a second image of a space in front of said user;

a processing unit;

receiving said first image and said second image, and deriving from said first image and said second image a three-dimensional reconstruction comprising a plurality of three-dimensional points,

dividing said space in front of said user into a plurality of collision frusta each having a three-dimensional location and an orientation relative to said user for detecting one of;

a presence of objects proximate a head of said user and an absence of a floor ahead of said user, and

determining one of;

said presence of objects proximate said head of said user and said absence of said floor ahead of said user based on counting a number of said plurality of three-dimensional points in each one of said plurality of collision frusta; and

one or more controllers activating one or more vibro-tactile devices, in a particular way, in response to said determining by said processing unit;

wherein said plurality of collision frusta comprises;

a right collision frustum, a center collision frustum, and a left collision frustum, each representing a right, a center, and a left portion of said space in front of said user, respectively; and

wherein said one or more vibro-tactile devices, comprise a right, a center, and a left vibro-tactile devices, and wherein said activating one or more vibro-tactile devices comprises;

activating a right vibro-tactile device in response to said determining in said right collision frustum, activating a center vibro-tactile device in response to said determining in said center collision frustum, and activating a left vibro-tactile device in response to said determining in said left collision frustum.