Assisting a vision-impaired user with navigation based on a 3D captured image stream

• US 8,588,464 B2
• Filed: 01/12/2007
• Issued: 11/19/2013
• Est. Priority Date: 01/12/2007
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Albertson, Jacob C., Arnold, Kenneth C., Goldman, Steven D., Paolini, Michael A., Sessa, Anthony J.
• Primary Examiner(s): Bella, Matthew
• Assistant Examiner(s): BROOKS, JULIAN D
• Application Number:
  US11/622,696
• Publication Number:
  US 20080170118A1
• Time in Patent Office: 2,503 Days
• Field of Search: 382/103, 382/104, 382/107, 340/438, 340/435, 340/436, 340/944, 340/903, 340/961
• US Class Current: 382/103
• CPC Class Codes: A61H 2003/063 with tactile perception
• CPC Class Codes: A61H 2003/065 in the form of braille
• CPC Class Codes: A61H 3/061 with electronic detecting or guiding means
• CPC Class Codes: G01C 21/3804 Creation or updating of map data

Abstract

An object-enabled navigation system assists a vision-impaired user in navigating an environment. The system captures 3D movement of a moving object within the environment, wherein the three-dimensional movement is determined using at least one image capture device aimed at the moving object. The system predicts a mobile path of the visually-impaired user. The system determines whether the movement of the moving object will intersect with the mobile path of the vision-impaired user and informs the vision-impaired user whether the movement of the moving object will intersect the mobile path of the vision-impaired user.

Claims

1. A computer-implemented method for assisting a user with information about an environment relative to the user, comprising:

capturing a three-dimensional movement of a moving object within the environment, wherein the three-dimensional movement is determined using at least one image capture device aimed at the moving object;

predicting a physical movement mobile path of the user based on a detected direction and pace of the physical movement of the user;

determining whether the movement of the moving object will intersect with the mobile path of the user; and

informing the user whether the movement of the moving object will intersect the mobile path of the user by adjusting an intensity of a tactile feedback along at least one point of a tactile glove worn by the user to simulate the change in surface area the user would feel if actually intersecting with the moving object, without the user having to actually approach the detected moving object.

2. The computer-implemented method according to claim 1, further comprising:

capturing a three-dimensional location of a stationary object within the environment, wherein the three-dimensional location is determined using at least one image capture device aimed at the stationary object;

determining whether the three-dimensional location of the stationary object will intersect with the mobile path of the user; and

informing the user whether the stationary object will intersect the mobile path of the user.

3. The computer-implemented method according to claim 1, further comprising:

identifying a three-dimensional object properties stream using the captured three-dimensional movement of the moving object;

identifying a particular type of object representing the three-dimensional object properties stream by comparing the identified three-dimensional object properties steam with a plurality of object definitions for the environment by mapping the three-dimensional movement of the moving object to a particular gesture object definition from among a plurality of gesture object definitions, wherein the particular gesture object definition identifies a particular gesture by another user, wherein the three-dimensional movement of the moving object is the particular gesture by another user; and

in response to identifying the particular type of object comprising the particular gesture from among the plurality of object definitions, informing the user of the particular gesture detected within the environment.

4. The computer-implemented method according to claim 1, wherein informing the user whether the movement of the moving object will intersect the mobile path of the user further comprises:

capturing the three-dimensional movement of the moving object comprising a movement of another user'"'"'s hand toward the user; and

informing the user of the movement required of the user'"'"'s hand to intersect with the another user'"'"'s hand to shake the another user'"'"'s hand by adjusting outputs to a tactile glove covering the user'"'"'s hand to direct the user'"'"'s hand to intersect with the another user'"'"'s hand.

5. The method of claim 1, further comprising:

determining a percentage probability that the physical movement mobile path is accurately predicted; and

adjusting an output informing the user whether the movement of the moving object will intersect the mobile path of the user to indicate the percentage probability, wherein the output is tactile detectable feedback to at least one tactile device worn by the user, wherein the intensity of feedback output via the at least one tactile device is adjustable.

6. The method of claim 1, further comprising:

predicting that the moving object is a particular type of object from the captured three-dimensional movement from a particular object definition;

prompting the user to indicate whether the predicted particular type of object is correct; and

responsive to the user indicating the predicted particular type of object is correct, updating the object definition with at least one object property of the captured three-dimensional movement.

7. The method of claim 1, further comprising:

predicting that the physical movement mobile path of the user is a particular type of movement from a particular movement definition;

prompting the user to indicate whether the predicted particular type of movement matches the user'"'"'s actual mobile path; and

responsive to the user indicating the predicted particular type of movement matches the user'"'"'s actual mobile path, updating the movement definition with at least one object property of the predicted physical movement mobile path of the user.

8. A system for assisting a user with information about an environment relative to the user, comprising:

an object processing system, comprising at least one processor, operative to capture a three-dimensional movement of a moving object within the environment, wherein the three-dimensional movement is determined using at least one image capture device aimed at the moving object, predicting a physical movement mobile path of the user based on a detected direction and pace of the physical movement of the user, and determining whether the movement of the moving object will intersect with the mobile path of the user; and

a navigation system, comprising at least one additional processor, operative to receive from the object processing system the determination whether the movement of the moving object will intersect with the mobile path of the user and control outputs for informing the user whether the movement of the moving object will intersect the mobile path of the user by adjusting an intensity of a tactile feedback along at least one point of a tactile glove worn by the user to simulate the change in surface area the user would feel if actually intersecting with the moving object, without the user having to actually approach the detected moving object.

9. The system according to claim 8, further comprising:

said object processing system operative to capture a three-dimensional location of a stationary object within the environment, wherein the three-dimensional location is determined using at least one image capture device aimed at the stationary object; and

said object processing system operative to determine whether the three-dimensional location of the stationary object will intersect with the mobile path of the user; and

said navigation system operative to inform the user whether the stationary object will intersect the mobile path of the user.

10. The system according to claim 8, further comprising:

said object processing system operative to identify a three-dimensional object properties stream using the captured three-dimensional movement of the moving object; and

said object processing system operative to identify a particular type of object representing the three-dimensional object properties stream by comparing the identified three-dimensional object properties steam with a plurality of object definitions for the environment by mapping the three-dimensional movement of the moving object to a particular gesture object definition from among a plurality of gesture object definitions, wherein the particular gesture object definition identifies a particular gesture by another user, wherein the three-dimensional movement of the moving object is the particular gesture by another user; and

said navigation system operative, in response to receiving an identifier from said object processing system of the particular type of object comprising the particular gesture from among the plurality of object definitions, for informing the user of the particular gesture detected within the environment.

11. The system according to claim 8, further comprising:

said object processing system operative to capture the three-dimensional movement of the moving object comprising a movement of another user'"'"'s hand toward the user; and

said navigation system operative to inform the user of the movement required of the user'"'"'s hand to intersect with the another user'"'"'s hand to shake the another user'"'"'s hand by adjusting outputs to a tactile glove covering the user'"'"'s hand to direct the user'"'"'s hand to intersect with the another user'"'"'s hand.

12. A computer program product for assisting a user with information about an environment relative to the user, the computer program product comprising:

one or more computer-readable tangible storage devices;

program instructions, stored on at least one of the one or more storage devices, to capture a three-dimensional movement of a moving object within the environment, wherein the three-dimensional movement is determined using at least one image capture device aimed at the moving object;

program instructions, stored on at least one of the one or more storage devices, to predict a physical movement mobile path of the user based on a detected direction and pace of the physical movement of the user;

program instructions, stored on at least one of the one or more storage devices, to determine whether the movement of the moving object will intersect with the mobile path of the user; and

program instructions, stored on at least one of the one or more storage devices, to inform the user whether the movement of the moving object will intersect the mobile path of the user by adjusting an intensity of a tactile feedback along at least one point of a tactile glove worn by the user to simulate the change in surface area the user would feel if actually intersecting with the moving object, without the user having to actually approach the detected moving object.