Object detection and localized extremity guidance

• US 9,613,505 B2
• Filed: 03/13/2015
• Issued: 04/04/2017
• Est. Priority Date: 03/13/2015
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Inventors: Martinson, Eric, Sisbot, Emrah Akin, Djugash, Joseph, Oguchi, Kentaro, Takaoka, Yutaka, Nakano, Yusuke
• Primary Examiner(s): Nguyen, Hung T
• Application Number:
  US14/658,138
• Publication Number:
  US 20160267755A1
• Time in Patent Office: 753 Days
• Field of Search: 3404071, 3405391, 34053913, 3405731, 345156, 701412
• CPC Class Codes: G08B 6/00 Tactile signalling systems, e.g. personal calling systems

Abstract

Technology for localized guidance of a body part of a user to specific objects within a physical environment using a vibration interface is described. An example system may include a vibration interface wearable on an extremity by a user. The vibration interface includes a plurality of motors. The system includes sensor(s) coupled to the vibrotactile system and a sensing system coupled to the sensor(s) and the vibration interface. The sensing system is configured to analyze a physical environment in which the user is located for a tangible object using the sensor(s), to generate a trajectory for navigating the extremity of the user to the tangible object based on a relative position of the extremity of the user bearing the vibration interface to a position of the tangible object within the physical environment, and to guide the extremity of the user along the trajectory by vibrating the vibration interface.

Claims

1. A method comprising:

analyzing a physical environment in which a user bearing a vibration interface including a plurality of motors is located for a tangible object within a field of view using one or more sensors coupled to electronically communicate with the vibration interface;

determining a relative position of an extremity of the user bearing the vibration interface to a position of the tangible object within the physical environment;

generating a trajectory for navigating the extremity of the user to the tangible object based on the relative position of the extremity to the position of tangible object; and

guiding the extremity of the user along the trajectory by vibrating one or more of the motors of the vibration interface.

2. The method of claim 1, further comprising:

sensing movement of the extremity by the user using the one or more sensors;

responsive to sensing the movement, re-determining the relative position of the extremity of the user to the tangible object;

updating the trajectory for navigating the extremity of the user to the tangible object based on a change to the relative position of the extremity of the user to the tangible object; and

guiding the extremity of the user along the trajectory, as updated, by vibrating the one or more of the motors of the vibration interface.

3. The method of claim 1, wherein

determining the relative position of the extremity includes

determining an orientation of the extremity using sensing data captured by the one or more sensors, the sensing data reflecting movement of the extremity by the user,

calculating a ray originating from a predetermined point of the extremity and extending through a predetermined point of the tangible object, and

calculating an angular offset θ

between the orientation of the extremity and the ray, and

generating the trajectory for navigating the extremity of the user to the tangible object is based on the angular offset.

4. The method of claim 1, further comprising:

determining a vibratory pattern for vibrating the one or more of the motors of the vibration interface based on the trajectory generated for navigating the extremity of the user to the tangible object, wherein guiding the extremity of the user along the trajectory by vibrating the one or more of the motors of the vibration interface further includes vibrating the one or more of the motors according to the vibratory pattern to convey a direction for movement of the extremity to reach the tangible object.

5. The method of claim 4, wherein the vibratory pattern includes one or more of linear motion and rotational dimensions and vibrating the one or more of the motors of the vibration interface includes vibrating the one or more of the motors based on the one or more of the linear motion and rotational dimensions of the vibratory pattern to convey the direction for movement of the extremity to reach the tangible object.

6. The method of claim 5, further comprising:

identifying a bit sequence and vibration intensity value for the one or more of the motors, the bit sequence and vibration intensity value reflecting the one or more of the linear motion and rotational dimensions of the vibratory pattern, wherein vibrating the one or more of the motors of the vibration interface includes vibrating the one or more of the motors using the bit sequence and vibration intensity value.

7. The method of claim 1, further comprising:

determining that the extremity of the user has reached the position of the tangible object within the physical environment; and

terminating vibrating the one or more of the motors of the vibration interface to cease guiding the extremity of the user.

8. The method of claim 1, wherein the position of the tangible object is fixed or variable.

9. The method of claim 1, wherein determining the relative position of the extremity of the user to the position of the tangible object within the physical environment includes determining a central position of the tangible object, determining a centroid of the vibration interface, and calculating the relative position based on a distance between the central position of the tangible object and the centroid of vibration interface.

10. The method of claim 1, further comprising:

identifying an obstacle within the physical environment between the tangible object and the extremity of the user using the one or more sensors, wherein generating the trajectory for navigating the extremity of the user to the tangible object based on the relative position of the extremity to the position of tangible object is further based on a path that circumnavigates the obstacle.

11. The method of claim 1, wherein analyzing the physical environment in which the user is located for the tangible object using one or more sensors includes locating the tangible object within the physical environment using a visual perception system.

12. The method of claim 1, further comprising:

detecting an operational problem associated with the vibration interface;

determining a unique vibratory pattern for the operational problem;

vibrating one or more of the motors of the vibration interface based on the unique vibratory pattern; and

receiving input from the user via an input device providing assistance to address the operational problem.

13. The method of claim 1, further comprising:

receiving input data from the user via an input device indicating the tangible object as an object of interest.

14. A system comprising:

a vibration interface wearable on an extremity by a user, the vibration interface including a plurality of motors;

one or more sensors coupled to the vibration interface;

a sensing system coupled to the one or more sensors and the vibration interface, the sensing system being configured to analyze a physical environment in which the user is located for a tangible object within a field of view using the one or more sensors, to generate a trajectory for navigating the extremity of the user to the tangible object based on a relative position of the extremity of the user bearing the vibration interface to a position of the tangible object within the physical environment, and to guide the extremity of the user along the trajectory by vibrating the vibration interface.

15. The system of claim 14, comprising:

an input device configured to receive input data from the user indicating the tangible object, the input device being coupled to the sensing system to communicate data reflecting the tangible object to the sensing system.

16. The system of claim 14, wherein the one or more sensors are further configured to receive transponder signals from the tangible object, the transponder signals including identification data identifying the tangible object, and the sensing system being executable by the one or more processors to determine a unique identify of the tangible object based on the identification data.

17. The system of claim 14, wherein the one or more sensors include a perceptual system configured to capture image data including images of the physical environment and objects located with the physical environment, the perceptual system being coupled to the sensing system to provide the image data including the images, and the sensing system being further configured to process the image data to determine a location of the tangible object.

18. The system of claim 14, wherein the sensing system is further configured to determine the relative position of the extremity of the user bearing the vibration interface.

19. The system of claim 14, wherein the sensing system is further configured to sense movement of the extremity by the user using the one or more sensors, to re-determine the relative position of the extremity of the user to the tangible object responsive to sensing the movement, to update the trajectory for navigating the extremity of the user to the tangible object based on a change to the relative position of the extremity of the user to the tangible object, and to guide the extremity of the user along the updated trajectory, as updated, using the vibration interface.

20. The system of claim 14, wherein the sensing system is further configured to determine a vibratory pattern for vibrating one or more of the motors of the vibration interface based on the trajectory generated for navigating the extremity of the user to the tangible object and to vibrate the vibration interface by vibrating the one or more of the motors according to the vibratory pattern to convey a direction for movement of the extremity to reach the tangible object.

21. The system of claim 20, further comprising:

a motor control unit configured to vibrate the one or more of the motors of the vibration interface according to the vibratory pattern determined by the sensing system, the motor control unit being coupled to the sensing system to receive control signals.