Wearable system for providing walking directions

• US 10,012,505 B2
• Filed: 11/11/2016
• Issued: 07/03/2018
• Est. Priority Date: 11/11/2016
• Status: Active Grant

• Current Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Dayal, Rajiv, Chen, Tiffany L., Ching, Patrick K., Northcutt, Brandon D.
• Primary Examiner(s): Black, Thomas G
• Assistant Examiner(s): LOUIE, WAE LENNY
• Application Number:
  US15/349,935
• Publication Number:
  US 20180135986A1
• Time in Patent Office: 599 Days
• Field of Search: 701408
• CPC Class Codes: G01C 21/1654 with electromagnetic compass
• CPC Class Codes: G01C 21/1656 with passive imaging devices, e.g. cameras
• CPC Class Codes: G01C 21/265 constructional aspects of navigation devices, e.g. housings, mountings, displays G01C21/3688 takes precedence
• CPC Class Codes: G01C 21/3602 Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
• CPC Class Codes: G01C 21/3605 Destination input or retrieval
• CPC Class Codes: G01C 21/3644 Landmark guidance, e.g. using POIs or conspicuous other objects
• CPC Class Codes: G01S 19/14 specially adapted for specific applications
• CPC Class Codes: G01S 19/47 the supplementary measurement being an inertial measurement, e.g. tightly coupled inertial
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06Q 30/0639 Item locations
• CPC Class Codes: G06V 10/751 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
• CPC Class Codes: G06V 30/224 of printed characters having additional code marks or containing code marks

Abstract

A wearable computing device includes an input device, a global positioning system (GPS) sensor, an inertial measurement unit (IMU), a camera, and a memory that is designed to store a database of landmarks and corresponding locations. The device includes a mobile processor that is designed to receive first user input indicating that the user is traveling from a starting location to a destination location and second user input indicating that new landmarks are present. The processor is designed to determine locations of the new landmarks based on the current location, the inertial measurement data, and/or the detected image data. The processor is designed to update the database to include the new landmarks and corresponding locations. The processor is designed to update the database to include an ordered list of landmarks including the new landmarks in order of their appearance from the starting location to the destination location.

Claims

1. A wearable computing device for creating a database usable for navigation comprising:

an input device configured to receive input from the user;

a global positioning system (GPS) sensor configured to detect a current location of the wearable computing device;

an inertial measurement unit (IMU) configured to detect inertial measurement data of the wearable computing device;

a camera configured to detect image data corresponding to an environment of the wearable computing device;

a memory configured to store a database including a plurality of landmarks and corresponding locations; and

a mobile processor coupled to the input device, the GPS sensor, the IMU, the camera, and the memory and configured to;

receive first user input from the input device indicating that the user is traveling from a starting location to a destination location,

receive second user input from the input device as the user is traveling indicating that new landmarks are present,

determine locations of the new landmarks based on at least one of the current location of the wearable computing device, the inertial measurement data of the wearable computing device, or the detected image data corresponding to the environment of the wearable computing device,

update the database to include the new landmarks and the locations of the new landmarks, and

update the database to include an ordered list of landmarks including the new landmarks placed in order of their appearance from the starting location to the destination location such that the ordered list of landmarks corresponds to a route from the starting location to the destination location.

2. The wearable computing device of claim 1 further comprising an input/output port coupled to the mobile processor and configured to communicate with a remote server and wherein the database is stored in the remote server and is accessible by multiple computing devices.

3. The wearable computing device of claim 1 further comprising an output device coupled to the mobile processor and configured to output data, wherein the mobile processor is further configured to:

receive third user input from the input device at a later time corresponding to a request for navigation instructions from the starting location to the destination location;

determine the navigation instructions based on the ordered list of landmarks and at least one of the current location of the wearable computing device, the inertial measurement data of the wearable computing device, or the detected image data corresponding to the environment of the wearable computing device; and

cause the output device to output the navigation instructions.

4. The wearable computing device of claim 3 wherein the mobile processor is further configured to:

update the database to include the new landmarks by storing the detected image data associated with the new landmarks in the memory; and

further determine the navigation instructions by determining a location of one of the new landmarks relative to the wearable computing device by comparing the detected image data to the stored image data and by determining a heading from the one of the new landmarks to a next landmark in the ordered list of landmarks.

5. The wearable computing device of claim 1 wherein the mobile processor is further configured to:

periodically determine a location marker corresponding to a location or an image as the user is traveling from the starting location to the destination location;

update the database to include the location marker; and

update the ordered list of landmarks to include the location marker in its ordered location relative to the new landmarks from the starting location to the destination location.

7. The wearable computing device of claim 1 further comprising an output device configured to output data and an input/output port coupled to the mobile processor and configured to communicate with a remote server that stores product availability data corresponding to whether each of a plurality of products is available in a retail location and wherein the mobile processor is further configured to:

receive third user input including a desired retail location and a list of desired products;

communicate with the remote server via the input/output port to determine whether each of the desired products is available at the desired retail location; and

cause the output device to output data indicating that each of the desired products is available at the desired retail location or output data indicating which of the desired products is not available at the desired retail location.

8. The wearable computing device of claim 1 wherein each of the plurality of landmarks includes at least one of a building, a plant, a tree, a sign, a road, a sidewalk, a light, a door, or a hallway.

6. The wearable computing device of claim I wherein:

the memory is further configured to store product availability data corresponding to whether each of a plurality of products is available in a retail location;

the destination location includes one of the plurality of products;

the detected image data indicates that the one of the plurality of products is unavailable; and

the mobile processor is further configured to update the product availability data to indicate that the one of the plurality of products is unavailable.

9. A wearable computing device for providing navigation instructions to a user comprising:

an input device configured to receive input from the user;

an output device configured to output data;

a global positioning system (GPS) sensor configured to detect a current location of the wearable computing device;

an inertial measurement unit (IMU) configured to detect inertial measurement data of the wearable computing device;

a camera configured to detect image data corresponding to an environment of the wearable computing device;

a memory configured to store a database including a plurality of landmarks and corresponding locations; and

a mobile processor coupled to the input device, the output device, the GPS sensor, the IMU, the camera, and the memory and configured to;

receive first user input from the input device indicating that the user is traveling from a starting location to a destination location,

receive second user input from the input device as the user is traveling indicating that new landmarks are present,

determine locations of the new landmarks based on at least one of the current location of the wearable computing device, the inertial measurement data of the wearable computing device, or the detected image data corresponding to the environment of the wearable computing device,

update the database to include the new landmarks and the locations of the new landmarks,

update the database to include an ordered list of landmarks including the new landmarks placed in order of their appearance from the starting location to the destination location such that the ordered list of landmarks corresponds to a route from the starting location to the destination location,

receive third user input from the input device at a later time corresponding to a request for navigation instructions from the starting location to the destination location,

determine the navigation instructions based on the ordered list of landmarks, and cause the output device to output the navigation instructions.

10. The wearable computing device of claim 9 further comprising an input/output port coupled to the mobile processor and configured to communicate with a remote server and wherein the database is stored in the remote server and is accessible by multiple computing devices.

11. The wearable computing device of claim 9 wherein the mobile processor is further configured to:

further determine the navigation instructions based on at least one of the current location of the wearable computing device, the inertial measurement data of the wearable computing device, or the detected image data corresponding to the environment of the wearable computing device;

further update the database to include the new landmarks by storing the detected image data associated with the new landmarks in the memory; and

further determine the navigation instructions by determining a location of one of the new landmarks relative to the wearable computing device by comparing the detected image data to the stored image data and by determining a heading from the one of the new landmarks to a next landmark in the ordered list of landmarks.

12. The wearable computing device of claim 9 wherein the mobile processor is further configured to:

periodically determine a location marker corresponding to a location or an image as the user is traveling from the starting location to the destination location;

update the database to include the location marker; and

update the ordered list of landmarks to include the location marker in its ordered location relative to the new landmarks from the starting location to the destination location.

13. The wearable computing device of claim 9 wherein:

the memory is further configured to store product availability data corresponding to whether each of a plurality of products is available in a retail location;

the destination location includes one of the plurality of products;

the detected image data indicates that the one of the plurality of products is unavailable; and

the mobile processor is further configured to update the product availability data to indicate that the one of the plurality of products is unavailable.

14. The wearable computing device of claim 9 further comprising an input/output port coupled to the mobile processor and configured to communicate with a remote server that stores product availability data corresponding to whether each of a plurality of products is available in a retail location and wherein the mobile processor is further configured to:

receive fourth user input including a desired retail location and a list of desired products;

communicate with the remote server via the input/output port to determine whether each of the desired products is available at the desired retail location; and

cause the output device to output data indicating that each of the desired products is available at the desired retail location or output data indicating which of the desired products is not available at the desired retail location.

15. A method for creating a database usable for navigation by a wearable computing device comprising:

detecting, by a global positioning system (GPS) sensor, a current location of the wearable computing device;

detecting, by an inertial measurement unit (IMU), inertial measurement data of the wearable computing device;

detecting, by a camera, image data corresponding to an environment of the wearable computing device;

storing, in a memory, a database including a plurality of landmarks and corresponding locations;

receiving, via an input device, first user input indicating that the user is traveling from a starting location to a destination location;

receiving, via the input device, second user input indicating that new landmarks are present as the user is traveling from the starting location to the destination location;

determining, by a mobile processor, locations of the new landmarks based on at least one of the current location of the wearable computing device, the inertial measurement data of the wearable computing device, or the detected image data corresponding to the environment of the wearable computing device;

updating, by the mobile processor, the database to include the new landmarks and the locations of the new landmarks; and

updating, by the mobile processor, the database to include an ordered list of landmarks including the new landmarks placed in order of their appearance from the starting location to the destination location such that the ordered list of landmarks corresponds to a route from the starting location to the destination location.

16. The method of claim 15 further comprising accessing, via an input/output port, the database, wherein the database is positioned on a remote server and is accessible by multiple computing devices.

17. The method of claim 15 further comprising:

receiving, via the input device, third user input corresponding to a request for navigation instructions from the starting location to the destination location; and

determining, by the mobile processor, the navigation instructions based on the ordered list of landmarks and at least one of the current location of the wearable computing device, the inertial measurement data of the wearable computing device, or the detected image data corresponding to the environment of the wearable computing device; and

outputting, by an output device, the navigation instructions.

18. The method of claim 17, wherein:

updating, by the mobile processor, the database to include the new landmarks further includes storing image data associated with the new landmarks in the memory; and

determining, by the mobile processor, the navigation instructions further includes determining a location of one of the new landmarks relative to the wearable computing device by comparing the detected image data to the stored image data and by determining a heading from the one of the new landmarks to a next landmark in the ordered list of landmarks.

19. The method of claim 15 further comprising:

periodically deteiinining, by the mobile processor, a location marker corresponding to a location or an image as the user is traveling from the starting location to the destination location;

updating, by the mobile processor, the database to include the location marker; and

updating, by the mobile processor, the ordered list of landmarks to include the location marker in its ordered location relative to the new landmarks from the starting location to the destination location.

20. The method of claim 15 further comprising:

receiving, via the input device, third user input including a desired retail location and a list of desired products;

accessing, via an input/output port, a remote server that stores product availability data corresponding to whether each of a plurality of products is available in a retail location to determine whether each of the desired products is available at the desired retail location; and

outputting, by an output device, data indicating that each of the desired products is available at the desired retail location or data indicating which of the desired products is not available at the desired retail location.