Retracing steps

• US 10,408,623 B2
• Filed: 06/12/2009
• Issued: 09/10/2019
• Est. Priority Date: 06/12/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Brush, Alice Jane, Scott, James W., Hunt, Galen C., Sarin, Raman K, Jacobs, Andrew W, Bond, Barry C., Murillo, Oscar E, Karlson, Amy
• Primary Examiner(s): Chace, Christian
• Assistant Examiner(s): Kim, Kyung J
• Application Number:
  US12/484,089
• Publication Number:
  US 20100318293A1
• Time in Patent Office: 3,742 Days
• Field of Search: 701400, 701408-409, 701431, 701433-434, 701445-446, 701465, 701467-469, 701491, 701500-501, 701516, 701526-527, 701541, 3405391, 34053911, 34053913
• CPC Class Codes: G01C 21/1654 with electromagnetic compass
• CPC Class Codes: G01C 21/1656 with passive imaging devices, e.g. cameras
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence

Abstract

Techniques for creating breadcrumbs for a trail of activity are described. The trail of activity may be created by recording movement information based on inferred actions of walking, not walking, or changing floor levels. The movement information may be recorded with an accelerometer and a pressure sensor. A representation of a list of breadcrumbs may be visually displayed on a user interface of a mobile device, in a reverse order to retrace steps. In some implementations, a compass may additionally or alternatively be used to collect directional information relative to the earth'"'"'s magnetic poles.

Claims

1. A method implemented at least partially by a mobile device, the method comprising:

responsive to determining that a signal from a global positioning system (GPS) associated with the mobile device is unavailable, collecting, by the mobile device, location data representative of movement information from an accelerometer and a pressure sensor;

determining, based at least in part on the location data, that at least one condition associated with generating breadcrumbs for a trail has been satisfied;

collecting ambient sensor data representative of ambient information from one or more wireless sources to create an ambient signature for individual breadcrumbs of the trail;

determining, based at least in part on the location data, that a user of the mobile device is walking and a number of steps;

determining that a change in barometric pressure has occurred;

determining that the user is changing floor levels based on the change in barometric pressure occurring at a predetermined rate during a time interval;

analyzing the movement information to designate whether to add to an existing breadcrumb of the trail or to create a new breadcrumb for the trail;

recording the movement information based on collected sensor data;

categorizing the breadcrumbs into logical groups that define the trail based on the recorded movement information including details on how each breadcrumb was navigated, wherein the logical groups correspond to different activity types;

comparing sensor data for ambient information of a current location of the mobile device with the ambient signatures to identify whether the current location of the mobile device is at a specific breadcrumb of the breadcrumbs;

visually displaying, on a display of the mobile device, (i) the movement information including details on how at least one breadcrumb in the logical groups was navigated and (ii) a representation that the mobile device is near the specific breadcrumb based at least on the compared ambient signatures, wherein the details on how at least one breadcrumb in the logical groups was navigated includes an identification of an activity type corresponding to the logical group of the at least one breadcrumb; and

sharing the trail, including the breadcrumbs categorized into the logical groups that define the trail, with another mobile device.

2. The method of claim 1, wherein recording the movement information comprises at least one of (i) activating a start function in response to detecting a manual action of the user on a user interface of the mobile device, and (ii) automatically recording in response to (a) determining the mobile device is no longer moving at a travelling vehicle speed based on an absence of engine noise, (b) detecting the user is walking versus riding in a vehicle, or (c) detecting a wireless personal area network device signal is no longer in range of the mobile device.

3. The method of claim 1, further comprising discontinuing recording the movement information automatically (i) when the mobile device has been stationary for a period of time, (ii) when a change in temperature reading is detected for a predetermined time interval, (iii) when a GPS signal is detected, (iv) when a wireless signal is detected, (v) when a signal from a Wi-Fi access point is detected, or (vi) when a broadcast radio signal is detected.

4. The method of claim 1, wherein sharing the trail with another mobile device includes:

saving the trail as a file in a memory of the mobile device;

receiving a selection of a saved file; and

attaching or embedding the saved file in a communication to transmit to the other mobile device.

5. The method of claim 1, wherein recording the movement information further comprises augmenting the movement information for the trail with data from at least one of:

a compass, a magnetometer, a camera, a wireless local area network module, audio data, a GPS module, a cellular radio module, an amplitude modulated (AM) radio frequency, or a frequency modulated (FM) radio module.

6. The method of claim 1, further comprising:

recording directional information using a compass for the trail;

associating the movement information with the directional information for the trail;

grouping the movement information and the directional information recorded for breadcrumbs into logical groups that define the trail;

audibly or visually presenting the trail in a reverse order; and

audibly or visually presenting a number of steps to walk towards a direction or a number of floor levels.

7. The method of claim 1, further comprising computing the ambient information based on the one or more wireless sources, the one or more wireless sources comprising at least one of a strength of one or more local area wireless access points, a GPS signal, one or more stationary devices broadcasting radio communications signals, or electromagnetic radiation.

8. One or more non-transitory computer-readable media encoded with instructions that, when executed by a processor of a mobile device, cause the processor to perform acts comprising:

responsive to determining that a signal from a global positioning system (GPS) associated with the mobile device is unavailable, obtaining, from sensors associated with the mobile device, sensor data associated with movement of the mobile device;

determining, based at least in part on the sensor data, that a user of the mobile device is walking;

generating, in response to determining the user is walking, a first breadcrumb for a trail of activity;

assigning the first breadcrumb to a first logical group of a plurality of logical groups corresponding to different activity types, the first logical group corresponding to a walking activity type;

determining, based at least in part on the sensor data, a number of steps of the user;

associating the number of steps of the user with the first breadcrumb;

determining, based at least in part on the sensor data, that the user is changing floor levels;

generating, in response to determining the user is changing floors, a second breadcrumb for the trail of activity;

assigning the second breadcrumb to a second logical group of the plurality of logical groups, the second logical group corresponding to an elevator activity type;

presenting, on a display of the mobile device, representations of the first and second breadcrumbs both (i) individually and (ii) sequentially in a reverse chronological order starting with a breadcrumb associated with most recently obtained sensor data, wherein the representation of the first breadcrumb identifies the first logical group assigned to the first breadcrumb and the representation of the second breadcrumb identifies the second logical group assigned to the second breadcrumb, and

sharing the trail, including the first and second breadcrumbs, with another mobile device.

9. The one or more non-transitory computer-readable media of claim 8, wherein the instructions, when executed by the processor of the mobile device, cause the processor to perform acts further comprising:

presenting a representation of the breadcrumbs of the trail of activity on a user interface, wherein each of the breadcrumbs in the representation of the breadcrumbs is selectable via the user interface to generate on the user interface corresponding sensor data obtained for the respective breadcrumb.

10. The one or more non-transitory computer-readable media of claim 8, wherein the instructions, when executed by the processor of the mobile device, cause the processor to perform acts further comprising:

presenting a representation of a breadcrumb selected from the trail of activity on a user interface;

responsive to determining that a signal from the GPS associated with the mobile device is available, presenting a representation of GPS coordinates on a map corresponding to the trail of activity on the user interface; and

presenting a representation of an image of an environment corresponding to the breadcrumb selected on the user interface.

11. The one or more non-transitory computer-readable media of claim 8, wherein the instructions, when executed by the processor of the mobile device, cause the processor to perform acts further comprising:

collecting new sensor data of current location of a mobile device with a wireless local area network module or a global positioning system (GPS) module; and

determining a probability whether the new sensor data matches the breadcrumb that has been recorded and saved for the trail of activity.

12. The one or more non-transitory computer-readable media of claim 8, wherein the instructions, when executed by the processor of the mobile device, cause the processor to perform acts further comprising:

recording directional information from a compass;

associating the directional information with the movement information;

converting the movement information and the directional information from a text description to audible content; and

audibly or visually presenting the movement information and the directional information in a reverse order of what was being recorded for the trail of activity.

13. The one or more non-transitory computer-readable media of claim 8, wherein obtaining sensor data comprises collecting sensor data without the use of a GPS signal or cellular signal.

14. One or more non-transitory computer-readable media storing instructions that, when executed by a processor of a mobile device, cause the mobile device to perform acts comprising:

responsive to determining that a signal from a global positioning system (GPS) associated with the mobile device is unavailable, collecting accelerometer data and pressure sensor data from an initial starting position to an end position of the mobile device, the data to represent breadcrumbs of a trail of activity;

collecting ambient data received from one or more wireless sources from the initial starting position to the end position of the mobile device;

storing, based on the collected ambient data, an ambient signature for each of the breadcrumbs of the trail of activity;

categorizing each of the represented breadcrumbs based on the collected accelerometer data and pressure sensor data, each representation of the breadcrumbs is categorized according to a type of activity associated with the breadcrumb, the type of activity inferred from the collected accelerometer data and pressure sensor data;

comparing, via the processor of the mobile device, ambient data of a current location of the mobile device with the stored ambient signature of a categorized breadcrumb to identify whether the current location of the mobile device is at a specific breadcrumb of the breadcrumbs;

visually presenting, on a display of the mobile device, (i) a first representation of the breadcrumbs of the trail of activity on a user interface and (ii) a second representation on the user interface that the mobile device is at a specific breadcrumb based at least on the compared ambient data of the current location and the ambient signature, wherein visually presenting the first representation of the breadcrumbs includes visually identifying the type of activity associated with each breadcrumb; and

sharing the trail, including the categorized breadcrumbs, with another mobile device.

15. The one or more non-transitory computer-readable media of claim 14, the user interface further comprising presenting a representation of a breadcrumb as:

a number of floor levels to walk up or down stairs;

a number of floor levels to ride in an elevator;

or

a vehicle.

16. The one or more non-transitory computer-readable media of claim 14, the user interface further comprising presenting an amount of time in minutes and seconds and a number of approximate steps for travelling the trail of activity from the end position to the initial starting position following the breadcrumbs.

17. The one or more non-transitory computer-readable media of claim 14, the user interface further comprising presenting a representation of a breadcrumb with directional information of north, east, south, or west combined with a number of walking steps.

18. The one or more non-transitory computer-readable media of claim 14, the user interface further comprising:

presenting a representation of a breadcrumb selected in the trail of activity;

presenting a representation of GPS coordinates on a map, the GPS coordinates corresponding to the breadcrumbs in the trail of activity when a GPS signal is available; and

presenting a representation of an image of an environment corresponding to the breadcrumb selected in the trail of activity.

19. The one or more non-transitory computer-readable media of claim 14, the user interface further comprising:

presenting a user interface usable to start recording a new breadcrumb for a new trail; and

presenting a user interface usable to select a trail from the list of trails saved.