Localized map generation

• US 9,964,409 B1
• Filed: 05/27/2015
• Issued: 05/08/2018
• Est. Priority Date: 05/27/2014
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Flint, Alex, Naroditsky, Oleg, Grygorenko, Andriy, Bergig, Oriel
• Primary Examiner(s): Smith, Jelani A
• Assistant Examiner(s): Martinez Borrero, Luis A
• Application Number:
  US14/722,911
• Time in Patent Office: 1,077 Days
• Field of Search: 701523, 701409, 701 1, 701 25, 701 28, 701300, 701461, 701466, 701469, 701400, 701491
• CPC Class Codes: G01C 21/1656 with passive imaging devices, e.g. cameras
• CPC Class Codes: G01C 21/206 specially adapted for indoor navigation
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

A method of creating a local map includes: receiving, at a mobile electronic data processing apparatus, a request from a server to generate a map of a specified destination; sending to the server a message accepting the request to generate the map responsive to receiving, at a user input of the mobile electronic data processing device, a user command indicating acceptance of the request; generating, using a processor, information related to construction of the map; an transmitting, from the mobile electronic data processing apparatus, the information related to construction of the map.

Claims

1. A method of mapping an area, the method comprising:

receiving, at a mobile device, a request to generate mapping data;

receiving image data representing a first image and a second image, the first image captured by an image detector of the mobile device at a first location and the second image captured by the image detector at a second location;

receiving inertial data captured by an inertial measurement unit of the mobile device and associated with movement of the mobile device from the first location to the second location;

in response to determining that the mobile device is located at the second location for a period of time that satisfies a threshold, generating, by one or more processors, boundary data indicating a boundary is located at the second location; and

initiating transmission of the mapping data, the mapping data based on the image data, the inertial data, and the boundary data.

2. The method of claim 1, wherein the request identifies a destination, and wherein the mapping data is generated in response to determining that the mobile device is within a predefined distance of the destination.

3. The method of claim 1, further comprising initiating transmission of information identifying a location of the mobile device to a server, wherein the request to generate mapping data is received from the server subsequent to the transmission of the information.

4. The method of claim 1, wherein the request identifies a destination, the method further comprising:

receiving, from a user interface of the mobile device, a confirmation that the mobile device is substantially positioned at the destination; and

generating the mapping data in response to receiving the confirmation.

5. The method of claim 1, wherein the mapping data represents a map determined based on a trajectory and the boundary data, the trajectory determined based on the image data and the inertial data.

6. The method of claim 5, further comprising:

receiving second image data; and

generating a refined trajectory based on the trajectory and the second image data, wherein the map is created based on the refined trajectory.

7. The method of claim 6, wherein generating the refined trajectory includes determining, based on the second image data, that the mobile device has visited at least one position along the trajectory more than once.

8. The method of claim 6, further comprising:

presenting via a user interface of the mobile device a second message requesting capture of a set of images corresponding to the second image data, wherein the second image data is received subsequent to presenting the second message.

9. The method of claim 5, further comprising:

receiving, from a user interface of the mobile device, additional boundary data, wherein creation of the map is further based on the additional boundary data.

10. The method of claim 9, wherein receiving the additional boundary data comprises receiving a depiction of a floor plan.

11. The method of claim 10, wherein receiving the depiction comprises detecting, at the user interface, a user drawing the floor plan.

12. The method of claim 1, wherein the boundary comprises a non-traversable boundary.

13. The method of claim 1, wherein the boundary data indicates a location and orientation of an edge.

14. The method of claim 13, wherein the edge corresponds to a wall.

15. The method of claim 13, wherein the edge corresponds to a side of a fixture.

16. The method of claim 1, further comprising presenting, via a user interface of the mobile device, a message instructing a user to place the mobile device at the boundary.

17. The method of claim 5, further comprising receiving, from a user interface of the mobile device, an annotation associated with a point along the trajectory, wherein the annotation comprises a description of an object located at a position corresponding to the point.

18. The method of claim 5, further comprising presenting, via a user interface of the mobile device, a message instructing a user to move the mobile device through an area, the trajectory associated with a path through the area taken by the user.

19. The method of claim 5, wherein determining the trajectory comprises:

identifying a feature depicted in the first image and the second image; and

determining a position and orientation of the mobile device, relative to an initial position and an initial orientation, based on the feature and the inertial data.

20. The method of claim 1, wherein the inertial data identifies an acceleration of the mobile device and an angular velocity of the mobile device.

21. The method of claim 1, further comprising receiving signal data during the movement of the mobile device, the signal data indicating one or more strength values for one or more electromagnetic signals, wherein the mapping data is further based on the signal data.

22. The method of claim 1, further comprising outputting a map to a display, the map generated based on the mapping data.

23. The method of claim 22, further comprising overlaying on the map an indication of signal strength for an electromagnetic signal detected during the movement of the mobile device.

24. The method of claim 23, wherein the electromagnetic signal includes a low-energy Bluetooth signal or a Wi-Fi signal.

25. A method of obtaining a local map, the method comprising:

sending, from a server device to a first mobile device, a first request for mapping data associated with a specified destination;

sending, from the server device to a second mobile device, a second request for the mapping data associated with the specified destination;

receiving, from the first mobile device, a message accepting the first request;

in response to the message accepting the first request, sending a retraction message to the second mobile device, the retraction message indicating retraction of the second request; and

subsequent to receiving the message accepting the first request, receiving the mapping data from the first mobile device, wherein the mapping data comprises;

image data and inertial data,

trajectory data indicating a trajectory based on the image data and the inertial data, and

boundary data; and

generating a map using the image data, the trajectory, and the boundary data.

26. The method of claim 25, comprising:

receiving a first indication of a first location of the first mobile device; and

receiving a second indication of a second location of the second mobile device, wherein the first request is sent responsive to determining, based on the first location, that the first mobile device is within a predefined distance of the specified destination, and wherein the second request is sent responsive to determining, based on the second location, that the second mobile device is within the predefined distance of the specified destination.

27. The method of claim 25, comprising:

receiving, at the server device, a notification from the first mobile device, the notification indicating a user interest in generating the map, wherein the first request is sent to the first mobile device based on the notification.

28. The method of claim 25, wherein the boundary data identifies a position at which the first mobile device stopped moving.

29. The method of claim 25, further comprising generating, at the server device, refined trajectory data based on the trajectory data and the image data, wherein the map is generated based on the refined trajectory data.

30. The method of claim 25, wherein the boundary data indicates an outline of a fixture, locations and orientations of walls defining an outline of a floor plan, or a combination thereof.

31. The method of claim 25, further comprising sending a boundary request from the server device to the first mobile device, the boundary request configured to cause the first mobile device to prompt a user to place the first mobile device at a boundary.

32. The method of claim 25, wherein the mapping data further indicates a signal strength of an electromagnetic source recorded during movement of the first mobile device, wherein the map is generated based further on the signal strength.

33. The method of claim 27, further comprising initiating a payment to an account associated with a user of the first mobile device in response to receiving the mapping data.

34. The method of claim 25, further comprising receiving, at the server device, a request for the map of the specified destination, wherein the first request and the second request are sent in response to receiving the request for the map.

35. A system comprising:

a server device, comprising;

one or more processors; and

a memory coupled to the one or more processors and comprising instructions executable by the one or more processors that cause the server device to;

send to a first mobile device a first request for mapping data associated with a specified destinations;

send to a second mobile device, a second request for the mapping data associated with the specified destination;

receive, from the first mobile device, a message accepting the first request;

in response to the message accepting the first request, send a retraction message to the second mobile device, the retraction message indicating retraction of the second request; and

receive the mapping data from the first mobile device, wherein the mapping data includes;

image data representing a first image and a second image, the first image captured at a first location at the specified destination and the second image captured at a second location at the specified destination;

inertial data captured during movement of the first mobile device from the first location to the second location;

trajectory data based on the image data and the inertial data; and

boundary data; and

generate a map using the image data, the trajectory data, and the boundary data.

36. The system of claim 35, wherein the instructions further cause the server device to determine to send the first request for the mapping data and the second request for the mapping data in response to a determination that the first mobile device and the second mobile device are within a predefined distance of the specified destination.

37. The system of claim 35, wherein the instructions further cause the server device to:

receive, from the first mobile device a notification indicating a user interest in generating the mapping data; and

send the first request for the mapping data to the first mobile device based on the notification.