Method, system and computer program for locating mobile devices based on imaging

• US 9,418,284 B1
• Filed: 03/30/2015
• Issued: 08/16/2016
• Est. Priority Date: 04/09/2014
• Status: Active Grant

• Current Assignee: Vortex Intellectual Property Holding LLC
• Original Assignee: Vortex Intellectual Property Holding LLC
• Inventors: Houri, Cyril
• Primary Examiner(s): Nakhjavan, Shervin
• Application Number:
  US14/673,186
• Time in Patent Office: 505 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/5862 using texture
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning
• CPC Class Codes: G06V 20/00 Scenes; Scene-specific elements control of digital cameras H04N23/60
• CPC Class Codes: H04W 4/02 Services making use of location information
• CPC Class Codes: H04W 4/029 Location-based management or tracking services

Abstract

Method for generating and using a database of pixel patterns includes obtaining images using a mobile device including a common pixel pattern and processing each image based on a size of the pixel pattern and/or angular orientation of the pixel pattern relative to an imaging direction to derive positional information about the pixel pattern. Location of the mobile device is determined by obtaining an image including a stationary pixel pattern, searching the database for the obtained pixel pattern, and retrieving positional information about a closest match pixel pattern. With the retrieved positional information, an angle between an imaging direction in which the image including the closest match pixel pattern was obtained and an imaging direction in which the image was obtained and/or a size differential between the closest match pixel pattern and the pixel pattern in the obtained image is/are analyzed to derive positional information about the mobile device'"'"'s location.

Claims

1. A method for determining position of a mobile device with at least one imaging device, comprising:

obtaining, using the at least one imaging device of the mobile device, at least one image including a stationary pattern of pixels;

searching through a database of pixel patterns for a pixel pattern that most closely matches the pattern of pixels in the obtained at least one image;

retrieving from the database as a result of the searching, positional information about one of the pixel patterns that is deemed to most closely match the pattern of pixels in the obtained at least one image; and

analyzing, using a processor of the mobile device, in combination with the retrieved positional information, relative size of or a size differential between the most closely matching pixel pattern and the pattern of pixels in the obtained at least one image to derive positional information about a position of the mobile device.

2. The method of claim 1, wherein the step of obtaining at least one image comprises obtaining a plurality of images using the at least one imaging device of the mobile device, each obtained image of the plurality of images including a different stationary pattern of pixels, and for each obtained image,

searching through the database of pixel patterns for a pixel pattern that most closely matches the pattern of pixels in that obtained image;

retrieving from the database as a result of the searching, positional information about one of the pixel patterns that is deemed to most closely match the pattern of pixels in that obtained image; and

analyzing, using the processor of the mobile device, in combination with the retrieved positional information, relative size of or size differential between the most closely matching pixel pattern and the pattern of pixels in that obtained image to derive positional information about the position of the mobile device.

3. The method of claim 1, further comprising generating the database of pixel patterns with positional information by:

obtaining, using the at least one imaging device of the mobile device, at least one image including a common pattern of pixels; and

processing, using a processor, each of the at least one obtained image based on a size of the pattern of pixels in the image and angular orientation of the pattern of pixels relative to an imaging direction in which the image was obtained by the mobile device to derive positional information about the pattern of pixels.

4. The method of claim 3, wherein the step of obtaining at least one image when generating the database comprises obtaining a plurality of images from different positions of the mobile device relative to a common, stationary pattern of pixels, each of the plurality of images including the common, stationary pattern of pixels.

5. The method of claim 4, wherein the step of obtaining a plurality of images comprises:

obtaining, using the at least one imaging device of the mobile device, the plurality of images which each include the common, stationary pattern of pixels and which is each obtained from a respective one of the plurality of different positions of the mobile device relative to the common, stationary pattern of pixels;

processing, using a processor, the plurality of obtained images based on at least one of size and angular orientation of the pattern of pixels in the obtained images to derive positional information about the pattern of pixels; and

repeating the obtaining and processing steps.

6. The method of claim 5, wherein the step of processing, using a processor, the plurality of obtained images based on at least one of size and angular orientation of the pattern of pixels in the obtained images to derive positional information about the pattern of pixels comprises performing this processing using the processor of the mobile device.

7. The method of claim 4, wherein the step of obtaining a plurality of images comprises:

positioning the mobile device in each of the different positions relative to the common, stationary pattern of pixels at different times; and

pressing an image obtaining actuator on the mobile device while the mobile device is in each of the different positions.

8. The method of claim 4, wherein the step of obtaining a plurality of images comprises:

positioning the mobile device in a first position relative to the pattern of pixels;

pressing an image obtaining actuator on the mobile device while the mobile device is in the first position;

moving the mobile device to a second position relative to the pattern of pixels and different than the first position; and

pressing the image obtaining actuator on the mobile device while the mobile device is in the second position.

9. The method of claim 3, wherein the step of processing, using a processor, each of the at least one obtained image based on a size of the pattern of pixels in the image and angular orientation of the pattern of pixels relative to an imaging direction in which the image was obtained by the mobile device to derive positional information about the pattern of pixels comprises performing this processing using the processor of the mobile device.

10. The method of claim 1, wherein the step of obtaining at least one image comprises obtaining the at least one image using a panoramic camera attached to the mobile device.

11. The method of claim 1, wherein the mobile device is a pilotless drone.

12. The method of claim 1, further comprising displaying on a screen of the mobile device, indications of the position of the mobile device and surrounding structure.

13. The method of claim 1, wherein the analyzing step comprises analyzing in combination with the retrieved positional information, an angle between an imaging direction in which the image including the most closely matching pattern of pixels was obtained by an imaging device and an imaging direction in which the at least one image was obtained by the at least one imaging device of the mobile device to derive positional information about the position of the mobile device.

14. A navigation method, comprising:

determining an initial position of a mobile device to be guided as according to the method of claim 1, and then

guiding movement of the mobile device, after the initial position of the mobile device is determined, by analyzing using the processor, movement of the mobile device relative to the initial position.

15. The method of claim 14, wherein the step of guiding movement of the mobile device comprises obtaining multiple images each including a common pattern of pixels and obtained at different times separated by movement of the mobile device and analyzing, using the processor, the multiple images to determine relative movement of the mobile device.

16. The method of claim 14, wherein the step of guiding movement of the mobile device comprises obtaining multiple images at different times separated by movement of the mobile device and analyzing, using the processor, changes in patterns in the multiple images to determine relative movement of the mobile device.

17. The method of claim 14, further comprising:

determining an accurate updated position of the mobile device to be guided, after the movement of the mobile device has been guided by analyzing movement of the mobile device relative to the initial position; and

then

using this updated position, guiding movement of the mobile device, after the updated position of the mobile device is determined, by analyzing using the processor, movement of the mobile device relative to the updated position.

18. The method of claim 14, wherein the mobile device is a pilotless drone.

19. The method of claim 1, wherein the mobile device is a first mobile device, further comprising:

generating the database of pixel patterns with positional information by;

obtaining, using at least one imaging device of a second mobile device, at least one image including a common pattern of pixels; and

processing, using a processor of the second mobile device, each of the at least one obtained image based on a size of the pattern of pixels in the image and angular orientation of the pattern of pixels relative to an imaging direction in which the image was obtained by the second mobile device to derive positional information about the pattern of pixels; and

providing the database to the first mobile device from the second mobile device.

20. The method of claim 1, further comprising displaying on a screen of the mobile device, indications of the position of the mobile device, surrounding structure and the position of the pattern of pixels in the obtained at least one image.

21. The method of claim 1, wherein the size of the pattern of pixels in the obtained at least one image relative to a size of the closest match pixel pattern is analyzed in the analyzing step to derive the positional information about the position of the mobile device.

22. The method of claim 1, wherein the size differential between the pattern of pixels in the obtained at least one image and the closest match pixel pattern is analyzed in the analyzing step to derive the positional information about the position of the mobile device.