Localization method and apparatus

• US 10,012,733 B2
• Filed: 06/07/2016
• Issued: 07/03/2018
• Est. Priority Date: 06/07/2015
• Status: Active Grant

• Current Assignee: Craig William Neely, Geoffrey Louis Barrows, Gregory Brill
• Original Assignee: Craig William Neely, Geoffrey Louis Barrows, Gregory Brill
• Inventors: Barrows, Geoffrey Louis, Brill, Gregory, Neely, Craig William
• Primary Examiner(s): Ramesh, Krishnan
• Application Number:
  US15/175,577
• Publication Number:
  US 20160357189A1
• Time in Patent Office: 756 Days
• Field of Search: None
• CPC Class Codes: G01S 17/42 Simultaneous measurement of distance and other co-ordinates indirect measurement G01S17/46
• CPC Class Codes: G01S 17/93 for anti-collision purposes
• CPC Class Codes: G01S 5/16 using electromagnetic waves other than radio waves
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: H04B 10/116 Visible light communication
• CPC Class Codes: H04B 10/50 Transmitters
• CPC Class Codes: H04B 10/60 Receivers

Abstract

An apparatus for determining the location and state of an object in an environment is disclosed. Three or more light-emitting beacons are located in the environment, and each beacon transmits a light pattern that contains information needed to identify the beacon. An optical receiver is attached to the object whose location needs to be determined. The optical receiver comprising one or more cameras images the beacons, and identifies the apparent direction from which the beacon is coming based on the pixel or pixels illuminated by the beacon. The optical receiver also decodes the light pattern transmitted by each beacon to identify it. Finally, based on knowledge of the locations of the beacons in the environment and based on the apparent directions from which the beacons appear to be coming, the optical receiver determines its location, and thus the location of the object.

Claims

1. A system for determining the location of an object in an environment comprising:

a plurality of optical transmitters, each configured to emit an optical transmission different from the optical transmission of another of the plurality of optical transmitters, wherein each optical transmission comprises a signal, and wherein each optical transmitter is positioned at a location spaced apart from another of the plurality of optical transmitters within the environment;

one or more optical receivers configured to be mounted to the object, the one or more optical receivers further configured to receive one or more of the optical transmissions and to generate a set of transmitter angle vectors, wherein each transmitter angle vector is associated with an image of one of the plurality of optical transmitters, and is based on the angle between the associated optical transmitter and a reference frame of the one or more optical receivers, and the location of the associated optical transmitter; and

a computing device communicatively coupled to the one or more optical receivers and configured to estimate the location of the object based on the set of transmitter angle vectors and the locations of the three or more optical transmitters.

2. The system of claim 1, wherein:

the signal is a substantially cyclical signal that is modulated according to a data stream associated with the optical transmitter emitting the optical transmission, and

for each optical transmission, the one or more optical receivers are capable of extracting the data stream from the signal, and determining the location of the optical transmitter emitting the optical transmission based on the data stream.

3. The system of claim 2, wherein the substantially cyclical signal has a phase that is modulated according to the data stream.

4. The system of claim 1, wherein each of the one or more optical receivers comprises a camera system.

5. The system of claim 4, wherein for each optical transmission, optics of the camera system are configured so that the majority of a light of the optical transmission is acquired by an associated window of pixels that comprises substantially fewer pixels than a total number of pixels within the camera system, wherein the associated window of pixels comprises a plurality of rows of pixels and a plurality of columns of pixels, and the one or more optical receivers generates the transmitter angle vector using the light acquired by the associated window of pixels.

6. The system of claim 5, wherein the camera system is capable of digitizing only the pixels within the window of pixels.

7. The system of claim 6, wherein the camera system achieves a higher frame rate than what is possible if the camera system acquires every pixel of the total amount of pixels within the camera system.

8. The system of claim 6, wherein for each optical transmission, the camera system acquires the window of pixels associated with the optical transmission by acquiring pixel signals from an acquisition window of pixels that overlaps the associated window of pixels, and the camera system is configured to move the acquisition window of pixels in response to a relative motion between the optical receiver and the plurality of optical transmitters to continue acquiring the associated window of pixels and to keep the associated window of pixels illuminated by light from at least one of the plurality of optical transmitters.

9. The system of claim 1, wherein:

each of the one or more optical receivers comprises a camera system, and for each optical transmission, optics of the camera system is configured so that the majority of a light of the optical transmission is focused on an associated window of pixels that comprises substantially fewer pixels than a total number of pixels within the camera system, wherein the associated window of pixels comprises a plurality of rows of pixels and a plurality of columns of pixels, and the transmitter angle vector associated with the optical transmission is generated based on a location of a brightest pixel of the window of pixels that receives more light than every other pixel of the window of pixels.

10. The system of claim 9, wherein for each optical transmission, the location of the brightest pixel of the window of pixels associated with the optical transmission is determined to a sub-pixel precision using a light intensity of the brightest pixel and light intensities of the pixels adjacent to the brightest pixel in the window of pixels.

11. The system of claim 1, wherein each transmitter angle vector is associated with an image of the associated optical transmitter, and is computed based on a subpixel precision estimate of the location of the image of the associated optical transmitter.

12. The system of claim 1, wherein a span angle of the plurality of optical transmitters is greater than 90 degrees when measured from the location of the one or more optical receivers.

13. The system of claim 1, wherein a span angle of the optical transmitters is greater than 120 degrees.

14. The system of claim 1, wherein a field of view of the one or more optical receivers is greater than 90 degrees.

15. The system of claim 1, wherein a field of view of the one or more optical receivers is greater than 180 degrees.

16. The system of claim 1, wherein a field of view of the one or more optical receivers is greater than 270 degrees.

17. The system of claim 1, wherein a field of view of the one or more optical receivers is substantially omnidirectional.

18. The system of claim 1, wherein:

the one or more optical receivers comprises a plurality of image sensor chips and a processor, wherein each image sensor chip of the plurality of image sensor chips is separate from every other image sensor chip of the plurality of image sensor chips;

the processor is configured to operate the plurality of image sensor chips and receive image data from the plurality of image sensor chips; and

each image sensor chip is configured to monitor a field of view having at least a portion that is different from the other image sensor chips of the plurality of image sensor chips.

19. The system of claim 1, wherein:

the one or more optical receivers comprises a plurality of cameras;

each camera comprises a camera processor and an image sensor, and is configured to monitor a field of view having at least a portion that is different from the other camera of the plurality of cameras; and

each camera processor is configured to operate the image sensor of the camera, and is configured to send image data to the computing device.

20. A receiver system attached to an object for determining a location of the object in an environment comprising:

one or more optical receivers configured to receive optical data from two or more optical transmitters and to generate a set of transmitter angle vectors; and

a computing device communicatively coupled to the one or more optical receivers and configured to estimate the location of the object based on the set of transmitter angle vectors and locations of the two or more optical transmitters;

wherein

each transmitter angle vector is associated with an image of one of the two or more optical transmitters, and is based on the angle between the associated optical transmitter and a reference frame of the one or more optical receivers and the location of the associated optical transmitter.

21. The receiver system of claim 20, wherein:

each transmitter angle vector is associated with a pixel location in the optical receiver and a set of pixels surrounding the pixel location, the one or more optical receivers comprises a total set of pixels, and is configured to define a first set of pixels to acquire and a second set of pixels to not acquire, wherein the first set of pixels and the second set of pixels are subsets of the total set of pixels, the first set of pixels comprises a two dimensional array of pixels, and

the one or more optical receivers is configured to acquire the first set of pixels without acquiring the second set of pixels.

22. The receiver system of claim 21, wherein the one or more optical receivers is configured to acquire the first set of pixels at a rate greater than 1 second/(the total set of pixels×

T_pixel), where T_pixel is a time required for the one or more optical receivers to acquire a single pixel.

23. The receiver system of claim 21, wherein the one or more optical receivers is configured to acquire the first set of pixels at a rate greater than 10×

(1 second/(the total set of pixels×

T_pixel)), where T_pixel is a time required for the one or more optical receivers to acquire a single pixel.

24. The receiver system of claim 21, wherein the one or more optical receivers defines the first set of pixels and the second set of pixels based on the pixel locations associated with the set of transmitter angle vectors.

25. The receiver system of claim 20, wherein each transmitter angle vector is associated with a pixel location in the optical receiver and a set of pixels surrounding the pixel location, and the pixel location is determined to an accuracy that is less than a pitch between adjacent pixels of the set of pixels.

26. The receiver system of claim 20, wherein the one or more optical receivers are configured to decode an identifier associated with each optical transmitter generating the optical transmissions, and to determine the location of the one or more optical receivers based on a look-up table and the identifier associated with each optical transmitter.

27. The receiver system of claim 20, wherein the receiver system is configured to determine the identities of the two or more optical transmitters based on the optical transmissions, and is capable of optically tracking the two or more optical transmitters based on the optical transmissions independent of determining the identities of the two or more optical transmitters.

28. The receiver system of claim 20, wherein the receiver system is configured to receive the locations of the two or more optical transmitters from a wireless transmitter in the environment.

29. The receiver system of claim 20, wherein:

the object is a mobile robot;

the receiver system is configured to send location information to the mobile robot; and

the mobile robot is configured to control its motion based on the location information.

30. The receiver system of claim 29, wherein the receiver system or the mobile robot further comprises object detection devices capable of detecting objects in the environment that are nearby the mobile robot and generating nearby object information, and the mobile robot is configured to control is motion based additionally on the nearby object information.

31. A method for determining the location of an object in an environment comprising the steps of:

receiving a plurality of optical transmissions from a plurality of optical transmitters with an optical receiver comprising one or more image sensor chips, wherein each image sensor chip of the one or more image sensor chips is associated with an optical device capable of forming an image of part of the environment on the image sensor chip;

extracting an associated signal from each optical transmission;

generating a plurality of image locations based on the plurality of optical transmissions, wherein each image location is generated based on an optical transmission and a location of the associated optical transmission on the one or more image sensor chips, and based on the geometries of the pixel arrays of the one or more image sensor chips;

generating a plurality of transmitter angle vectors based on the plurality of image locations and based on the associated signal of each optical transmission; and

determining the location of the object based on the plurality of transmitter angle vectors and a location of each optical transmitter.

32. The method of claim 31, wherein each image location is additionally generated based on light intensities of a plurality of pixels associated with the image location, and is substantially accurate to a precision that is smaller than the pitch of adjacent pixels of the plurality of pixels.

33. The method of claim 31, further comprising:

selecting a first set of pixels to acquire, wherein the first set of pixels comprises a plurality of rows of pixels and a plurality of columns of pixels;

selecting a second set of pixels to not acquire; and

acquiring the first set of pixels without acquiring the second set of pixels, wherein the plurality of optical transmissions are received based on the first set of pixels.

34. The method of claim 33, wherein the plurality of optical transmissions is received at a rate greater than 1 second/(a total number of pixels of the one or more image sensor chips×

T_pixel), where T_pixel is a time required by the optical receiver to acquire a single pixel from the one or more image sensor chips.

35. A receiver system attached to an object for determining a location of the object in an environment comprising:

one or more optical receivers configured to receive optical transmissions from two or more optical transmitters and to generate a set of transmitter angle vectors, wherein the one or more optical receivers comprises one or more image sensors and one or more optical devices arranged according to a geometric configuration, the one or more image sensors are associated with one or more pixel geometries, and each transmitter angle vector of the set of transmitter angle vectors is generated based on the geometric configuration and the one or more pixel geometries; and

a computing device communicatively coupled to the one or more optical receivers and configured to estimate the location of the object based on the set of transmitter angle vectors and locations of the one or more optical transmitters;

wherein each transmitter angle vector is associated with one of the two or more optical transmitters, and is based on the angle between the associated optical transmitter and a reference frame of the one or more optical receivers and the location of the associated optical transmitter, and

each transmitter angle vector is associated with a pixel location in the one or more optical receivers, and the pixel location is determined to an accuracy that is less than the minimum distance between any two adjacent pixels of the one or more optical receivers.

36. A receiver system attached to an object for determining a location of the object in an environment comprising:

one or more optical receivers configured to receive optical transmissions from two or more optical transmitters and to generate a set of transmitter angle vectors; and

a computing device communicatively coupled to the one or more optical receivers and configured to estimate the location of the object based on the set of transmitter angle vectors and locations of the one or more optical transmitters;

wherein each transmitter angle vector is associated with one of the two or more optical transmitters, and is based on the angle between the associated optical transmitter and a reference frame of the one or more optical receivers and the location of the associated optical transmitter; and

the one or more optical receivers is configured to define a first set of pixels to acquire and a second set of pixels to not acquire, wherein the first set of pixels comprises a two dimensional array of pixels, acquire the first set of pixels without acquiring the second set of pixels, and generate the set of transmitter angle vectors based on the first set of pixels.

37. The receiver system of claim 36, wherein the one or more optical receivers is configured to receive the optical transmissions at a rate greater than the 1 second/(a total number of pixels of the one or more optical receivers×

T_pixel), where T_pixel is a time required by the one or more optical receivers to acquire a single pixel.