Method, tool, and device for determining the coordinates of points on a surface by means of an accelerometer and a camera

• US 9,041,796 B2
• Filed: 08/01/2010
• Issued: 05/26/2015
• Est. Priority Date: 08/01/2010
• Status: Active Grant

• Current Assignee: Sensopia, Inc.
• Original Assignee: Sensopia, Inc.
• Inventors: Malka, Francis Ruben, Desforges, Gilles Jean
• Primary Examiner(s): Czekaj, Dave
• Assistant Examiner(s): BRUMFIELD, SHANIKA M
• Application Number:
  US12/848,204
• Publication Number:
  US 20120026322A1
• Time in Patent Office: 1,759 Days
• Field of Search: 348/113, 348/135, 348/139, 348/142, 348/152, 348/187, 348/207.1, 348/207.99, 356/3.1, 356/141.5, 356/147, 356/601, 356/614, 382/106
• US Class Current: 348/142
• CPC Class Codes: G01C 11/02 Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures

Abstract

A method, tool, and device for determining the coordinates or a plurality of points on a substantially plane surface by means of an accelerometer and a camera. The coordinates of each point are computed by determining a perpendicular projection of the camera onto the surface, identifying each point, measuring an inclination (the angle between the line of sight of the camera and gravity) using the accelerometer, and determining an azimuth (the angle of the line of sight of the camera around gravity).

Claims

1. A method for determining coordinates of a plurality of points P_i on a substantially plane surface S by means of a device comprising an accelerometer, a camera, a screen, and a processor, the method comprising:

actuating the camera of the device;

displaying a video stream obtained by the camera on the screen of the device;

for each point P_i;

displaying by the processor a drawing representing a boundary of the surface S in augmented reality over the video stream on the screen of the device;

identifying by a user of the device or by the processor of the device the point P_i on the video stream using the augmented reality drawing displayed on the screen of the device;

determining by the processor of the device a perpendicular projection C_i of a location of the camera of the device onto the surface S, C_i being in a different location than P_i;

reading by the processor of the device an inclination value I_i from the accelerometer of the device;

computing by the processor of the device a distance D_i between P_i and C_i using the inclination value I_i;

determining by the processor of the device an azimuth A_i;

computing the coordinates of the point P_i using A_i, D_i, and C_i and using the coordinates to generate a map, such as a floor plan, by means of the processor of the device.

2. The method of claim 1, wherein the azimuth A_i is determined by at least one of the following:

analyzing an image captured by the camera while aiming at P_i;

receiving a user input from an input unit;

receiving a value from a gyroscope;

receiving a value from a compass;

using a known angle formed by P<sub>i−

2</sub>, P<sub>i−

1</sub>, and P_i;

or

using predetermined values.

3. The method of claim 1, wherein the perpendicular projection C_i is determined by at least one of the following:

analyzing an image captured by the camera while aiming at P_i;

receiving a user input from an input unit;

receiving a value from a gyroscope;

receiving a value from the accelerometer;

placing the perpendicular projection C_i on a known path;

rotating the camera around a center of rotation that is substantially the same for all points P_i;

using a biomechanical model of a user holding the device;

or

using predetermined values.

4. The method of claim 1, further comprising a calibration phase determining at least one calibration parameter for at least one of the following:

correcting the video stream received from the camera by taking into account its optical characteristics;

determining at least one biometric parameter of a user;

or

adjusting the inclination value I_i received from the accelerometer to an environment in which the device is being used.

5. A device for determining coordinates of a plurality of points P_i on a substantially plane surface S, the device comprising:

a camera for capturing a video stream;

a screen for displaying the video stream;

an accelerometer for measuring an inclination value I_i;

a processor;

a point determination unit performing for each point P_i;

displaying by the processor a drawing representing a boundary of the surface S in augmented reality over the video stream on the screen of the device;

identifying by a user of the device or by the processor the point P_i on the video stream using the augmented reality drawing displayed on the screen;

reading by the processor an inclination value I_i from the accelerometer;

determining by the processor an azimuth value A_i;

a coordinates computation unit performing for each point P_i;

determining a perpendicular projection C_i of a location of the camera onto the surface S, C_i being in a different location than P_i;

computing by the processor a distance D_i between P_i and C_i using the received inclination value I_i;

computing the coordinates of P_i using A_i, D_i, and C_i and using the coordinates to generate a map, such as a floor plan.

6. The device of claim 5, further comprising:

a gyroscope for determining the azimuth value A_i for each point P_i.

7. The device of claim 5, further comprising:

a compass for determining the azimuth value A_i for each point P_i.

8. The device of claim 5, wherein the azimuth value A is determined by at least one of the following:

analyzing an image captured by the camera while aiming at P_i;

capturing a user input by means of an input unit;

using a known angle formed by P<sub>i−

2</sub>, P<sub>i−

1</sub>, and P_i;

or

using predetermined values.

9. The device of claim 5, wherein the coordinates computation unit determines the perpendicular projection C_i by at least one of the following:

analyzing an image captured by the camera while aiming at P_i;

capturing a user input by means of an input unit;

using a gyroscope;

using the accelerometer;

placing the perpendicular projection C_i on a known path;

placing the perpendicular projection C_i such as the lines of sight of the camera for all points P_i intersect at a single point;

using a biomechanical model of a user holding the device;

or

using predetermined values.

10. The device of claim 5, wherein the point determination unit further performs image analysis on an image for each point P_i.

11. The device of claim 5, further comprising a calibration unit for determining at least one calibration parameter for at least one of the following:

correcting the video stream captured by the camera by taking into account its optical characteristics;

determining at least one biometric parameter of a user;

or

adjusting the values read from the accelerometer to an environment in which the device is being used.

12. A computer-implementable tool for determining coordinates of a plurality of points P_i on a substantially plane surface S, the tool comprising instructions that can be downloaded electronically or wirelessly to and executed by a device comprising an accelerometer, a screen, a processor and a camera, the tool comprising:

a point determination unit performing for each point P_i;

displaying a video stream obtained by the camera on the screen of the device;

displaying by the processor a drawing representing a boundary of the surface S in augmented reality over the video stream on the screen of the device;

identifying by a user of the device or by the processor of the device the point P_i on the video stream using the augmented reality drawing displayed on the screen of the device;

receiving an inclination value from the accelerometer;

determining by the processor an azimuth value A_i;

a coordinates computation unit for determining coordinates of each point P_i by;

determining a perpendicular projection C_i of a location of the camera onto the surface S, C_i being in a different location than P_i;

computing by the processor a distance D_i between P_i and C_i on the surface S; and

computing the coordinates of P_i using A_i;

D_i, and C_i and using the coordinates to generate a map, such as a floor plan.

13. The tool of claim 12, wherein the point determination unit determines the azimuth value A_i by at least one of the following:

analyzing an image captured by the camera while aiming at P_i;

displaying the video stream on a screen;

receiving a user input from an input unit;

receiving a value from a gyroscope;

receiving a value from a compass;

rotating the camera around a center of rotation that is substantially the same for all points P_i;

using a known angle formed by P<sub>i−

2</sub>, P<sub>i−

1</sub>, and P_i;

or

using predetermined values.

14. The tool of claim 12, wherein the coordinates computation unit determines the perpendicular projection C_i by at least one of the following:

analyzing an image captured by the camera while aiming at P_i;

receiving a user input from an input unit;

receiving a value from a gyroscope;

receiving a value from the accelerometer;

placing the perpendicular projection C_i on a known path;

placing the perpendicular projection C_i such as the lines of sight of the camera for all points P_i substantially intersect at a single point;

using a biomechanical model of the user holding the device;

or

using predetermined values.

15. The tool of claim 12, further comprising a calibration unit for determining at least one calibration parameter for at least one of the following:

correcting the video stream received from the camera by taking into account its optical characteristics;

determining at least one biometric parameter of a user;

or

adjusting a plurality of values read from the accelerometer to an environment in which the device is being used.