METHOD OF AND ARRANGEMENT FOR MAPPING RANGE SENSOR DATA ON IMAGE SENSOR DATA

• US 20100296705A1
• Filed: 11/07/2007
• Published: 11/25/2010
• Est. Priority Date: 11/07/2007
• Status: Active Grant

• Current Assignee: Tomtom Global Content B.V (TomTom NV)
• Original Assignee: Tomtom Global Content B.V (TomTom NV)
• Inventors: Gliszczynski, Rafa Jan, Miksa, Krysztof, Taborowski, Lukasz Piotr
• Granted Patent: US 9,880,010 B2
• Time in Patent Office: Days
• US Class Current: 382/106
• CPC Class Codes: G01C 11/02 Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
• CPC Class Codes: G01C 21/28 with correlation of data from several navigational instruments
• CPC Class Codes: G01C 21/3848 Data obtained from both position sensors and additional sensors
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal

Abstract

A method of and arrangement for mapping first range sensor data from a first range sensor to image data from a camera are disclosed. In at least one embodiment, the method includes: receiving time and position data from a position determination device on board a mobile system, as well as the first range sensor data from the first range sensor on board the mobile system and the image data from the camera on board the mobile system; identifying a first points cloud within the first range sensor data, relating to at least one object; producing a mask relating to the object based on the first points cloud; mapping the mask on object image data relating to the same object as present in the image data from the at least one camera; and performing a predetermined image processing technique on at least a portion of the object image data.

Claims

1. Computer arrangement, comprising:

a processor; and

at least one memory, connected to the processor, the at least one memory comprising a computer program comprising data and instructions arranged to allow said processor to;

receive time and position data from a position determination device on board a mobile system, and receive first range sensor data from at least a first range sensor on board said mobile system and image data from at least one camera on board said mobile system;

identify a first points cloud within said first range sensor data, relating to at least one object;

produce a mask relating to said object and based on said first points cloud;

map said mask on object image data relating to said same object as present in said image data from said at least one camera; and

perform an image processing technique on at least a portion of said object image data.

2. Computer arrangement according to claim 1, wherein said computer program is arranged to allow said processor to:

receive second range sensor data from a second range sensor on board said mobile system;

identify a second points cloud within said second range sensor data, relating to said same at least one object;

calculate a motion vector for said at least one object from said first and second points clouds; and

map said mask on said object image data while using said motion vector.

3. Computer arrangement according to claim 2, wherein said computer program is arranged to allow said processor to:

calculate a real size of said object based on said first and second range sensor data; and

use said real length while producing said mask.

4. Computer arrangement according to claim 3, wherein said computer program is arranged to allow said processor to:

calculate an observed size of said object based on one of said first and second range sensor data; and

calculate said mask based on said observed length and said real length.

5. Computer arrangement according to claim 2, wherein said processor calculates said motion vector assuming that a speed and direction of motion of any object detected within said range sensor data from said at least first and second range sensor is substantially constant.

6. Computer arrangement according to claim 2, wherein said identifying said second points cloud comprises distinguishing said at least one object from a reference object fixed to the earth.

7. Computer arrangement according to claim 1, wherein said identifying said first points cloud comprises distinguishing said at least one object from a reference object fixed to the earth.

8. Computer arrangement according to claim 6, wherein said reference object is a building.

9. Computer arrangement according to claim 1, wherein said image processing technique includes at least one of blurring said at least one portion, defocusing said at least one portion and substituting said at least one portion by image data.

10. Computer arrangement according to claim 1, wherein said computer program is arranged to allow said processor to identify said at least one portion by using at least one of an object recognition technique and a character recognition technique.

11. Computer arrangement according to claim 1, wherein said at least one portion includes privacy sensitive data.

12. Computer arrangement according to claim 1, wherein said object image data belongs to a certain scene, said image processing technique includes removing said object image data in said image data, and replacing said object image data in said scene by data that would have been visible in that scene if said object would not have been present.

13. Computer arrangement according to claim 1, wherein said image data relates to a plurality of images each one of them showing said same object, and said processor is arranged to produce processed object image by said action of performing said predetermined image processing technique on at least said portion of said object image data in one of said plurality of images, and to substitute said object in other ones of said plurality of images by said processed object image.

14. Data processing system comprising:

a computer arrangement according to claim 1; and

a mobile system, said mobile system including

a position determination device for providing said time and position and orientation data,

at least a first range sensor for providing said first range sensor data, and

at least one camera for providing said image data.

15. Method of mapping first range sensor data from a first range sensor to image data from at least one camera, said first range sensor and said at least one camera being located on a mobile system in a fixed relation to one another, the method comprising:

receiving time and position data from a position determination device on board said mobile system, receiving said first range sensor data from said first range sensor on board said mobile system and receiving said image data from said at least one camera on board said mobile system;

identifying a first points cloud within said first range sensor data, relating to at least one object;

producing a mask relating to said object and based on said first points cloud;

mapping said mask onto object image data relating to said same object as present in said image data from said at least one camera; and

performing an image processing technique on at least a portion of said object image data.

16. Method according to claim 15, wherein said method further comprises:

receiving second range sensor data from a second range sensor on board said mobile system;

identifying a second points cloud within said second range sensor data, relating to said same at least one object;

calculating a motion vector for said at least one object from said first and second points clouds; and

mapping said mask onto said object image data by using said motion vector.

17. Method according to claim 16, wherein said method further comprises:

calculating a real size of said object based on said first and second range sensor data; and

using said real size while producing said mask.

18. Computer program product comprising data and instructions that can be loaded by a computer arrangement, allowing said computer arrangement to perform the methods according to claim 15.

19. Data carrier provided with a computer program product as claimed in claim 18.

26. A computer readable medium including program segments for, when executed on a computer device, causing the computer device to implement the method of claim 15.

20. Computer arrangement, comprising:

a processor; and

at least one memory connected to the processor, the at least one memory comprising a computer program comprising data and instructions arranged to allow said processor to;

receive time and position data from a position determination device on board a mobile system, receive first range sensor data from at least a first range sensor on board said mobile system and receive second range sensor data from a second range sensor on board said mobile system;

identify a first points cloud within said first range sensor data, relating to at least one object;

identify a second points cloud within said second range sensor data, relating to said same at least one object; and

calculate a motion vector for said at least one object from said first and second points clouds.

21. Computer arrangement according to claim 20, wherein said computer program product is arranged to allow said processor to:

calculate said motion vector while calculating a first mass centre in said first points cloud and a second mass centre in said second points cloud, and

establish a path length between said first mass centre and said second mass centre.

22. Computer arrangement according to claim 20, wherein said computer program is arranged to allow said processor to:

calculate a real length of said object based on said first and second range sensor data.

23. Method of calculating a motion vector for an object, comprising:

receiving time and position data from a position determination device on board a mobile system, receiving first range sensor data from at least a first range sensor on board said mobile system and receiving second range sensor data from a second range sensor on board said mobile system;

identifying a first points cloud within said first range sensor data, relating to at least one object;

identifying a second points cloud within said second range sensor data, relating to said same at least one object; and

calculating a motion vector for said at least one object from said first and second points clouds.

24. Computer program product comprising data and instructions that can be loaded by a computer arrangement, allowing said computer arrangement to perform the methods according to claims 23.

25. Data carrier provided with a computer program product as claimed in claim 24.

27. A computer readable medium including program segments for, when executed on a computer device, causing the computer device to implement the method of claim 23.