INFORMATION PROCESSING DEVICE, INFORMATION PROCESSING METHOD, INFORMATION PROCESSING PROGRAM, IMAGE PROCESSING DEVICE, AND IMAGE PROCESSING SYSTEM

• US 20200134783A1
• Filed: 03/27/2018
• Published: 04/30/2020
• Est. Priority Date: 04/28/2017
• Status: Active Grant

• Current Assignee: Sony Group Corp.
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: MORI, Hiroshi, SANO, Hiroyuki, MIYAMAKI, Hideo, TSUJIE, Masato, SATO, Yoichiro
• Granted Patent: US 11,341,608 B2
• Time in Patent Office: Days
• CPC Class Codes: G03B 15/00 Special procedures for taking photographs; Apparatus therefor
• CPC Class Codes: G03B 17/24 with means for separately producing marks on the film, e.g. title, time of exposure
• CPC Class Codes: G03B 37/00 Panoramic or wide-screen photography; Photographing extended surfaces, e.g. for surveying; Photographing internal surfaces, e.g. of pipe
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/662 by using master/slave camera arrangements for affecting the control of camera image capture, e.g. placing the camera in a desirable condition to capture a desired image
• CPC Class Codes: H04N 23/695 Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
• CPC Class Codes: H04N 5/222 Studio circuitry; Studio devices; Studio equipment cameras or camera modules comprising electronic image sensors, or control thereof H04N23/00

Abstract

Provided is an information processing device configured to: acquire an image captured according to an imaging instruction by an imaging device mounted to a mobile object; acquire sensor information including position information about the mobile object from a sensor device according to a signal transmitted from the imaging device in response to the imaging instruction; and associate the acquired sensor information with the acquired image.

Claims

1. An information processing device, comprising

processing circuitry configured to

acquire an image captured according to an imaging instruction by an imaging device mounted on a mobile object,

acquire sensor information including position information about the mobile object from a sensor device according to a signal transmitted from the imaging device in response to the imaging instruction, and

associate the acquired sensor information with the acquired image.

2. The information processing device according to claim 1, wherein

the sensor information includes altitude information about the mobile object.

3. The information processing device according to claim 1, wherein

the sensor information includes inclination information about the mobile object.

4. The information processing device according to claim 1, wherein

the signal indicates an exposure timing related to the capture of the image by the imaging device.

5. The information processing device according to claim 4, wherein

the sensor information is most recent sensor information at a point in time of the exposure timing.

6. The information processing device according to claim 1, wherein

the processing circuitry is further configured to compute new sensor information from the sensor information, and associate the new sensor information with the image.

7. The information processing device according to claim 4, wherein

the exposure timing is a timing of an end of an exposure that the imaging device performs to capture the image, a timing of a beginning of the exposure that the imaging device performs to capture the image, or a timing at which the imaging instruction for the imaging device to capture the image is issued.

10. The information processing device according to claim 1, wherein

the processing circuitry is further configured to create a composite image by arranging a plurality of images on a basis of the sensor information, the plurality of images including the acquired image.

11. The information processing device according to claim 10, wherein

the processing circuitry is further configured to arrange the plurality of images while correcting positions on a basis of the sensor information.

12. The information processing device according to claim 10, wherein

the sensor information includes altitude information about the mobile object, and

the processing circuitry is further configured to arrange the acquired image while enlarging or reducing the acquired image on a basis of the altitude information.

13. The information processing device according to claim 10, wherein

the sensor information includes inclination information about the mobile object, and

the processing circuitry is further configured to arrange the acquired image while performing a keystone correction and/or a trimming process on a basis of the inclination information.

14. The information processing device according to claim 1, wherein

the processing circuitry is further configured to acquire elevation data indicating an elevation of a ground surface imaged by the imaging device, and associate the elevation data with the sensor information.

15. The information processing device according to claim 14, wherein

the processing circuitry is further configured to associate the elevation data with the image.

16. The information processing device according to claim 14, wherein

the elevation data is acquired from map data containing information about a position and an elevation, on a basis of the position information about the mobile object.

17. The information processing device according to claim 14, wherein

the elevation data is acquired on a basis of distance information acquired by a ranging sensor mounted on the mobile object.

18. The information processing device according to claim 17, wherein

the processing circuitry is further configured to acquire the elevation data from the ranging sensor according to the signal transmitted from the imaging device in response to the imaging instruction, and associate the acquired elevation data with the sensor information.

8-9. -9. (canceled)

19. An information processing method, comprising:

acquiring an image captured according to an imaging instruction by an imaging device mounted on a mobile object;

acquiring sensor information including position information about the mobile object from a sensor device according to a signal transmitted from the imaging device in response to the imaging instruction; and

associating the acquired sensor information with the acquired image.

20. (canceled)

21. An image processing device, comprising:

processing circuitry configured to

receive a plurality of images with sensor information including position information about a mobile object acquired according to a signal transmitted from an imaging device in response to an imaging instruction, and

create a composite image by arranging the plurality of images on a basis of the sensor information.

22. The image processing device according to claim 21, wherein the processing circuitry is further configured to

create the composite image by arranging the plurality of images on a basis of elevation data indicating an elevation of a ground surface imaged by the imaging device.

23. The image processing device according to claim 22, wherein the processing circuitry is further configured to

define a first coordinate system corresponding to an imaging range of the imaging device in association with a movement of the mobile object and a second coordinate system corresponding to a plane in which one of the plurality of images is arranged,

compute a projected position of the one of the plurality of images on the ground surface in the first coordinate system on a basis of the sensor information and the elevation data, and

arrange the one of the plurality of images by converting the projected position to the second coordinate system.

26. The image processing device according to claim 22, wherein

the elevation data is acquired from map data containing information about a position and an elevation.

28. The image processing device according to claim 22, wherein

the elevation data is acquired by a ranging sensor mounted on the mobile object.

24-25. -25. (canceled)

27. (canceled)

29-31. -31. (canceled)