Omnidirectional image editing program and omnidirectional image editing apparatus

• US 9,692,965 B2
• Filed: 04/17/2012
• Issued: 06/27/2017
• Est. Priority Date: 04/25/2011
• Status: Active Grant

• Current Assignee: Mitsuo Hayashi
• Original Assignee: Mitsuo Hayashi
• Inventors: Hayashi, Mitsuo
• Primary Examiner(s): Crawford, Jacinta M
• Assistant Examiner(s): Nguyen, Phong
• Application Number:
  US14/113,330
• Publication Number:
  US 20140085412A1
• Time in Patent Office: 1,897 Days
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 3/12 Panospheric to cylindrical image transformations
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

This invention has an object to provide an image editing program and apparatus allowing users to intuitively produce images covering the omnidirection. In a virtual space, an omnidirectional image is projected according to a coordinate conversion and displayed at a display device 20. For omnidirectional image edition, a view direction and a view angle are changed in the virtual space with a pointing device 30, and under control of a central control unit 11, display of the omnidirectional image in a screen image is renewed to perform two-dimensional drawing to an image for drawing having the same coordinate system as that of the screen image with the pointing device 30 using the screen image as a clue. After the end of drawing, the coordinate conversion is made to the drawing image to project the image at a proper position of the omnidirectional image. The omnidirectional image is thus completed by repeating the steps of view direction changes and drawing.

Claims

1. An omnidirectional image editing apparatus, comprising:

a manipulation input unit configured to receive an input for manipulation of an omnidirectional image;

a displaying unit configured to display an image; and

a central control unit and a non-transitory medium containing instructions of an omnidirectional image editing program, execution of the instructions by the central control unit causing the central control unit to

receive a coordinate and a button input from the manipulation input unit,

determine that the manipulation of the omnidirectional image is in a view direction change mode, and subsequently renew a view direction angle and a view angle of the omnidirectional image based on the input from the manipulation input unit, perform screen renewal, and output a resultant image to the displaying unit,

determine that the manipulation of the omnidirectional image is in a drawing mode, and subsequently select a drawing parameter, determine a drawing image, and draw on the drawing image based on the drawing parameter, and

determine that the drawing mode has been completed, and subsequently perform omnidirectional image renewal by ascertaining a direction of each pixel of the omnidirectional image using predetermined direction pixel relationship data that associate a direction with a position of each pixel, delete the drawing image, perform another screen renewal, and output a resultant image to the displaying unit, wherein

a view point of the omnidirectional image remains fixed,

each screen renewal includes converting the omnidirectional image in a three-dimensional omnidirectional image coordinate system to a display image in a two-dimensional screen coordinate system, and

the omnidirectional image renewal includes converting the drawing image in the two-dimensional screen coordinate system to the omnidirectional image in the three-dimensional omnidirectional image coordinate system.

2. The omnidirectional image editing apparatus according to claim 1, wherein the central control unit is configured to overlap the drawing image on a screen image presented by the displaying unit based on a prescribed synthesizing method and to output the overlapped images to the displaying unit.

3. The omnidirectional image editing apparatus according to claim 2, wherein the central control unit, during the screen renewal, calculates a coordinate of a polygon apex corresponding to the view direction angle, calculates a coordinate of a polygon apex corresponding to the view angle, and performs a drawing processing for a screen image presented by the displaying unit based on a texture image.

4. The omnidirectional image editing apparatus according to claim 3, wherein

the screen image includes a plurality of polygon surfaces;

the texture image includes at least one layer, each layer corresponding to a number of the polygon surfaces; and

the central control unit, during the drawing processing for the screen image based on the texture image, scans pixels of the screen image, extracts and calculates opacity and color information from pixels of the texture image and pixels near the pixels of the texture image corresponding to the pixels of the screen image, renews the pixels of the screen image using the opacity and color information, and repeats the scanning, extracting, calculating and renewing until completion of the scanning corresponding to each polygon surface in the screen image, completion of extraction of the polygon surfaces corresponding to each layer, and completion of extraction of the at least one layer.

5. The omnidirectional image editing apparatus according to claim 2, wherein the central control unit, when performing the omnidirectional image renewal, calculates a coordinate of a polygon apex corresponding to the view direction angle, calculates a coordinate of a polygon apex corresponding to the view angle, and performs a drawing processing for a texture image based on the drawing image.

6. The omnidirectional image editing apparatus according to claim 5, wherein

the texture image includes a plurality of polygon surfaces;

the central control unit, during the drawing processing for the texture image based on the drawing image, scans pixels of the texture image, extracts and calculates opacity and color information from pixels of the drawing image and pixels near the pixels of the drawing image corresponding to the pixels of the texture image, renews the pixels of the texture image using the opacity and color information based on a prescribed synthesizing method, and repeats the scanning, extracting, calculating and renewing until completion of the scanning corresponding to each polygon surface in the texture image, and completion of extraction of the plurality of polygon surfaces.

7. The omnidirectional image editing apparatus according to claim 1, wherein the central control unit is configured to perform the another screen renewal after drawing on the drawing image and overlapping the drawing image on a screen image presented by the displaying unit based on a prescribed synthesizing method and outputting the overlapped images to the displaying unit.

8. The omnidirectional image editing apparatus according to claim 7, wherein the central control unit, during the screen renewal, calculates a coordinate of a polygon apex corresponding to the view direction angle, calculates a coordinate of a polygon apex corresponding to the view angle, and performs a drawing processing for a screen image presented by the displaying unit based on a texture image.

9. The omnidirectional image editing apparatus according to claim 8, wherein

the screen image includes a plurality of polygon surfaces;

the texture image includes a plurality of layers, each layer corresponding to a number of the polygon surfaces; and

the central control unit, during the drawing processing for the screen image based on the texture image, scans pixels of the screen image, extracts and calculates opacity and color information from pixels of the texture image and pixels near the pixels of the texture image corresponding to the pixels of the screen image, renews the pixels of the screen image using the opacity and color information, and repeats the scanning, extracting, calculating and renewing until completion of the scanning corresponding to each polygon surface in the screen image, completion of extraction of the polygon surfaces corresponding to each layer, and completion of extraction of the plurality of layers.

10. The omnidirectional image editing apparatus according to claim 7, wherein the central control unit, when performing the omnidirectional image renewal, calculates a coordinate of a polygon apex corresponding to the view direction angle, calculates a coordinate of a polygon apex corresponding to the view angle, and performs a drawing processing for a texture image based on the drawing image.

11. The omnidirectional image editing apparatus according to claim 10, wherein

the texture image includes a plurality of polygon surfaces;

the central control unit, during the drawing processing for the texture image based on the drawing image, scans pixels of the texture image, extracts and calculates opacity and color information from pixels of the drawing image and pixels near the pixels of the drawing image corresponding to the pixels of the texture image, renews the pixels of the texture image using the opacity and color information based on a prescribed synthesizing method, and repeats the scanning, extracting, calculating and renewing until completion of the scanning corresponding to each polygon surface in the texture image, and completion of extraction of the plurality of polygon surfaces.

12. The omnidirectional image editing apparatus according to claim 1, wherein the central control unit, during the screen renewal, calculates a coordinate of a polygon apex corresponding to the view direction angle, calculates a coordinate of a polygon apex corresponding to the view angle, and performs a drawing processing for a screen image presented by the displaying unit based on a texture image.

13. The omnidirectional image editing apparatus according to claim 12, wherein

the screen image includes a plurality of polygon surfaces;

the texture image includes at least one layer, each layer corresponding to a number of the polygon surfaces; and

the central control unit, during the drawing processing for the screen image based on the texture image, scans pixels of the screen image, extracts and calculates opacity and color information from pixels of the texture image and pixels near the pixels of the texture image corresponding to the pixels of the screen image, renews the pixels of the screen image using the opacity and color information, and repeats the scanning, extracting, calculating and renewing until completion of the scanning corresponding to each polygon surface in the screen image, completion of extraction of the polygon surfaces corresponding to each layer, and completion of extraction of the at least one layer.

14. The omnidirectional image editing apparatus according to claim 1, wherein the central control unit, when performing the omnidirectional image renewal, calculates a coordinate of a polygon apex corresponding to the view direction angle, calculates a coordinate of a polygon apex corresponding to the view angle, and performs a drawing processing for a texture image based on the drawing image.

15. The omnidirectional image editing apparatus according to claim 14, wherein

the texture image includes a plurality of polygon surfaces;

the central control unit, during the drawing processing for the texture image based on the drawing image, scans pixels of the texture image, extracts and calculates opacity and color information from pixels of the drawing image and pixels near the pixels of the drawing image corresponding to the pixels of the texture image, renews the pixels of the texture image using the opacity and color information based on a prescribed synthesizing method, and repeats the scanning, extracting, calculating and renewing until completion of the scanning corresponding to each polygon surface in the texture image, and completion of extraction of the plurality of polygon surfaces.

16. The omnidirectional image editing apparatus of claim 1, wherein the drawing parameter is a viewing angle that is between 140 degree and 179.75 degree.

17. A non-transitory computer readable medium storing instructions of an omnidirectional image editing program for manipulation of an omnidirectional image, execution of the instructions by a central control unit of a computer causing the central control unit to perform the steps of:

receiving information of a coordinate and a button from a manipulation input unit;

determining, by the central control unit, that the manipulation of the omnidirectional image is in a view direction change mode, and subsequently renewing a view direction angle and a view angle of the omnidirectional image based on the information received from the manipulation input unit, performing screen renewal, and outputting a resultant image to a displaying unit;

determining, by the central control unit, that the manipulation of the omnidirectional image is in a drawing mode, and subsequently selecting a drawing parameter, determining a drawing image and drawing on the drawing image based on the drawing parameter; and

determining, by the central control unit, that the drawing mode has been completed, and subsequently performing omnidirectional image renewal by ascertaining a direction of each pixel of the omnidirectional image using predetermined direction pixel relationship data that associate a direction with a position of each pixel, deleting the drawing image, performing another screen renewal, and outputting a resultant image to the displaying unit, wherein

a view point of the omnidirectional image remains fixed,

each screen renewal includes converting the omnidirectional image in a three-dimensional omnidirectional image coordinate system to a display image in a two-dimensional screen coordinate system, and

the omnidirectional image renewal includes converting the drawing image in the two-dimensional screen coordinate system to the omnidirectional image in the three-dimensional omnidirectional image coordinate system.

18. The non-transitory computer readable medium of claim 17, further comprising instructions execution of which by the central control unit causes the central control unit

to overlap the drawing image on a screen image presented by the displaying unit based on a prescribed synthesizing method, and

to output the overlapped images to the displaying unit.

19. The non-transitory computer readable medium of claim 17, wherein the step of performing the another screen renewal is performed by the central control unit after the step of drawing on the drawing image, a step of overlapping the drawing image on a screen image presented by the displaying unit based on a prescribed synthesizing method, and a step of outputting the overlapped images to the displaying unit.

20. The non-transitory computer readable medium of claim 17, wherein the drawing parameter is a viewing angle that is between 140 degree and 179.75 degree.

21. A method for manipulation of an omnidirectional image, comprising:

receiving information of a coordinate and a button from a manipulation input unit;

determining, by a central control unit of a computer, that the manipulation of the omnidirectional image is in a view direction change mode, and subsequently renewing a view direction angle and a view angle of the omnidirectional image based on the information received from the manipulation input unit, performing screen renewal, and outputting a resultant image to a displaying unit;

determining, by the central control unit, that the manipulation of the omnidirectional image is in a drawing mode, and subsequently selecting a drawing parameter, determining a drawing image, and drawing on a drawing image based on the drawing parameter; and

determining, by the central control unit, that the drawing mode has been completed, and subsequently performing omnidirectional image renewal by ascertaining a direction of each pixel of the omnidirectional image using predetermined direction pixel relationship data that associate a direction with a position of each pixel, deleting the drawing image, performing another screen renewal, and outputting a resultant image to the displaying unit, wherein

a view point of the omnidirectional image remains fixed,

each screen renewal includes converting the omnidirectional image in a three-dimensional omnidirectional image coordinate system to a display image in a two-dimensional screen coordinate system, and

the omnidirectional image renewal includes converting the drawing image in the two-dimensional screen coordinate system to the omnidirectional image in the three-dimensional omnidirectional image coordinate system.

22. The method of claim 21, further comprising, overlapping, by the central control unit, the drawing image on a screen image presented by the displaying unit based on a prescribed synthesizing method and outputting the overlapped images to the displaying unit.

23. The method of claim 21, wherein the step of performing the another screen renewal is performed by the central control unit after the step of drawing on the drawing image, a step of overlapping the drawing image on a screen image presented by the displaying unit based on a prescribed synthesizing method, and a step of outputting the overlapped images to the displaying unit.

24. The method of claim 21, wherein the drawing parameter is a viewing angle that is between 140 degree and 179.75 degree.