System and methods for displaying panoramic content

• US 9,904,982 B2
• Filed: 05/20/2014
• Issued: 02/27/2018
• Est. Priority Date: 05/31/2013
• Status: Active Grant

• Current Assignee: Nintendo Company Limited
• Original Assignee: Nintendo Company Limited
• Inventors: Suzuki, Toshiaki, Kira, Satoshi, Umehara, Akihiro
• Primary Examiner(s): Sams, Michelle L
• Application Number:
  US14/282,710
• Publication Number:
  US 20140354683A1
• Time in Patent Office: 1,379 Days
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 2200/1614 Image rotation following screen orientation, e.g. switching from landscape to portrait mode
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/20212 Image combination
• CPC Class Codes: G06T 3/067 Reshaping or unfolding 3D tree structures onto 2D planes
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: H04N 13/332 Displays for viewing with the aid of special glasses or head-mounted displays [HMD]

Abstract

An example of an information processing system displays, on a display device, a plurality of panoramic images associated with locations on a map. Here, a predetermined virtual object is set for at least one of the plurality of panoramic images. an information processing system 1 determines one of the locations on the map as a current location based on a user operation. Where the virtual object is set for a panoramic image associated with the current location, the virtual object is placed at a predetermined position of the panoramic image. The information processing system 1 displays, on the display device, an image of a partial area of the panoramic image associated with the current location. A predetermined information process is performed in response to satisfaction of a predetermined condition regarding the virtual object while the panoramic image for which the virtual object is set is displayed.

Claims

1. A non-transitory computer-readable storage medium storing a panoramic image display program for execution on a computer of an information processing device for displaying, on a display, a plurality of panoramic images each associated with a different location on a map,

wherein a first virtual object is set as a first target object for a first panoramic image associated with a first location on the map and a second virtual object is set as a second target object for a second panoramic image associated with a second location on the map different from the first location,

the panoramic image display program, when executed, causing the computer to perform operations comprising:

determining, based on a user operation, one of the locations on the map as a current location; and

based on determining that the current location corresponds to the first location on the map,

placing the first target object at a first position on the first panoramic image;

placing an additional object corresponding to the second target object at a second position on the first panoramic image, wherein the additional object has a shape related to a shape of the second virtual object and the second position for the additional object on the first panoramic image is determined based on a direction from the first location to the second location;

displaying, on the display, based on a user operation, an image of a partial area of the first panoramic image; and

performing a first information process based on satisfaction of a first condition regarding the first target object while the first panoramic image for which the first target object is set is displayed.

2. The storage medium according to claim 1, wherein the operations performed by the computer further comprise obtaining panoramic image data provided in advance in the information processing device or another information processing device, obtaining virtual object data provided in advance in the information processing device or another information processing device, and displaying, on the display, the image of the partial area of the first panoramic image using at least the obtained panoramic image data.

3. The storage medium according to claim 1, wherein:

the first condition comprises a condition regarding multiple target objects.

4. The storage medium according to claim 1, wherein the first condition comprises the first target object being displayed on the display.

5. The storage medium according to claim 1, wherein the first condition comprises detecting an operation for specifying the first target object displayed on the display.

6. The storage medium according to claim 1, wherein the first condition comprises the first panoramic image for which the first target object is set being displayed.

7. The storage medium according to claim 1, wherein a virtual object set as a target object for a respective panoramic image is displayed together with the respective panoramic image on a condition that a line-of-sight direction corresponding to an area of the respective panoramic image to be displayed is a direction within a range of directions.

8. The storage medium according to claim 1, wherein the operations performed by the computer further comprise displaying suggestion information on the display together with the first panoramic image, wherein the suggestion information suggests a position of the first target object on the map or on a respective panoramic image.

9. The storage medium according to claim 8, wherein a suggestion object is displayed, as the suggestion information, at a position on the first panoramic image determined based on a position of the first target object on the map.

10. The storage medium according to claim 8, wherein:

a suggestion object suggesting a position of the first target object on the first panoramic image is displayed as the suggestion information; and

the suggestion object indicates a direction toward the first target object from a position of the suggestion object on the first panoramic image.

11. The storage medium according to claim 1, wherein the first target object is erased when a second condition regarding the first target object is satisfied.

12. The storage medium according to claim 1, wherein the first information process comprises a process of providing the user a bonus.

13. The storage medium according to claim 1, wherein:

the first target object is associated with a location different from a location associated with the first panoramic image for which the first target object is set; and

the first information process comprises selecting the different location associated with the first target object based on satisfaction of the first condition, and displaying, on the display, a panoramic image associated with the selected location instead of the first panoramic image.

14. The storage medium according to claim 1, wherein the operations performed by the computer further comprise determining a placement position of the first target object on the second panoramic image based on a direction toward the first location from the second location on the map.

15. The storage medium according to claim 14, wherein the placement position of the first target object on the second panoramic image is determined based on a positional relationship on the map between a viewpoint of the second location and a position of the first target object on the first panoramic image.

16. The storage medium according to claim 15, wherein the placement position of the first target object on the second panoramic image is determined based on a distance from the second location to the first location, wherein the distance is calculated based on information indicating positions of the second location and the first location on the map.

17. The storage medium according to claim 14, wherein an orientation of the first target object at the first position on the second panoramic image represents an orientation of the first target object as viewed from a viewpoint of the second location.

18. The storage medium according to claim 1, wherein a position for first target object on the second panoramic image corresponds to the first position for the first target object on the first panoramic image relative to a viewpoint associated with the second location.

19. A panoramic image display system comprising:

a display for displaying a plurality of panoramic images each associated with a different location on a map;

a memory for storing virtual object information, wherein a first virtual object is set as a first target object for a first panoramic image associated with a first location on the map and a second virtual object is set as a second target object for a second panoramic image associated with a second location on the map different from the first location; and

one or more processors configured to perform operations comprising;

determining, based on a user operation, one of the locations on the map as a current location;

based on determining that the current location corresponds to the first location on the map,

placing the first target object at a first position on the first panoramic image;

placing an additional object corresponding to the second target object at a second position on the first panoramic image, wherein the additional object has a shape related to a shape of the second virtual object and the second position for the additional object on the first panoramic image is determined based on a direction from the first location to the second location;

displaying, on the display, based on a user operation, an image of a partial area of the first panoramic image; and

performing a first information process based on satisfaction of a first condition regarding the first target object while the first panoramic image for which the first target object is set is displayed.

20. The panoramic image display system according to claim 19, wherein a position for first target object on the second panoramic image corresponds to the first position for the first target object on the first panoramic image relative to a viewpoint associated with the second location.

21. A panoramic image display device for displaying, on a display, a plurality of panoramic images each associated with a different location on a map,

wherein a first virtual object is set as a first target object for a first panoramic image associated with a first location on the map and a second virtual object is set as a second target object for a second panoramic image associated with a second location on the map different from the first location, and

one or more processors of the panoramic image display device are configured to execute:

determining, based on a user operation, one of the locations on the map as a current location;

based on determining that the current location corresponds to the first location on the map,

placing the first target object at a first position on the first panoramic image;

placing an additional object corresponding to the second target object at a second position on the first panoramic image, wherein the additional object has a shape related to a shape of the second virtual object and the second position for the additional object on the first panoramic image is determined based on a direction from the first location to the second location;

displaying, on the display, based on a user operation, an image of a partial area of the first panoramic image; and

performing a first information process based on satisfaction of a first condition regarding the first target object while the first panoramic image for which the virtual object is set is displayed.

22. The panoramic image display device according to claim 21, wherein a position for first target object on the second panoramic image corresponds to the first position for the first target object on the first panoramic image relative to a viewpoint associated with the second location.

23. A panoramic image display method for carrying out on an information processing system for displaying, on a display, a plurality of panoramic images each associated with a different location on a map,

wherein a first virtual object is set as a first target object for a first panoramic image associated with a first location on the map and a second virtual object is set as a second target object for a second panoramic image associated with a second location on the map different from the first location,

the method comprising:

determining, based on a user operation, one of the locations on the map as a current location;

based on determining that the current location corresponds to the first location on the map,

placing the first target object at a first position on the first panoramic image;

placing an additional object corresponding to the second target object at a second position on the first panoramic image, wherein the additional object has a shape related to a shape of the second virtual object and the second position for the additional object on the first panoramic image is determined based on a direction from the first location to the second location;

displaying, on the display, based on a user operation, an image of a partial area of the first panoramic image; and

performing a first information process based on satisfaction of a first condition regarding the first target object while the first panoramic image for which the virtual object is set is displayed.

24. The panoramic image display method according to claim 23, wherein a position for first target object on the second panoramic image corresponds to the first position for the first target object on the first panoramic image relative to a viewpoint associated with the second location.