Image display system, image display method and program

• US 9,689,688 B2
• Filed: 03/27/2014
• Issued: 06/27/2017
• Est. Priority Date: 03/30/2010
• Status: Active Grant

• Current Assignee: NS Solutions Corporation (Nippon Steel Corporation)
• Original Assignee: NS Solutions Corporation (Nippon Steel Corporation)
• Inventors: Yokoyama, Masaru, Sasao, Kazuhiro, Sakurai, Arata, Okamoto, Ricardo Musashi
• Primary Examiner(s): Beck, Leron
• Application Number:
  US14/228,159
• Publication Number:
  US 20140210853A1
• Time in Patent Office: 1,188 Days
• Field of Search: 348 16
• CPC Class Codes: B63B 2213/02 using satellite radio beacon positioning systems, e.g. the Global Positioning System GPS
• CPC Class Codes: B63B 43/18 preventing collision or grounding; reducing collision damage reducing pollution by collision B63B25/082
• CPC Class Codes: B63B 49/00 Arrangements of nautical instruments or navigational aids nautical measuring instruments G01C; radio navigation, analogous arrangements using other waves G01S
• CPC Class Codes: G01C 13/008 measuring depth of open water
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01C 21/203 Specially adapted for sailing ships
• CPC Class Codes: G01C 21/34 Route searching; Route guidance
• CPC Class Codes: G01C 21/3407 specially adapted for specific applications
• CPC Class Codes: G01C 21/3685 the POI's being parking facilities
• CPC Class Codes: G01C 21/3889 Transmission of selected map data, e.g. depending on route
• CPC Class Codes: G02B 27/01 Head-up displays
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders
• CPC Class Codes: G06Q 30/0207 Discounts or incentives, e.g. coupons or rebates
• CPC Class Codes: G06Q 30/06 Buying, selling or leasing transactions
• CPC Class Codes: G06Q 30/0601 Electronic shopping [e-shopping]
• CPC Class Codes: G06Q 30/0639 Item locations
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 19/003 : Navigation within 3D models or images
• CPC Class Codes: G06T 19/006 : Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G08G 1/04 : using optical or ultrasonic detectors
• CPC Class Codes: G08G 1/143 : inside the vehicles
• CPC Class Codes: G08G 3/00 : Traffic control systems for marine craft marking of navigational route B63B22/16, B63B51/00
• CPC Class Codes: G09B 29/007 : using computer methods
• CPC Class Codes: Y02A 90/30 : Assessment of water resources

Abstract

A mobile terminal obtains image data related to a product posted on a flier by imaging and sends the image data related to the product to a shopping management server. Upon reception of the image data related to the product from the mobile terminal, the shopping management server determines the product by analyzing the image data related to the product, obtains at least one of inventory information and sales information of the product, and sends at least one of the inventory information and the sales information of the product to the mobile terminal. The mobile terminal combines computer-graphics image data indicating at least one of the inventory information and the sales information of the product sent from the shopping management server with an image of the flier in real space and presents the combined image, thereby solving the problem.

Claims

1. An image display system, comprising:

a position information obtaining device obtaining position information of a ship,

an information storage device storing an electronic marine chart including seafloor shapes and a tide table,

an image display device,

an image providing device providing an image to the image display device, and

a direction obtaining device obtaining the direction of the ship,

wherein the image providing device is configured to;

obtain a draft of the ship according to at least information related to cargo or a number of passengers of the ship stored in the information storage device and a depth below sea level at a position of the ship based on the position information of the ship obtained in the position information obtaining device, the tide table, and information related to depths indicated on the electronic marine chart,

determine, based on the draft, whether a distance from a lowest part of a current position of the ship to the seafloor shape is within a predetermined distance range to avoid a collision,

determine a visual field from the image display device based on position information of the image display device determined based on the position information of the ship obtained in the position information obtaining device, the direction obtained in the direction obtaining device, and a size of the image display device,

create a virtual space image of a seafloor shape below sea level at the depth within the visual field based on the draft, the position information, the determined visual field, the depth below sea level, and seafloor topography image data indicated on the electronic marine chart, the virtual space image indicating a relationship between the distance and the predetermined distance range to avoid a collision, and

provide the virtual space image to the image display device, and

wherein the image display device is configured to display the virtual space image of the seafloor shape by combining with a view from the image display device.

2. The image display system according to claim 1, further comprising a radar device,

wherein the image providing device compares an image below sea level based on an echo of the radar device with seafloor topography image data indicated on the electronic marine chart, creates, when any obstacle not indicated on the electronic marine chart is detected below sea level, a virtual space image of the seafloor shape below sea level including the obstacle, and provides the virtual space image to the image display device.

3. The image display system according to claim 1, further comprising a detecting device detecting a posture of the ship,

wherein the image providing device performs coordinate conversion so that a sea level of the created virtual space image of the seafloor shape becomes horizontal with a sea level of the view from the image display device based on the posture of the ship detected in the detecting device, and provides the converted image to the image display device.

4. The image display system according to claim 1, further comprising a receiving device receiving information,

wherein the receiving device receives information of other ships in a sea area where the ship is present from a marine traffic center,

wherein the image providing device creates a virtual space image including the information of other ships received in the receiving device and provides the virtual space image to the image display device, and

wherein the image display device displays the virtual space image of the seafloor shape provided by the image providing device and the virtual space image including the information of other ships in a manner overlaid on a view from the image display device.

5. The image display system according to claim 1, further comprising:

a radar device; and

a receiving device receiving information,

wherein the receiving device receives from another ship an echo of a radar device of the other ship which is obtained by the other ship,

wherein the image providing device compares an image based on the echo of the radar device with an image based on the echo of the radar device of the other ship received by the receiving device to thereby create a virtual space image including an obstacle which is detected with the echo of the radar device and an obstacle which is not detected with the echo of the radar device and provides the created virtual space image to the image display device, and

wherein the image display device displays the virtual space image of the seafloor shape provided by the image providing device and the virtual space image including the obstacle which is detected with the echo of the radar device and the obstacle which is not detected with the echo of the radar device, in a manner overlaid on a view from the image display device.

8. The image display system according to claim 1, further comprising a transmissive display system attached to a windshield of the ship to display the virtual space image of the seafloor shape by combining with the view from the image display device and a view from the windshield.

6. An image display method in an image display system, comprising:

a position information obtaining device obtaining position information of a ship,

an information storage device storing an electronic marine chart including seafloor shapes and a tide table,

an image display device,

an image providing device providing an image to the image display device, and

a direction obtaining device obtaining the direction of the ship,

the image display method comprising;

obtaining a draft of the ship according to at least information related to cargo or a number of passengers of the ship stored in the information storage device and a depth below sea level at a position of the ship based on the position information of the ship obtained in the position information obtaining device, the tide table, and information related to depths indicated on the electronic marine chart,

determining, based on the draft, whether a distance from a lowest part of a current position of the ship to the seafloor shape is within a predetermined distance range to avoid a collision,

determining a visual field from the image display device based on position information of the image display device determined based on the position information of the ship obtained in the position information obtaining device, the direction obtained in the direction obtaining device, and a size of the image display device,

creating a virtual space image of a seafloor shape below sea level at the depth within the visual field based on the draft, the position information, the determined visual field, the depth below sea level, and seafloor topography image data indicated on the electronic marine chart, the virtual space image indicating a relationship between the distance and the predetermined distance range to avoid a collision, and

providing the virtual space image to the image display device by the image providing device; and

displaying the virtual space image of the seafloor shape by combining with a view from the image display device by the image display device.

7. A non-transitory computer-readable medium storing thereon a program causing a computer of an image display system which comprises a position information obtaining device obtaining position information of a ship, an information storage device storing an electronic marine chart including seafloor shapes and a tide table, an image display device, and the computer providing an image to the image display device, and a direction obtaining device obtaining the direction of the ship to execute:

obtaining a draft of the ship according to at least information related to cargo or a number of passengers of the ship stored in the information storage device and a depth below sea level at a position of the ship based on the position information of the ship obtained in the position information obtaining device, the tide table, and information related to depths indicated on the electronic marine chart;

determining, based on the draft, whether a distance from a lowest part of a current position of the ship to the seafloor shape is within a predetermined distance range to avoid a collision;

determining a visual field from the image display device based on position information of the image display device determined based on the position information of the ship obtained in the position information obtaining device, the direction obtained in the direction obtaining device, and a size of the image display device; and

creating a virtual space image of a seafloor shape below sea level at the depth within the visual field based on the draft, the position information, the visual field, the depth below sea level, and seafloor topography image data indicated on the electronic arine chart, the virtual space image indicating a relationship between the distance and the predetermined distance range to avoid a collision, and

providing the virtual space image to the image display device which displays the virtual space image by combining with a view from the image display device.