Electronic device and method for providing map service

• US 10,140,769 B2
• Filed: 11/24/2015
• Issued: 11/27/2018
• Est. Priority Date: 11/24/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kim, Kyung-Tae, Lee, Shin-Jun, Noh, Eun-Seon, Lee, Sun-Kee, Cheong, Cheol-Ho, Park, Hyung-Jin, Lee, Yo-Han, Han, Dong-Soo
• Primary Examiner(s): Patel, Jitesh
• Application Number:
  US14/950,168
• Publication Number:
  US 20160148417A1
• Time in Patent Office: 1,099 Days
• CPC Class Codes: G05D 1/0044 by providing the operator with a computer generated representation of the environment of the vehicle, e.g. virtual reality, maps maps used for automatic navigation G05D1/0274; flight directors G01C23/005
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09G 2370/025 LAN communication management
• CPC Class Codes: G09G 2370/04 Exchange of auxiliary data, i.e. other than image data, between monitor and graphics controller
• CPC Class Codes: G09G 2370/16 Use of wireless transmission of display information

Abstract

Provided is an electronic device and method for providing three-dimensional (3D) map processing and a 3D map service. The electronic device includes a memory configured to store an image set and a map platform module which is functionally connected with the memory and is implemented with a processor. The map platform module is configured to obtain an image set comprising a plurality of images for a path on an external space surrounding the electronic device, to determine an area corresponding to an object included in the external space from at least one of the plurality of images, to obtain information about the object based on whether the object is configured to communicatively connect with the electronic device, and to display the information in association with the area through a display functionally connected with the electronic device. Other embodiments are also possible.

Claims

1. A method for controlling an electronic device, the method comprising:

obtaining a first plurality of images corresponding to a first location in an external space surrounding the electronic device and a second plurality of images corresponding to a second location in the external space, wherein the obtaining comprises;

displaying a capturing guide indicating a direction for the electronic device to be moved; and

when a capturing focus of a camera of the electronic device matches the direction, automatically capturing an image corresponding to the direction to be included in the first plurality of images or the second plurality of images;

when a difference between a first time of capture for the first plurality of images and a second time of capture for the second plurality of images is smaller than a predetermined value, generating a 3D map of the external space based on the first plurality of images and the second plurality of images;

when the difference is equal to or greater than the predetermined value, overwriting previously stored image data with the first plurality of images or the second plurality of images;

identifying in the Three-Dimensional (3D) map an area corresponding to a first object and another area corresponding to a second object;

determining whether the first object and the second object are each capable of communicatively connecting with the electronic device;

when the first object is determined to be capable of communicatively connecting with the electronic device, connecting to the first object to obtain information relating to the first object;

displaying at least one of identification information of the first object, an indicator for indicating whether the first object is capable of communicating with the electronic device or not, a control interface for controlling the first object, and operation state information of the first object based on the information relating to the first object on a display;

when the second object is determined to be incapable of communicatively connecting with the electronic device;

obtaining information relating to the second object;

receiving a selection of the second object by a user; and

transmitting a control signal to the first object that includes the information relating to the second object.

2. The method of claim 1, further comprising displaying a first image and a second image, which are included in the first plurality of images or the second plurality of images, by switching the first image with the second image.

3. The method of claim 2, wherein the displaying of the first image and the second image by switching the first image with the second image comprises:

displaying at least one transition effect corresponding to the first image or the second image when the first image is switched with the second image.

4. The method of claim 1, wherein the obtaining of the image set comprises:

identifying a first feature point from at least a part of a first image included in the first plurality of images or the second plurality of images;

identifying a second feature point from at least a part of a second image included in the first plurality of images or the second plurality of images; and

generating (3D) information regarding a path corresponding to the external space based on comparison between the first feature point and the second feature point.

5. The method of claim 1, wherein the area corresponding to the first object and the other area corresponding to the second object are identified based on reference images for the first object and the second object.

6. The method of claim 1, wherein the obtaining of the information relating to the first object comprises:

obtaining the information from at least one of an external device, a user input, or the first object, based on a determination that the first object is capable of a communication connection with the electronic device.

7. The method of claim 1, further comprising:

simulating a function of the first object based on simulation information of the first object; and

displaying a virtual effect to be applied to on the first object or a virtual effect to be applied to the external space based on the simulation.

8. The method of claim 1, further comprising:

inserting the information into a part of the 3D map of the external space; and

converting the 3D map of the external space into a two-dimensional map.

9. The method of claim 1, further comprising transmitting the information relating to the first object or the second object to an external device.

10. An electronic device comprising:

a memory configured to store an image set; and

at least one processor operatively coupled to the memory, the at least one processor configured to;

obtain a first plurality of images corresponding to a first location in an external space surrounding the electronic device and a second plurality of images corresponding to a second location in the external space, wherein obtaining the first or second plurality of images comprises;

display a capturing guide indicating a direction for the electronic device to be moved; and

when a capturing focus of a camera of the electronic device matches the direction, automatically capture an image corresponding to the direction to be included in the first plurality of images or the second plurality of images;

when a difference between a first time of capture for the first plurality of images and a second time of capture for the second plurality of images is smaller than a predetermined value, generate a 3D map of the external space based on the first plurality of images;

when the difference is equal to or greater than the predetermined value, overwrite previously stored image data with the first plurality of images or the second plurality of images;

identify in the 3D map an area corresponding to a first object and another area corresponding to a second object;

determine whether the first object and the second object are each capable of communicatively connecting with the electronic device;

when the first object is determined to be capable of communicatively connecting with the electronic device, connect to the first object to obtain information relating the first object;

display at least one of identification information of the first object, an indicator for indicating whether the first object is capable of communicating with the electronic device or not, a control interface for controlling the first object, and operation state information of the first object based on the information relating to the first object on a display;

when the second object is determined to be incapable of communicatively connecting with the electronic device;

obtain information relating to the second object;

receive a selection of the second object by a user; and

transmit a control signal to the first object that includes the information relating to the second object.

11. The electronic device of claim 10, wherein the at least one processor is configured to display a first image and a second image of the first plurality of images or the second plurality of images by switching the first image with the second image.

12. The electronic device of claim 11, wherein the at least one processor is configured to display at least one transition effect corresponding to the first image or the second image when the first image is switched with the second image.

13. The electronic device of claim 10, wherein the at least one processor is configured to identify a first feature point from at least a part of a first image included in the first plurality of images or the second plurality of images, identify a second feature point from at least a part of a second image included in the first plurality of images or the second plurality of images, and generates three-dimensional information regarding a path corresponding to the external space based on comparison between the first feature point and the second feature point.

14. The electronic device of claim 10, wherein the area corresponding to the first object and the other area corresponding to the second object are identified based on reference images for the first object and the second object.

15. The electronic device of claim 10, wherein the at least one processor is configured to obtain the information relating to the first object from at least one of an external device, a user input, or the first object, based on a determination that the first object is capable of communication connection with the electronic device.

16. The electronic device of claim 10, wherein the information relating to the second object comprises at least one of identification information, connection information, control information, operation state information, and simulation information regarding the second object.

17. The electronic device of claim 10, wherein the at least one processor is configured to:

insert the information into a part of the 3D map of the external space; and

convert the 3D map of the external space into a two-dimensional map.

18. The electronic device of claim 10, wherein the at least one processor is configured to transmit the information relating to the first object or the second object to an external device.

19. A non-transitory computer-readable medium configured to direct at least one processor to perform at least one operation, the at least one operation comprising:

obtaining a first plurality of images corresponding to a first location in an external space surrounding an electronic device and a second plurality of images corresponding to a second location in the external space, wherein the obtaining comprises;

displaying a capturing guide indicating a direction for the electronic device to be moved; and

when a capturing focus of a camera of the electronic device matches the direction, automatically capturing an image corresponding to the direction to be included in the first plurality of images or the second plurality of images;

when a difference between a first time of capture for the first plurality of images and a second time of capture for the second plurality of images is smaller than a predetermined value, generating a 3D map of the external space based on the first plurality of images and the second plurality of images;

when the difference is equal to or greater than the predetermined value, overwriting previously stored image data with the first plurality of images or the second plurality of images;

identifying in the 3D map an area corresponding to a first object and another area corresponding to a second object;

determining whether the first object and the second object are each capable of communicatively connecting with the electronic device;

when the first object is determined to be capable of communicatively connecting with the electronic device, connecting to the first object to obtain information relating the first object;

displaying at least one of identification information of the first object, an indicator for indicating whether the first object is capable of communicating with the electronic device or not, a control interface for controlling the first object, and operation state information of the first object based on the information relating to the first object on a display;

when the second object is determined to be incapable of communicatively connecting with the electronic device;

obtaining information relating to the second object;

receiving a selection of the second object at the electronic device by a user; and

transmitting a control signal to the first object that includes the information relating to the second object.

20. A method for controlling an electronic device, the method comprising:

obtaining a first plurality of images corresponding to a first location in an external space surrounding the electronic device and a second plurality of images corresponding to a second location in the external space, wherein the obtaining comprises;

displaying a capturing guide indicating a direction for the electronic device to be moved; and

when a capturing focus of a camera of the electronic device matches the direction, automatically capturing an image corresponding to the direction to be included in the first plurality of images or the second plurality of images;

when a difference between a first time of capture for the first plurality of images and second time of capture for the second plurality of images is smaller than a predetermined value, generating a 3D map of the external space based on the first plurality of images and the second plurality of images;

when the difference is equal to or greater than the predetermined value, overwriting previously stored image data with the first plurality of images or the second plurality of images;

transmitting the generated 3D map to an external device;

determining a location of the external device on the generated 3D map; and

displaying, at the external device, the generated 3D map from a perspective based on the location of the external device.

21. An electronic device comprising:

a memory configured to store an image set; and

at least one processor operatively coupled to the memory, the at least one processor configured to;

obtain a first plurality of images corresponding to a first location of an external space surrounding the electronic device and a second plurality of images corresponding to a second location in the external space, wherein obtaining the first or second plurality of images comprises;

display a capturing guide indicating a direction for the electronic device to be moved; and

when a capturing focus of a camera of the electronic device matches the direction, automatically capture an image corresponding to the direction to be included in the first plurality of images or the second plurality of images;

when a difference between a first time of capture for the first plurality of images and second time of capture for the second plurality of images is smaller than a predetermined value, generate a 3D map of the external space based on the first plurality of images and the second plurality of images;

when the difference is equal to or greater than the predetermined value, overwrite previously stored image data with the first plurality of images or the second plurality of images;

transmit the generated 3D map to an external device;

determine a location of the external device on the generated 3D map; and

display, at the external device, the generated 3D map from a perspective based on the location of the external device.