Apparatus and method for device administration using augmented reality in electronic device

• US 10,210,377 B2
• Filed: 02/10/2015
• Issued: 02/19/2019
• Est. Priority Date: 02/10/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kim, Sung-Chul, Kim, Hong-Ryol
• Primary Examiner(s): Thompson, James A
• Application Number:
  US14/618,457
• Publication Number:
  US 20150228124A1
• Time in Patent Office: 1,470 Days
• Field of Search: None
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 1/1696 the I/O peripheral being a printing or scanning device
• CPC Class Codes: G06F 1/1698 the I/O peripheral being a sending/receiving arrangement to establish a cordless communication link, e.g. radio or infrared link, integrated cellular phone details of antennas disposed inside a computer H01Q1/2266
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20212 Image combination
• CPC Class Codes: G06V 20/20 in augmented reality scenes
• CPC Class Codes: G06V 20/64 Three-dimensional objects

Abstract

Methods and apparatuses are provided for operation for an electronic device. Video is output on a display unit of the electronic device. The display unit displays a screen showing a range for device recognition, and displays at least one result of the device recognition corresponding to at least one device.

Claims

1. An electronic device comprising:

a display;

a first sensor;

a transceiver; and

a processor configured to;

acquire an image by using the first sensor;

control the transceiver to receive, from an external device, information including images and positions of a plurality of devices registered to the external device;

identify at least one device from the image based on the received information of the plurality of devices using at least a selected one of an object recognition, a text recognition, a near field communication recognition and a position information recognition;

present a menu for selecting a type of the at least one device to filter the selected type of the at least one device;

control the display to display a first interface comprising at least a part of the information of the filtered type of the at least one device overlaid on a position of the filtered type of the at least one device within the image; and

control the display, in response to an input on the first interface, to display a second interface for controlling the filtered type of the at least one device.

2. The electronic device of claim 1, wherein the processor is further configured to display a screen showing a range for device recognition.

3. The electronic device of claim 2, wherein the processor is further configured to display one of the image that is being taken through a camera unit of the electronic device and an electronic map.

4. The electronic device of claim 2, wherein the processor is further configured to recognize at least one of an image of a device, a name tag, an augmented reality (AR) marker, a quick response (QR) code, a bar code, an NFC signal, and user position information, from a video acquired through a camera of the electronic device.

5. The electronic device of claim 2, wherein the processor is further configured to control the display to display a screen pause button that freezes the image.

6. The electronic device of claim 2, wherein the image comprises at least one of an identification mode window, an explanatory note distinguishing device states, a radio detection and ranging (RADAR), and a distance adjustment meter.

7. The electronic device of claim 2, wherein the processor is further configured to set a distance range for the device recognition, in the image taken by the camera of the electronic device.

8. The electronic device of claim 2, wherein the processor is further configured to control the display to display the second interface on the electronic map.

9. The electronic device of claim 1, further comprising:

a memory configured to store device information, and

a second sensor configured to acquire information about a position of the electronic device and a camera view direction,

wherein the first sensor comprises a camera configured to acquire the image.

10. The electronic device of claim 9, wherein the second sensor comprises at least one of a global positioning system (GPS) receiver, a gyro sensor, a geomagnetic sensor, a gravity sensor, and a near field communication (NFC) sensor.

11. The electronic device of claim 1,

wherein the processor is further configured to present icons for selecting at least one of the object recognition, the text recognition, the near field communication recognition and the position information recognition.

12. The electronic device of claim 11,

wherein the processor is further configured to recognize the at least one device located within a camera shooting range and a preset distance based on a position of the electronic device and a camera view direction, and

wherein the camera shooting range displays a pole orientation of the electronic device and positions of the at least one device existing within the camera shooting range.

13. The electronic device of claim 12,

wherein the processor is further configured to present, on the second interface, at least one of an upper node and a name of the at least one device, a characteristic icon for checking a characteristic of the at least one device, a manual icon for checking a manual of the at least one device, an instruction icon for inputting an instruction to set the at least one device and specification information of the at least one device.

14. The electronic device of claim 13,

wherein, if a searched device does not exist within the camera shooting range based on the position of the electronic device and the camera view direction, the processor is further configure to control the display to display a direction of the searched device based on a position of the electronic device.

15. A method of operation for an electronic device, the method comprising:

acquiring an image by using a first sensor;

receiving, from an external device, information including images and positions of a plurality of devices registered to the external device;

identifying at least one device from the image based on the received information of the plurality of devices using at least a selected one of an object recognition, a text recognition, a near field communication recognition and a position information recognition;

presenting a menu for selecting a type of the at least one device to filter the selected type of the at least one device;

displaying a first interface comprising at least a part of the information of the filtered type of the at least one device overlaid on a position of the filtered type of the at least one device within the image; and

displaying, in response to an input on the first interface, a second interface for controlling the filtered type of the at least one device.

16. The method of claim 15, further comprising displaying a screen showing a range for device recognition.

17. The method of claim 16, wherein displaying the screen showing the range for device recognition comprises displaying one of the image that is being taken through a camera unit of the electronic device and an electronic map.

18. The method of claim 16, wherein displaying the first interface comprises recognizing at least one of an image of a device, a name tag, an augmented reality (AR) marker, a quick response (QR) code, a bar code, an NFC signal, and user position information.

19. The method of claim 16, wherein the image comprises a screen pause button for freezing the image on the display.

20. The method of claim 16, wherein the image comprises at least one of an identification mode window, an explanatory note distinguishing device states, a radio detection and ranging (RADAR), and a distance adjustment meter.

21. The method of claim 16, wherein displaying the first interface comprises setting a distance range to recognize a device, in the image that is being taken by the camera of the electronic device.

22. The method of claim 16, further comprising:

displaying the second interface on the electronic map.

23. The method of claim 15, further comprising:

acquiring information about a position of the electronic device and a camera view direction by using a second sensor,

wherein the first sensor comprises a camera.

24. The method of claim 23, wherein the second sensor comprises at least one of a global positioning system (GPS) receiver, a gyro sensor, a geomagnetic sensor, a gravity sensor, and a near field communication (NFC) sensor.

25. The method of claim 15, further comprising presenting icons for selecting at least one of the object recognition, the text recognition, the near field communication recognition and the position information recognition.

26. The method of claim 25,

wherein identifying the at least one device recognizes the at least one device located within a camera shooting range and a preset distance based on a position of the electronic device and a camera view direction, and

wherein the camera shooting range displays a pole orientation of the electronic device and positions of the at least one device existing within the camera shooting range.

27. The method of claim 26,

wherein the second interface further presents at least one of an upper node and a name of the at least one device, a characteristic icon for checking a characteristic of the at least one device, a manual icon for checking a manual of the at least one device, an instruction icon for inputting an instruction to set the at least one device and specification information of the at least one device.

28. The method of claim 27, further comprising, if a searched device does not exist within the camera shooting range on a basis of the position of the electronic device and the camera view direction, displaying a direction of the searched device based on a position of the electronic device.