Method for controlling display of electronic device using multiple controllers and device for the same

• US 10,814,220 B2
• Filed: 08/07/2018
• Issued: 10/27/2020
• Est. Priority Date: 08/29/2017
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Moon, Choon-Kyoung, Nah, Hyun-Soo, Choi, Hye-Jee
• Primary Examiner(s): Blancha, Jonathan M
• Application Number:
  US16/056,816
• Publication Number:
  US 20190060742A1
• Time in Patent Office: 812 Days
• Field of Search: 345 8
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: A63F 2300/8082 Virtual reality
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 2203/0382 Plural input, i.e. interface arrangements in which a plurality of input device of the same type are in communication with a PC
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

An electronic device is provided. The electronic device a display configured to output a screen, a communication circuit, at least one processor electrically connected with the display and the communication circuit, and a memory electrically connected with the at least one processor, wherein the memory stores instructions that configure the at least one processor to display a first pointer for a first controller based on first data obtained by a first sensor on the screen, and upon determining that a second controller points at the screen based on second data obtained by a second sensor of the second controller, control the display to delete the first pointer from the screen and to display a second pointer for the second controller based on the second data.

Claims

1. An electronic device, comprising:

a display configured to output a screen;

a communication circuit;

at least one processor electrically connected with the display and the communication circuit; and

a memory electrically connected with the at least one processor, wherein the memory stores instructions that configure the at least one processor to;

display, on the screen, a first pointer for a first controller based on first data obtained by a first sensor of the first controller, and

upon determining that a second pointer for a second controller points at the screen based on second data obtained by a second sensor of the second controller while displaying the first pointer for the first controller on the screen, control the display to delete the first pointer from the screen and to display the second pointer for the second controller based on the second data.

2. The electronic device of claim 1, wherein the memory further stores instructions that further configure the at least one processor to receive third data input into the second controller and control a first object indicated by the second pointer among a plurality of objects included in the screen based on the second data and the third data.

3. The electronic device of claim 1, wherein the memory further stores instructions that further configure the at least one processor to, when the first sensor is included in the first controller, receive the first data from the first controller using the communication circuit.

4. The electronic device of claim 3, wherein the memory further stores instructions that further configure the at least one processor to:

receive fourth data input into the first controller, and

when the second pointer for the second controller points to a location outside of the screen, control a second object indicated by the first pointer among a plurality of objects included in the screen based on the first data and the fourth data.

5. The electronic device of claim 4, wherein the memory further stores instructions that further configure the at least one processor to:

receive an input value for a button of the second controller, and

control the second object based on the input value, the first data, and the fourth data.

6. The electronic device of claim 4, wherein the memory further stores instructions that further configure the at least one processor to control the display to display a message to induce the second controller to move on the screen.

7. The electronic device of claim 1, wherein the memory further stores instructions that further configure the at least one processor to:

receive fifth data obtained by a third sensor of a third controller, and

upon determining that a third pointer for the third controller points at the screen based on the fifth data, control the display to display the third pointer on the screen.

8. The electronic device of claim 7, wherein the memory further stores instructions that further configure the at least one processor to:

display information for identifying the second controller or the third controller on the screen,

recognize the second controller upon receiving data from the second controller, and

recognize the third controller upon receiving data from the third controller.

9. The electronic device of claim 1, wherein the first controller or the second controller is configured to be worn on a user'"'"'s head or held in the user'"'"'s hand.

10. The electronic device of claim 1,

wherein the memory further stores instructions that further configure the at least one processor to generate a virtual reality (VR) space, and

wherein the screen is a portion of the VR space which is selected based on the first data.

11. The electronic device of claim 10, wherein the memory further stores instructions that further configure the at least one processor to control the display to display a lead line from a position of the second controller to the second pointer in the VR space.

12. The electronic device of claim 10, wherein the memory further stores instructions that further configure the at least one processor to:

when a position of the second controller in the VR space is included in a predetermined range, receive voice data obtained by the first sensor from the second controller, and

control the display based on the received voice data.

13. The electronic device of claim 1, wherein the memory further stores instructions that further configure the at least one processor to, when a distance between the first pointer and the second pointer is less than a threshold, associate data obtained by the first sensor with data obtained by the second sensor and control the display.

14. The electronic device of claim 13, wherein the memory further stores instructions that further configure the at least one processor to control the display to zoom in and display an object on the screen when the distance between the first pointer and the second pointer increases based on the data obtained by the first sensor and the data obtained by the second sensor and to zoom out and display the object on the screen when the distance between the first pointer and the second pointer decreases.

15. The electronic device of claim 13, wherein the memory further stores instructions that further configure the at least one processor to:

identify a third object corresponding to the first pointer and the second pointer among a plurality of objects displayed on the screen, and

control the third object based on data input into the first controller or the second controller.

16. A method for controlling, by an electronic device, a display using a plurality of controllers, the method comprising:

displaying, on a screen, a first pointer for a first controller based on first data obtained by a first sensor of the first controller;

receiving second data obtained by a second sensor of a second controller; and

upon determining that a second pointer for the second controller points at the screen based on the second data while displaying the first pointer for the first controller on the screen, removing the first pointer from the screen and displaying the second pointer for the second controller based on the second data.

17. The method of claim 16, further comprising:

receiving third data input into the second controller; and

controlling a first object on the screen that is selected by the second pointer based on the second data and the third data.

18. The method of claim 16, further comprising:

receiving fourth data input into the first controller; and

when the second pointer for the second controller points at a position outside of the screen, controlling a second object on the screen that is selected by the first pointer based on the first data and the fourth data.

19. The method of claim 18, further comprising:

receiving an input value for a button of the second controller; and

controlling the second object based on the input value, the first data, and the fourth data.

20. A computer-readable recording medium storing instructions configured to perform at least one operation by a processor, the at least one operation comprising:

displaying, on a screen, a first pointer for a first controller based on first data obtained by a first sensor of the first controller;

receiving second data obtained by a second sensor of a second controller; and

upon determining that a second pointer for the second controller points at the screen based on the second data while displaying the first pointer for the first controller on the screen, removing the first pointer from the screen and displaying the second pointer for the second controller based on the second data.