Controller device, information processing system, and communication method

• US 9,615,048 B2
• Filed: 01/08/2013
• Issued: 04/04/2017
• Est. Priority Date: 05/25/2012
• Status: Active Grant

• Current Assignee: Nintendo Company Limited
• Original Assignee: Nintendo Company Limited
• Inventors: Ito, Kuniaki, Shimohata, Yasuyuki, Ueki, Jumpei
• Primary Examiner(s): Sitta, Grant
• Application Number:
  US13/736,364
• Publication Number:
  US 20140160361A1
• Time in Patent Office: 1,547 Days
• Field of Search: 345 87, 345204, 345156-173, 348734
• CPC Class Codes: A63F 13/235 using a wireless connection, e.g. infrared or piconet
• CPC Class Codes: A63F 13/26 having at least one additional display device, e.g. on the game controller or outside a game booth
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: H04N 21/41265 having a remote control device for bidirectional communication between the remote control device and client device unidirectional remote control device H04N21/42204
• CPC Class Codes: H04N 21/42204 User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/42208 Display device provided on the remote control

Abstract

An object is to appropriately transmit operation data from a controller device to an information processing device. The controller device is capable of wirelessly communicating with the information processing device. The controller device includes a generating unit, a communication unit, and a display unit. The generating unit generates operation data based on an operation performed on the controller device. The communication unit receives one image'"'"'s worth of image data from the information processing device with a first frequency, the image data generated in the information processing device through a process performed based on the operation data, and transmits the operation data to the information processing device with a second frequency that is higher than the first frequency. The display unit displays an image based on the image data.

Claims

1. A controller device capable of wirelessly communicating with an information processing device, the controller device comprising:

at least one processor configured to generate operation data based on an operation performed on the controller device;

a receiver configured to receive, from the information processing device, one image'"'"'s worth of image data with a first frequency, the image data generated in the information processing device through a process performed based on the operation data;

a transmitter configured to transmit the operation data to the information processing device with a second frequency that is higher than the first frequency; and

a display configured to display an image based on the image data,

wherein the operation data is transmitted by the transmitter with a frequency that is an integer multiple of the frequency with which the image data is received, where the integer multiple is greater than one,

wherein the information processing device transmits a sync signal to the controller device, and transmits the image data with a timing determined based on the sync signal, and

the operation data is transmitted with the timing determined based on the sync signal transmitted from the information processing device.

2. The controller device according to claim 1, wherein the display updates and displays the image data with the timing determined based on the sync signal and with the first frequency.

3. The controller device according to claim 2, wherein the display starts rendering the image data transmitted with the timing determined based on the sync signal within an amount of time shorter than an interval of the first frequency since reception of the sync signal.

4. The controller device according to claim 1, wherein the receiver is further configured to receive sound data from the information processing device with a timing independent of the sync signal.

5. The controller device according to claim 1, wherein:

the controller device further includes a microphone; and

the transmitter is further configured to transmit sound data obtained by the microphone to the information processing device with a timing independent of the sync signal.

6. The controller device according to claim 1, wherein

the controller device further includes a camera;

and

the transmitter is further configured to transmit one image'"'"'s worth of camera image data generated by the camera to the information processing device with a frequency that is lower than the first frequency.

7. The controller device according to claim 1, wherein one packet of sound data is received from the information processing device with a frequency that is higher than the first frequency and lower than the second frequency.

8. The controller device according to claim 1, wherein:

the controller device further includes a microphone; and

the transmitter is further configured to transmit one packet of sound data with a frequency that is higher than the first frequency and lower than the second frequency, the sound data generated by the microphone to the information processing device.

9. The controller device according to claim 1, wherein the transmitter is further configured to transmit newly-generated operation data for each transmission.

10. The controller device according to claim 1, further comprising a sensor including at least one of a touch panel, an acceleration sensor, and a gyrosensor,

wherein the at least one processor is further configured to generate operation data including data based on an output of the sensor.

11. A controller device capable of wirelessly communicating with an information processing device, the controller device comprising:

a camera;

at least one processor circuit configured to generate operation data based on an operation performed on the controller device;

a transceiver configured to;

transmit camera image data generated by the camera to the information processing device with a predetermined frequency, and

transmit the operation data with a frequency that is an integer multiple of the frequency with which one image'"'"'s worth of the camera image data is transmitted, the integer multiple being greater than one,

wherein the transceiver is configured to receive a sync signal from the information processing device and the camera image data is transmitted based on the received sync signal, and

where the transceiver is further configured to transmit the operation data with the timing determined based on the received sync signal transmitted from the information processing device.

12. The controller device according to claim 11, wherein the transceiver is further configured to receive one image'"'"'s worth of image data from the information processing device with a first frequency, the image data generated in the information processing device through a process performed based on the operation data, and transmits the operation data to the information processing device with a second frequency that is higher than the first frequency.

13. The controller device according to claim 12, wherein the transceiver is further configured to transmit the operation data with a frequency that is a second integer multiple of the frequency with which the image data is received, the second integer multiple being greater than one.

14. The controller device according to claim 11, further comprising:

a display device configured to display the image data with the timing determined based on the sync signal and with the first frequency.

15. The controller device according to claim 14, wherein the display device is further configured to start rendering the image data transmitted with the timing determined based on the sync signal within an amount of time shorter than an interval of the first frequency since reception of the sync signal.

16. The controller device according to claim 11, wherein the transceiver is further configured to receive sound data from the information processing device with a timing independent of the sync signal.

17. The controller device according to claim 11, wherein:

the controller device further includes a microphone; and

transceiver is further configured to transmit sound data obtained by the microphone to the information processing device with a timing independent of the sync signal.

18. The controller device according to claim 12, wherein the transceiver is further configured to receive one packet of sound data from the information processing device with a frequency that is higher than the first frequency and lower than the second frequency.

19. The controller device according to claim 12, wherein:

the controller device further includes a microphone; and

transceiver is further configured to transmit one packet of sound data with a frequency that is higher than the first frequency and lower than the second frequency, the sound data generated by the microphone to the information processing device.

20. The controller device according to claim 11, wherein the transceiver is further configured to transmit newly-generated operation data for each transmission.

21. The controller device according to claim 11, further comprising a sensor including at least one of a touch panel, an acceleration sensor, and a gyrosensor,

wherein the at least one processor circuit is further configured to generate operation data including data based on an output of the sensor.

22. A controller device capable of wirelessly communicating with an information processing device, the controller device comprising:

a processing system that includes at least one hardware processor, the processing system configured to generate operation data based on an operation performed on the controller device;

a near field communication (NFC) transceiver configured to communicate, via NFC, with another external device different from the information processing device;

a transmitter configured to;

transmit, to the information processing device, communication management data together with the operation data with a predetermined frequency, the communication management data representing management information for communication with the external device, and

transmit, to the information processing device by using a plurality of transmission opportunities, first-type data at a first priority and transmit second-type data at a second priority that is a lesser priority than the first priority, where the first-type data includes the communication management data and the operation data, and the second-type data includes data received from the another external device; and

a receiver configured to receive one image'"'"'s worth of image data from the information processing device with a first frequency, the image data generated in the information processing device through a process performed based on the operation data,

wherein the operation data is transmitted to the information processing device with a second frequency that is higher than the first frequency,

wherein the information processing device is configured to transmit a sync signal and the image data to the controller device, the image data transmitted with a timing determined based on the sync signal,

wherein the transmitter is further configured to transmit the operation data with the timing determined based on the sync signal transmitted from the information processing device.

23. The controller device according to claim 22, wherein the transmitter is further configured to transmit the operation data with a frequency that is an integer multiple of the frequency with which the image data is received, the integer multiple being greater than one.

24. The controller device according to claim 22, further comprising:

a display screen integrated into a housing of the controller, the processing system further configured to update what is displayed on the display screen using the image data at the timing determined based on the sync signal and with the first frequency.

25. The controller device according to claim 24, wherein the processing system is further configured to start a rendering process for the transmitted image data within an amount of time shorter than an interval of the first frequency since reception of the sync signal.

26. The controller device according to claim 22, wherein the receiver is further configured to receive sound data from the information processing device with a timing independent of the sync signal.

27. The controller device according to claim 22, wherein:

the controller device further includes a microphone; and

the transmitter is further configured to transmit sound data obtained by the microphone to the information processing device with a timing independent of the sync signal.

28. The controller device according to claim 22, wherein the receiver is further configured to receive one packet of sound data from the information processing device with a frequency that is higher than the first frequency and lower than the second frequency.

29. The controller device according to claim 22, wherein:

the controller device further includes a microphone; and

the transmitter is further configured to transmit one packet of sound data with a frequency that is higher than the first frequency and lower than the second frequency, the sound data generated by the microphone to the information processing device.

30. The controller device according to claim 22, wherein the transmitter is further configured to transmit newly-generated operation data for each transmission.

31. The controller device according to claim 22, further comprising a sensor including at least one of a touch panel, an acceleration sensor, and a gyrosensor,

wherein the processing system is further configured to generate operation data including data based on an output of the sensor.

32. The controller device according to claim 22, wherein communication with the information processing device is performed via the transmitter and receiver using a wireless communication method in which the transmission of the first-type data is given a higher priority over the transmission of the second-type data, and the reception of the image data transmitted from the information processing device is given a higher priority over the transmission of the first-type data.

33. An information processing system comprising:

an information processing device that comprises;

a wireless transceiver configured to wirelessly transmit a plurality of images at a first temporal frequency over a time period, each one of the plurality of images represented by corresponding image data that has been generated, and

a processing system with at least one hardware processor, the processing system configured to generate the corresponding image data based on operation data received via the transceiver; and

a controller device that comprises;

a wireless transceiver configured to repeatedly receive corresponding image data that represents one image in the plurality of images;

at least one hardware processor configured to generate operation data based on an operation performed on the controller device; and

a display device configured to display images based on the repeatedly received corresponding image data,

the transceiver configured to transmit the operation data to the information processing device at a second frequency that is higher than the first frequency over the time period,

wherein the second frequency is an integer multiple of the first frequency that is greater than one,

wherein the information processing device transmits a sync signal to the controller device, and transmits the image data with a timing determined based on the sync signal, and

the operation data is transmitted with the timing determined based on the sync signal transmitted from the information processing device.

34. An information processing system comprising:

an information processing device that includes a wireless transceiver and at least one hardware processor, the wireless transceiver configured to transmit a sync signal; and

a controller device comprising;

a camera;

at least one hardware processor configured to generate operation data based on an operation performed on the controller device; and

a wireless transceiver configured to;

transmit, at a first frequency over a time period, camera image data generated by the camera to the information processing device, and

transmit, at a second frequency over the time period, the operation data to the information processing device, where the second frequency is a greater than one integer multiple of the first frequency with which one image'"'"'s worth of the camera image data is transmitted, and

receive the sync signal transmitted by the wireless transceiver of the information processing device,

wherein the wireless transceiver of the information processing device is configured to receive the operation data and the camera image data,

wherein the camera image is transmitted at a timing that is based on the received sync signal,

where the transceiver is further configured to transmit the operation data based on the received sync signal transmitted from the information processing device.

35. An information processing system comprising:

an information processing device that includes a wireless transceiver and at least one hardware processor; and

a controller device comprising;

at least one hardware processor configured to generate operation data based on an operation performed on the controller device;

a first transceiver configured to communicate, via near field communication (NFC), with another external device different from the information processing device and receive data from the external device;

a second transceiver configured to

transmit, to the information processing device, communication management data together with the operation data with a predetermined frequency, the communication management data representing management information for communication with the external device, and

receive one image'"'"'s worth of image data from the information processing device with a first frequency, the image data being generated in the information processing device through a process performed based on the operation data received by the information processing device, wherein the operation data is transmitted to the information processing device with a second frequency that is higher than the first frequency,

the wireless transceiver of the information processing device configured to receive the operation data and the communication management data; and

wherein the second transceiver is further configured to transmit, to the information processing device by using a plurality of transmission opportunities, first-type data at a first priority and transmit second-type data at a second priority that is a lesser priority than the first priority, where the first-type data includes communication management data and the operation data, and the second-type data includes data received from the another external device,

wherein the information processing device is configured to transmit a sync signal and the image data to the controller device, the image data transmitted with a timing determined based on the sync signal, and

wherein the transmission of the operation by the second transceiver is timed based on the sync signal transmitted from the information processing device.

36. A communication method carried out in a controller device capable of wirelessly communicating with an information processing device, the communication method comprising:

generating operation data based on an operation performed on the controller device;

receiving one image'"'"'s worth of image data from the information processing device at a first frequency, the image data generated in the information processing device through a process performed based on the operation data,

during a time period over which the one image'"'"'s worth of image data is being received, transmitting, to the to the information processing device, operation data at a second frequency that is an integer multiple of the first frequency that is greater than one;

displaying, on a display screen of the controller device, an image based on the image data; and

receiving a sync signal that has been transmitted from the information processing device, where the image data is transmitted at a timing that is determined based on the received sync signal, and

the operation data is transmitted with the timing determined based on the sync signal transmitted from the information processing device.

37. A communication method carried out in a controller device capable of wirelessly communicating with an information processing device, the communication method comprising:

acquiring, from a camera disposed on the controller device, camera image data;

generating, via at least one hardware processor, operation data based on an operation performed on the controller device;

transmitting, by using a wireless transceiver of the controller device, the camera image data to the information processing device with a predetermined frequency, and transmitting the operation data with a frequency that is a greater than one integer multiple of the frequency with which one image'"'"'s worth of the camera image data is transmitted;

receiving a sync signal from the information processing device,

where the camera image data is transmitted at a timing that is based on the received sync signal,

where the operation data is transmitted with the timing determined based on the received sync signal transmitted from the information processing device.

38. A communication method carried out in a controller device capable of wirelessly communicating with an information processing device, the communication method comprising:

generating operation data based on an operation performed on the controller device; and

communicating, via a near field communication (NFC) transmitter included with the controller device, with another external device different from the information processing device; and

transmitting, to the information processing device by using a plurality of transmission opportunities, first-type data at a first priority and transmitting second-type data at a second priority that is a lesser priority than the first priority, where the first-type data includes the communication management data and the operation data, and the second-type data includes data received from the another external device;

receiving, from the information processing device at a first frequency, one image'"'"'s worth of image data, where the image data is generated by the information processing device through a process that is performed based on the operation data; and

wherein the operation data is transmitted to the information processing device with a second frequency that is higher than the first frequency,

wherein the information processing device is configured to transmit a sync signal and the image data to the controller device, the image data transmitted with a timing determined based on the sync signal,

wherein the transmitter is further configured to transmit the operation data with the timing determined based on the sync signal transmitted from the information processing device.