Method of communicating via virtual space and system for executing the method

• US 10,432,679 B2
• Filed: 04/26/2018
• Issued: 10/01/2019
• Est. Priority Date: 04/26/2017
• Status: Active Grant

• Current Assignee: Colopl, Inc.
• Original Assignee: Colopl, Inc.
• Inventors: Nakabo, Takashi
• Primary Examiner(s): Hoang, Phi
• Application Number:
  US15/963,940
• Publication Number:
  US 20180316734A1
• Time in Patent Office: 523 Days
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 65/4053 without floor control
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04N 7/142 Constructional details of the terminal equipment, e.g. arrangements of the camera and the display
• CPC Class Codes: H04N 7/157 defining a virtual conference space and using avatars or agents computer conference optimisation or adaptation H04L12/1827

Abstract

A method to be executed by a first computer includes receiving at every first time interval a first signal output by a first sensor. The method further includes generating first data in accordance with the first signal. The method further includes receiving at every second time interval, which is shorter than the first time interval, a second signal output by a second sensor. The method further includes generating second data in accordance with the second signal. The method further includes generating a plurality of pieces of animation data including the first data and the second data. The method further includes transmitting the plurality of pieces of animation data to a second computer configured to communicate to/from the first computer. A total number of pieces of second data included in the plurality of pieces of animation data is smaller than a total number of pieces of second data generated by the first computer.

Claims

1. A method to be executed by a first computer, the method comprising:

receiving at every first time interval a first signal output from a first sensor;

generating first data in accordance with the received first signal;

receiving at every second time interval, which is shorter than the first time interval, a second signal output from a second sensor;

generating second data in accordance with the received second signal;

generating a plurality of pieces of animation data based on the first data and the second data, wherein each of the plurality of pieces of animation data includes at least one piece of the first data and at least one piece of the second data; and

transmitting each of the plurality of pieces of animation data to a second computer configured to communicate with the first computer,

wherein a total number of pieces of second data included in the plurality of pieces of animation data is less than a total number of pieces of second data generated by the first computer.

2. The method according to claim 1, further comprising associating a predetermined piece of the first data with a predetermined piece of the second data,

wherein each of the plurality of pieces of animation data includes the predetermined piece of first data and the predetermined piece of second data.

3. The method according to claim 2, further comprising:

identifying first time information associated with the received first signal;

identifying second time information associated with the received second signal;

associating the first time information with the first data;

associating the second time information with the second data;

identifying a predetermined piece of the second time information corresponding to a predetermined piece of the first time information; and

associating the first data associated with the predetermined piece of the first time information with the second data associated with the predetermined piece of second time information.

4. The method according to claim 2, further comprising:

storing the first data into a memory of the first computer;

storing the second data into the memory of the first computer;

identifying a most recent piece of the first data and a most recent piece of the second data; and

associating the latest piece of the second data with the latest piece of the first data.

5. The method according to claim 1, further comprising identifying an update count by which the second computer is configured to update an image per unit time,

wherein the second computer is configured to generate the image based on the plurality of pieces of animation data, and

wherein a total number of pieces of animation data to be generated by the first computer per unit time is equal to or smaller than the update count.

6. The method according to claim 1, further comprising:

generating an animation packet containing at least some of the plurality of pieces of animation data; and

transmitting the animation packet to the second computer at every third time interval.

7. The method according to claim 6, further comprising associating, with the animation packet, information representing a total number of pieces of the animation data to be generated by the first computer per unit time or information representing the third time interval.

8. The method according to claim 6, further comprising:

acquiring at every fourth time interval a sound signal from a third sensor;

generating a sound packet containing the sound signal acquired over a period of an integer multiple of the third time interval; and

transmitting the sound packet to the second computer.

9. The method according to claim 8, further comprising:

identifying third time information associated with the sound signal;

identifying fourth time information associated with a piece of animation data of the plurality of pieces of animation data generated earliest among the plurality of pieces of animation data contained in the animation packet;

identifying, based on the third time information, the sound signal acquired over the period from a timing indicated by the fourth time information; and

generating the sound packet containing the identified sound signal.

10. The method according to claim 8,

wherein the animation packet contains first identification information for distinguishing each of the plurality of animation packets,

wherein the sound packet contains second identification information, and

wherein the sound packet is associated with at least one animation packet of the plurality of animation packets in accordance with a relationship between the first identification information and the second identification information.

11. The method according to claim 1, further comprising:

defining a virtual space, wherein the virtual space comprises an avatar associated with the second computer;

receiving a second plurality of pieces of animation data from the second computer;

controlling a first part of the avatar in accordance with first data included in the second plurality of pieces of animation data;

controlling a second part of the avatar in accordance with second data included in the second plurality of pieces of animation data;

detecting a motion of a head-mounted device (HMD) connected to the first computer;

identifying a field of view in the virtual space in accordance with the detected motion; and

generating a field-of-view image in accordance with the field of view,

wherein the first part comprises a direction of a head of the avatar or a facial expression of the avatar, and

wherein the second part comprises a motion of an eyeball of the avatar.

12. A method to be executed by a first computer, the method comprising:

receiving at every first time interval a first signal output from a first sensor;

generating first data in accordance with the received first signal;

receiving at every second time interval, which is shorter than the first time interval, a second signal output from a second sensor;

generating second data in accordance with the received second signal;

associating a predetermined piece of the first data with a predetermined piece of the second data;

generating a plurality of pieces of animation data based on the first data and the second data, wherein each of the plurality of pieces of animation data includes the predetermined piece of first data and the predetermined piece of second data; and

transmitting each of the plurality of pieces of animation data to a second computer configured to communicate with the first computer,

wherein a total number of pieces of second data included in the plurality of pieces of animation data is less than a total number of pieces of second data generated by the first computer.

13. A method to be executed by a first computer, the method comprising:

receiving at every first time interval a first signal output from a first sensor;

generating first data in accordance with the received first signal;

receiving at every second time interval, which is shorter than the first time interval, a second signal output from a second sensor;

generating second data in accordance with the received second signal;

generating a plurality of pieces of animation data based on the first data and the second data;

acquiring at every fourth time interval a sound signal from a third sensor;

generating an animation packet containing at least some of the plurality of pieces of animation data;

transmitting the animation packet to a second computer at every third time interval, wherein the second computer is configured to communicate with the first computer,

generating a sound packet containing the sound signal acquired over a period of an integer multiple of the third time interval; and

transmitting the sound packet to the second computer,

wherein a total number of pieces of second data included in the plurality of pieces of animation data is less than a total number of pieces of second data generated by the first computer.