Coordinated version control system, method, and recording medium for parameter sensitive applications
First Claim
1. A coordinated version control system including a parameter server having a plurality of parameter sets and a plurality of learners communicating with the plurality of parameter sets, the system comprising:
- a leader parameter collecting circuit configured to collect parameter set data for a leader parameter set from the plurality of learners;
a follower parameter collecting circuit configured to collect parameter set data for a follower parameter set from the plurality of learners;
a leader aggregated parameter generating circuit configured to generate a new leader version of a leader aggregated parameter set based on the leader parameter set data;
an event broadcasting circuit configured to generate a broadcast event indicating that the new leader version of the leader aggregated parameter set has been generated;
a broadcast detecting circuit configured to detect the broadcast event;
a checking circuit configured to check if a version of the follower parameter set matches the new leader version of the leader aggregated parameter set based on the broadcast detecting circuit detecting the broadcast event; and
a follower aggregated parameter generating circuit configured to generate a new follower version of a follower aggregated parameter that matches the new leader version of the leader aggregated parameter set.
1 Assignment
0 Petitions
Accused Products
Abstract
Version vector-based rules are used to facilitate asynchronous execution of machine learning algorithms. The method uses version vector based rule to generate aggregated parameters and determine when to return the parameters. The method also includes coordinating the versions of aggregated parameter sets among all the parameter servers. This allows to broadcast to enforce the version consistency; generate parameter sets in an on-demand manner to facilitate version control. Furthermore the method includes enhancing the version consistency at the learner'"'"'s side and resolving the inconsistent version when mismatching versions are detected.
-
Citations
20 Claims
-
1. A coordinated version control system including a parameter server having a plurality of parameter sets and a plurality of learners communicating with the plurality of parameter sets, the system comprising:
-
a leader parameter collecting circuit configured to collect parameter set data for a leader parameter set from the plurality of learners; a follower parameter collecting circuit configured to collect parameter set data for a follower parameter set from the plurality of learners; a leader aggregated parameter generating circuit configured to generate a new leader version of a leader aggregated parameter set based on the leader parameter set data; an event broadcasting circuit configured to generate a broadcast event indicating that the new leader version of the leader aggregated parameter set has been generated; a broadcast detecting circuit configured to detect the broadcast event; a checking circuit configured to check if a version of the follower parameter set matches the new leader version of the leader aggregated parameter set based on the broadcast detecting circuit detecting the broadcast event; and a follower aggregated parameter generating circuit configured to generate a new follower version of a follower aggregated parameter that matches the new leader version of the leader aggregated parameter set. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. A coordinated version control method for a parameter server having a plurality of parameter sets and a plurality of learners communicating with the plurality of parameter sets, the method comprising:
-
first collecting parameter set data for a leader parameter set from the plurality of learners; second collecting parameter set data for a follower parameter set from the plurality of learners; generating a new leader version of a leader aggregated parameter set based on the leader parameter set data; broadcasting a broadcast event indicating that the new leader version of the leader aggregated parameter set has been generated; checking if a version of the follower parameter set matches the new leader version of the aggregated parameter set based on the broadcast event being detected; and generating a new follower version of a follower aggregated parameter that matches the new leader version of the leader aggregated parameter set. - View Dependent Claims (12, 13, 14, 15, 16, 17, 18, 19)
-
-
20. An asynchronous execution facilitation system including a parameter server having a plurality of parameter sets and a plurality of learners communicating with the plurality of parameter sets, the system comprising:
-
a server checking circuit configured to check if a new version of an aggregated parameter set can be generated based on an intermediate version vector for each of the plurality of learners; an epoch update checking circuit configured to check a number of updated epochs based on a number of the plurality of learners that have pushed updated parameter set data to the parameter server according to the intermediate version vector; and an aggregation triggering circuit configured to generate the new version of the aggregated parameter set if the number of updated epochs is equal to the number of the plurality of learners that have pushed the updated parameter set data to the parameter server multiplied by a predetermined threshold portion of the plurality of learners.
-
Specification