decent_bench.schemes

class decent_bench.schemes.AgentActivationScheme

Bases: ABC

Scheme defining how agents go active/inactive over the course of the algorithm execution.

Activation schemes are attached to agents by networks and are queried during algorithm execution.

abstractmethod is_active(iteration: int) → bool

Whether or not the agent is active.

Parameters:

iteration – current iteration of algorithm execution

class decent_bench.schemes.AlwaysActive

Bases: AgentActivationScheme

Scheme that makes the agent always active.

is_active(iteration: int) → bool

Whether or not the agent is active.

Parameters:

iteration – current iteration of algorithm execution

class decent_bench.schemes.UniformActivationRate(activation_probability: float)

Bases: AgentActivationScheme

Scheme where the agent is active with fixed probability.

Each call samples an independent Bernoulli event with probability activation_probability.

Parameters:

activation_probability – probability that the agent is active at a queried iteration.

Raises:

ValueError – if activation_probability is not in \([0, 1]\).

is_active(iteration: int) → bool

Whether or not the agent is active.

Parameters:

iteration – current iteration of algorithm execution

class decent_bench.schemes.MarkovChainActivation(inactive_to_active: float = 0.5, active_to_inactive: float = 0.5)

Bases: AgentActivationScheme

Scheme modeling activation with a 2-state Markov chain.

The scheme models activation with a 2-state (active and inactive) Markov chain. The agent transitions between the two states with the given probabilities.

Parameters:

• inactive_to_active – transition probability from inactive to active

• active_to_inactive – transition probability from active to inactive

Raises:

ValueError – if inactive_to_active or active_to_inactive are not in \([0, 1]\)

is_active(iteration: int) → bool

Whether or not the agent is active.

Parameters:

iteration – current iteration of algorithm execution

class decent_bench.schemes.PoissonActivation(mean_interval: float = 1.0)

Bases: AgentActivationScheme

Scheme modeling activation at random intervals determined by a Poisson distribution.

The agent activates at random intervals of length sampled from a Poisson distribution of given mean.

Parameters:

mean_interval – mean interval of inactivity

Raises:

ValueError – if mean_interval is negative

is_active(iteration: int) → bool

Whether or not the agent is active.

Parameters:

iteration – current iteration of algorithm execution

class decent_bench.schemes.CyclicActivation(active_for: int, inactive_for: int | None = None, offset: int = 0)

Bases: AgentActivationScheme

Scheme where an agent cycles through active and inactive intervals.

The agent is active for active_for iterations and inactive for inactive_for iterations in each cycle. If inactive_for is not provided, it defaults to active_for. offset shifts the phase of the cycle, allowing agents to follow the same cycle with staggered active windows.

Parameters:

• active_for – number of active iterations in each cycle.

• inactive_for – number of inactive iterations in each cycle. If None, it defaults to active_for.

• offset – phase offset applied to the cycle.

Raises:

ValueError – if active_for, inactive_for, or offset is negative, both intervals are zero, or iteration is negative.

is_active(iteration: int) → bool

Whether or not the agent is active.

Parameters:

iteration – current iteration of algorithm execution

class decent_bench.schemes.ClientSelectionScheme

Bases: ABC

Scheme defining how to select a subset of available clients.

Federated algorithms call select() once per round with the currently active clients. Implementations should return a subset without modifying the input sequence.

abstractmethod select(clients: Sequence[Agent], iteration: int) → list[Agent]

Select a subset of available clients.

Parameters:

• clients – available clients

• iteration – current iteration of algorithm execution

class decent_bench.schemes.UniformSelection(*, num_selected_clients: int | None = None, fraction_selected_clients: float | None = None)

Bases: ClientSelectionScheme

Uniform client selection.

The scheme samples clients uniformly without replacement. It selects either a fixed number of clients or a fraction of the clients passed to select().

Parameters:

• num_selected_clients – number of provided clients to sample.

• fraction_selected_clients – fraction of provided clients to sample.

Raises:

ValueError – if the selection size is invalid.

select(clients: Sequence[Agent], iteration: int) → list[Agent]

Select a subset of available clients.

Parameters:

• clients – available clients

• iteration – current iteration of algorithm execution

class decent_bench.schemes.DataSizeSelection(*, num_selected_clients: int | None = None, fraction_selected_clients: float | None = None)

Bases: ClientSelectionScheme

Data-size weighted client selection [1].

The scheme samples clients without replacement with probability proportional to each client’s local data size. The sampling probability for client \(i\) is

\[p_i = \frac{n_i}{\sum_{j \in \mathcal{C}} n_j},\]

where \(n_i\) is the client’s inferred local data size and \(\mathcal{C}\) is the client pool passed to select().

Parameters:

• num_selected_clients – number of provided clients to sample.

• fraction_selected_clients – fraction of provided clients to sample.

Raises:

ValueError – if the selection size is invalid or any client’s data size cannot be inferred.

[1]

Tao Qi, Fangzhao Wu, Lingjuan Lyu, Yongfeng Huang, and Xing Xie. Fedsampling: a better sampling strategy for federated learning. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, 4154–4162. 2023. URL: https://www.ijcai.org/proceedings/2023/462, doi:10.24963/ijcai.2023/462.

select(clients: Sequence[Agent], iteration: int) → list[Agent]

Select a subset of available clients.

Parameters:

• clients – available clients

• iteration – current iteration of algorithm execution

class decent_bench.schemes.FairSelection(*, num_selected_clients: int | None = None, fraction_selected_clients: float | None = None)

Bases: ClientSelectionScheme

Fair client selection inspired by fairness-aware client selection [2].

The scheme is a simplified count-based fairness rule that prioritizes clients with fewer past selections. It acts as a participation-balancing exploration rule: clients selected fewer times are prioritized so that the algorithm keeps exploring under-represented clients instead of repeatedly selecting the same ones. At round \(t\), let \(c_i(t)\) be the number of previous rounds in which client \(i\) was selected. For the client pool \(\mathcal{C}_t\) passed to select(), the selected set is

\[S_t \in \operatorname{arg\,min}_{S \subseteq \mathcal{C}_t,\ |S| = m} \sum_{i \in S} c_i(t),\]

where \(m\) is the resolved number of selected clients. Clients with the same count keep the order in which they were provided to select(). After selecting \(S_t\), the counts are updated as

\[c_i(t+1) = c_i(t) + \mathbf{1}\{i \in S_t\}.\]

Parameters:

• num_selected_clients – number of provided clients to sample.

• fraction_selected_clients – fraction of provided clients to sample.

Raises:

ValueError – if the selection size is invalid.

[2]

Yuxin Shi, Zelei Liu, Zhuan Shi, and Han Yu. Fairness-aware client selection for federated learning. In 2023 IEEE International Conference on Multimedia and Expo (ICME), 324–329. 2023. URL: https://arxiv.org/abs/2307.10738, arXiv:2307.10738, doi:10.1109/ICME55011.2023.00063.

select(clients: Sequence[Agent], iteration: int) → list[Agent]

Select a subset of available clients.

Parameters:

• clients – available clients

• iteration – current iteration of algorithm execution

class decent_bench.schemes.HighLossSelection(*, num_selected_clients: int | None = None, fraction_selected_clients: float | None = None)

Bases: ClientSelectionScheme

High-loss client selection inspired by Power-of-Choice [3].

The scheme evaluates each client’s local loss at its current local state x and selects the clients with highest loss, breaking ties at random. Unlike the Power-of-Choice strategy, this scheme does not trigger extra communication to evaluate losses at the current server model.

At round \(t\), for the client pool \(\mathcal{C}_t\) passed to select(), the selected set is

\[S_t \in \operatorname{arg\,max}_{S \subseteq \mathcal{C}_t,\ |S| = m} \sum_{i \in S} F_i(x_i),\]

where \(m\) is the resolved number of selected clients, \(F_i\) is client \(i\)’s local cost, and \(x_i\) is its current local state.

Parameters:

• num_selected_clients – number of provided clients to sample.

• fraction_selected_clients – fraction of provided clients to sample.

Raises:

• ValueError – if the selection size is invalid.

• RuntimeError – if any evaluated client’s x has not been initialized.

[3]

Yae Jee Cho, Jianyu Wang, and Gauri Joshi. Towards understanding biased client selection in federated learning. In Proceedings of the 25th International Conference on Artificial Intelligence and Statistics, volume 151 of Proceedings of Machine Learning Research, 10351–10375. PMLR, 2022. URL: https://proceedings.mlr.press/v151/jee-cho22a.html.

select(clients: Sequence[Agent], iteration: int) → list[Agent]

Select a subset of available clients.

Parameters:

• clients – available clients

• iteration – current iteration of algorithm execution

class decent_bench.schemes.CompressionScheme

Bases: ABC

Scheme defining how messages are compressed when sent over the network.

abstractmethod compress(msg: Array) → Array

Apply compression and return a new, compressed message.

compressed_msg_size(msg: Array) → int

Compute the size of the compressed version of msg.

class decent_bench.schemes.NoCompression

Bases: CompressionScheme

Scheme that leaves messages uncompressed.

compress(msg: Array) → Array

Apply compression and return a new, compressed message.

class decent_bench.schemes.Quantization(quantization_step: float)

Bases: CompressionScheme

Scheme applying uniform quantization to the message.

Given a message \(x\) and quantization step \(\Delta\), the scheme returns

\[q(x) = \Delta \operatorname{round}(x / \Delta)\]

where \(\operatorname{round}(\cdot)\) represents rounding to the nearest integer.

Raises:

ValueError – if quantization_step is not positive.

compress(msg: Array) → Array

Apply compression and return a new, compressed message.

class decent_bench.schemes.StochasticQuantization(n_levels: int)

Bases: CompressionScheme

Stochastic quantization used in QSGD [4].

The scheme quantizes each coordinate using n_levels stochastic levels scaled by the message norm. This keeps the compressed message unbiased in expectation while preserving the original message shape. Given a message \(x\) and \(s=\texttt{n\_levels}\), the quantizer computes

\[a_i = \frac{s |x_i|}{\lVert x \rVert_2}, \qquad \ell_i = \lfloor a_i \rfloor, \qquad p_i = a_i - \ell_i.\]

The quantization level is sampled as

\[\begin{split}\xi_i = \begin{cases} \ell_i + 1, & \text{with probability } p_i, \\ \ell_i, & \text{with probability } 1 - p_i, \end{cases}\end{split}\]

and the compressed coordinate is

\[Q_s(x_i) = \lVert x \rVert_2 \operatorname{sign}(x_i) \frac{\xi_i}{s}.\]

Parameters:

n_levels – number of stochastic quantization levels. Larger values give a finer quantization grid and usually lower quantization error. Smaller values give coarser quantization and stronger compression noise.

Raises:

ValueError – if n_levels is not positive.

Warning

This scheme computes the \(\ell_2\) norm of each message. This can be computationally expensive for large messages or when messages live on accelerator devices.

[4]

Dan Alistarh, Demjan Grubic, Jerry Li, Ryota Tomioka, and Milan Vojnovic. QSGD: communication-efficient SGD via gradient quantization and encoding. 2016. URL: https://arxiv.org/abs/1610.02132, arXiv:1610.02132, doi:10.48550/arXiv.1610.02132.

compress(msg: Array) → Array

Apply compression and return a new, compressed message.

class decent_bench.schemes.TopK(k: float)

Bases: CompressionScheme

Top-k compression which transmits only a subset of elements with largest magnitude.

The parameter k can be either:

• an int: transmit exactly k elements, or

• a float in \((0, 1]\): transmit a fraction k of elements.

Message size is preserved by transmitting zeros in place of non-transmitted elements.

Raises:

• ValueError – if k is a float and not in \((0, 1]\)

• ValueError – if k is an int and less than 1

Note

If k * n_elements < 1, at least one element is still transmitted.

compress(msg: Array) → Array

Apply compression and return a new, compressed message.

compressed_msg_size(msg: Array) → int

Compute the size of the compressed version of msg.

class decent_bench.schemes.RandK(k: float)

Bases: CompressionScheme

Rand-k compression which transmits only a random subset of elements.

The parameter k can be either:

• an int: transmit exactly k elements chosen uniformly at random (without replacement), or

• a float in \((0, 1]\): transmit a fraction k of elements.

Message size is preserved by transmitting zeros in place of non-transmitted elements.

Raises:

• ValueError – if k is a float and not in \((0, 1]\)

• ValueError – if k is an int and less than 1

Note

If k * n_elements < 1, at least one element is still transmitted.

compress(msg: Array) → Array

Apply compression and return a new, compressed message.

compressed_msg_size(msg: Array) → int

Compute the size of the compressed version of msg.

class decent_bench.schemes.DropScheme

Bases: ABC

Scheme defining how message drops occur over the network.

abstractmethod should_drop() → bool

Whether or not to drop.

class decent_bench.schemes.NoDrops

Bases: DropScheme

Scheme that never drops messages.

should_drop() → bool

Whether or not to drop.

class decent_bench.schemes.UniformDropRate(drop_rate: float)

Bases: DropScheme

Scheme that drops messages with uniform probability.

Each call samples an independent Bernoulli event with probability drop_rate.

Parameters:

drop_rate – probability that a message is dropped.

Raises:

ValueError – if drop_rate is not in \([0, 1]\).

should_drop() → bool

Whether or not to drop.

class decent_bench.schemes.GilbertElliott(drop_rate: float, bad_to_good: float = 0.5, good_to_bad: float = 0.5)

Bases: DropScheme

Drop scheme based on the Gilbert-Elliott model [5].

The Gilbert-Elliott model is characterized by a Markov chain with two states (good and bad), which can stay the same or transition into each other. In the bad state message drops occur with probability drop_rate, while in the good state no message drops occur.

Parameters:

• drop_rate – message drop rate while in the bad state

• bad_to_good – transition probability from bad to good state

• good_to_bad – transition probability from good to bad state

Raises:

ValueError – if drop_rate, bad_to_good or good_to_bad are not in \([0, 1]\)

[5]

Gerhard Hasslinger and Oliver Hohlfeld. The gilbert-elliott model for packet loss in real time services on the internet. In 14th GI/ITG Conference - Measurement, Modelling and Evalutation of Computer and Communication Systems, volume, 1–15. 2008. doi:.

should_drop() → bool

Whether or not to drop.

class decent_bench.schemes.NoiseScheme

Bases: ABC

Scheme defining the noise impacting messages.

abstractmethod make_noise(shape: tuple[int, ...], framework: SupportedFrameworks, device: SupportedDevices) → Array | None

Generate noise array of given shape (None if no noise).

class decent_bench.schemes.NoNoise

Bases: NoiseScheme

Scheme representing transmission without noise.

make_noise(_: tuple[int, ...], _framework: SupportedFrameworks, _device: SupportedDevices) → Array | None

Generate noise array of given shape (None if no noise).

class decent_bench.schemes.GaussianNoise(mean: float, std: float)

Bases: NoiseScheme

Scheme generating normal noise.

The scheme generates independent noise sampled from a normal distribution with mean mean and standard deviation std to each message entry.

Parameters:

• mean – mean of the normal noise.

• std – standard deviation of the normal noise.

Raises:

ValueError – if std is negative.

make_noise(shape: tuple[int, ...], framework: SupportedFrameworks, device: SupportedDevices) → Array

Generate noise array of given shape (None if no noise).