decent_bench.metrics.runtime_library#

Collection of pre-defined runtime metrics.

class decent_bench.metrics.runtime_library.RuntimeLoss(update_interval: int, save_path: str | Path | None = None)[source]#

Bases: RuntimeMetric

Runtime loss metric.

Computes the average loss across all agents at each iteration. This is useful for monitoring convergence and detecting issues early.

The loss is computed as:

\[\text{loss} = \frac{1}{N} \sum_{i=1}^{N} f_i(\mathbf{x}_i)\]

where \(N\) is the number of agents, \(f_i\) is agent i’s cost function, and \(\mathbf{x}_i\) is agent i’s current optimization variable.

description = 'Loss'#

x_log = False#

y_log = False#

compute(_: BenchmarkProblem, agents: Sequence[Agent], __: int) → float[source]#

Compute the metric value for the current iteration.

Parameters:

problem – benchmark problem being solved
agents – sequence of agents with their current state
iteration – current iteration number

Returns:

The computed metric value as a float.

class decent_bench.metrics.runtime_library.RuntimeRegret(update_interval: int, save_path: str | Path | None = None)[source]#

Bases: RuntimeMetric

Runtime regret metric.

Requires a BenchmarkProblem with x_optimal not None.

Regret is computed as:

\[\text{regret} = \frac{1}{N} \sum_{i=1}^{N} f_i(\mathbf{x}_i) - \frac{1}{N} \sum_{i=1}^{N} f_i(\mathbf{x}^*)\]

where \(N\) is the number of agents, \(f_i\) is agent i’s cost function, \(\mathbf{x}_i\) is agent i’s current optimization variable, and \(\mathbf{x}^*\) is the optimal solution.

description = 'Regret'#

x_log = False#

y_log = False#

compute(problem: BenchmarkProblem, agents: Sequence[Agent], _: int) → float[source]#

Compute the metric value for the current iteration.

Parameters:

problem – benchmark problem being solved
agents – sequence of agents with their current state
iteration – current iteration number

Returns:

The computed metric value as a float.

class decent_bench.metrics.runtime_library.RuntimeGradientNorm(update_interval: int, save_path: str | Path | None = None)[source]#

Bases: RuntimeMetric

Runtime gradient norm metric.

Computes the average gradient norm across all agents at each iteration. This is useful for monitoring if the algorithm is making progress towards a stationary point.

The gradient norm is computed as:

\[\text{grad norm} = \frac{1}{N} \sum_{i=1}^{N} \|\nabla f_i(\mathbf{x}_i)\|\]

where \(N\) is the number of agents, \(f_i\) is agent i’s cost function, and \(\mathbf{x}_i\) is agent i’s current optimization variable.

description = 'Gradient Norm'#

x_log = False#

y_log = True#

compute(_: BenchmarkProblem, agents: Sequence[Agent], __: int) → float[source]#

Compute the metric value for the current iteration.

Parameters:

problem – benchmark problem being solved
agents – sequence of agents with their current state
iteration – current iteration number

Returns:

The computed metric value as a float.

class decent_bench.metrics.runtime_library.RuntimeConsensusError(update_interval: int, save_path: str | Path | None = None)[source]#

Bases: RuntimeMetric

Monitors how well agents agree on their decision variables.

This is useful for diagnosing issues in decentralized algorithms where agents are supposed to reach consensus.

The consensus error is computed as:

\[\text{consensus error} = \frac{1}{N} \sum_{i=1}^{N} \|\mathbf{x}_i - \bar{\mathbf{x}}\|\]

where \(N\) is the number of agents, \(\mathbf{x}_i\) is agent i’s current optimization variable, and \(\bar{\mathbf{x}}\) is the average of all agents’ optimization variables.