decent_bench.benchmark#

decent_bench.benchmark.benchmark(algorithms: list[DstAlgorithm], benchmark_problem: BenchmarkProblem, plot_metrics: list[PlotMetric] = DEFAULT_PLOT_METRICS, table_metrics: list[TableMetric] = DEFAULT_TABLE_METRICS, table_fmt: Literal['grid', 'latex'] = 'grid', *, n_trials: int = 30, confidence_level: float = 0.95, log_level: int = logging.INFO, max_processes: int | None = None) → None[source]#

Benchmark distributed algorithms.

Parameters:

  • algorithms – algorithms to benchmark

  • benchmark_problem – problem to benchmark on, defines the network topology, cost functions, and communication constraints

  • plot_metrics – metrics to plot after the execution, defaults to DEFAULT_PLOT_METRICS

  • table_metrics – metrics to tabulate as confidence intervals after the execution, defaults to DEFAULT_TABLE_METRICS

  • table_fmt – table format, grid is suitable for the terminal while latex can be copy-pasted into a latex document

  • n_trials – number of times to run each algorithm on the benchmark problem, running more trials improves the statistical results, at least 30 trials are recommended for the central limit theorem to apply

  • confidence_level – confidence level of the confidence intervals

  • log_level – minimum level to log, e.g. logging.INFO

  • max_processes – maximum number of processes to use when running trials, multiprocessing improves performance but can be inhibiting when debugging or using a profiler, set to 1 to disable multiprocessing or None to use ProcessPoolExecutor’s default

On this page