ctf4science.benchmark_module

Benchmark Module for CTF models, benchmarks a model with optimal hyperparameters for a given dataset and pair_id.

This module provides a systematic evaluation of CTF models against a hidden test set. It also assesses model stability by running models multiple times with different random seeds.

Functions

main()

Run the ModelBenchmarker class from the command line.

Classes

ModelBenchmarker(config_path[, num_runs])

Benchmarks a model with optimal hyperparameters for a given dataset and pair_id.