gcubed.performance_regression

Benchmark and compare G-Cubed model-solution performance.

The benchmark runs each trial in a fresh Python process with memory profiling disabled. This keeps production runtime measurements separate from instrumented memory profiles, whose event collection adds overhead of its own.

REQUIRED_MODEL_BUILD: str = '197'

Model build required for memory-reduction performance evidence.

MINIMUM_REPETITIONS: int = 3

Minimum number of fresh-process benchmark trials in an accepted result.

DEFAULT_MAX_MEDIAN_SLOWDOWN_PERCENT: float = 5.0

Default median slowdown that triggers explicit performance review.

BENCHMARK_EXECUTION_ENV_VARS: tuple[str, ...] = ('GCUBED_SSF_SPARSE_HYBRID', 'GCUBED_SSF_WORKSPACE_CAP_GIB', 'GCUBED_SSF_PROCESS_MEMORY_LIMIT_GIB', 'GCUBED_SSF_PERFORMANCE_PROFILE')

Execution-policy settings recorded without requiring them to match.

@dataclass(frozen=True)

class BenchmarkScope:

Focused build-197 test used by a performance benchmark scope.

Attributes: test_path: Test-file path relative to the repository root. selector: Pytest -k selector for the measured construction path.

BenchmarkScope(test_path: str, selector: str)

test_path: str

selector: str

BENCHMARK_SCOPES: dict[str, BenchmarkScope] = {'state-space-form': BenchmarkScope(test_path='tests/generic/test_linearisation/test_solving_linear_model.py', selector='test_state_space_form_creation'), 'solved-model': BenchmarkScope(test_path='tests/generic/test_linearisation/test_solving_linear_model.py', selector='test_stable_manifold_creation_and_pickling'), 'complete-solved-model': BenchmarkScope(test_path='tests/generic/test_setup/test_setup_of_model.py', selector='test_model_solution')}

Benchmark scopes and their focused real-model pytest tests.

def validate_benchmark_model_id(model_id: str) -> tuple[str, str]:

Validate and split a build-197 benchmark model identifier.

Args: model_id: Model identifier in <VERSION>_<BUILD> form.

Returns: A tuple containing model version and build.

Raises: ValueError: Raised when the model ID is malformed or does not use build 197.

def summarize_durations(durations_seconds: Sequence[float]) -> dict[str, float]:

Calculate summary statistics for benchmark trial durations.

Args: durations_seconds: Positive wall-clock durations in seconds.

Returns: Mean, median, minimum, maximum, and population standard deviation.

Raises: ValueError: Raised when no durations are supplied or a duration is not positive.

def benchmark_command( model_id: str, scope: str, *, repository_root: pathlib.Path | None = None) -> tuple[str, ...]:

Build the focused pytest command for one benchmark trial.

Args: model_id: Build-197 model identifier. scope: One of the keys in :data:BENCHMARK_SCOPES. repository_root: Optional gcubedcode repository root. The source checkout containing this module is used when omitted.

Returns: Command arguments suitable for subprocess.run.

Raises: ValueError: Raised for a malformed model ID, a non-197 build, or an unsupported scope. FileNotFoundError: Raised when the focused test file is unavailable.

def run_benchmark( model_id: str, *, scope: str, repetitions: int = 3, label: str | None = None, output_file: pathlib.Path | None = None, repository_root: pathlib.Path | None = None) -> dict[str, typing.Any]:

Run a production-mode performance benchmark in fresh processes.

Args: model_id: Build-197 model identifier, normally 6W_197. scope: Benchmark scope, such as state-space-form. repetitions: Number of measured fresh-process trials. Must be at least :data:MINIMUM_REPETITIONS. label: Optional experiment label such as before_sparse_storage. output_file: Optional JSON artifact path. repository_root: Optional gcubedcode repository root.

Returns: JSON-compatible benchmark result containing every trial and summary timing statistics.

Raises: ValueError: Raised for invalid model, scope, or repetition values. FileNotFoundError: Raised when the benchmark test file is unavailable. RuntimeError: Raised when a benchmark subprocess fails. OSError: Propagated when a subprocess cannot be started or the result artifact cannot be written.

def compare_benchmark_results( reference: Mapping[str, typing.Any], candidate: Mapping[str, typing.Any], *, max_median_slowdown_percent: float = 5.0) -> dict[str, typing.Any]:

Compare pre-change and post-change performance benchmark results.

Args: reference: Pre-change benchmark result. candidate: Post-change benchmark result. max_median_slowdown_percent: Largest accepted candidate median slowdown.

Returns: JSON-compatible report. passed is false when the median slowdown exceeds the supplied threshold.

Raises: ValueError: Raised when the threshold is negative, results are incompatible, use a non-197 build, contain too few trials, were run with memory profiling, or came from different environments. KeyError: Raised when a required result field is absent. TypeError: Raised when required result values have incompatible types.

def compare_benchmark_files( reference_file: pathlib.Path, candidate_file: pathlib.Path, *, report_file: pathlib.Path | None = None, max_median_slowdown_percent: float = 5.0) -> dict[str, typing.Any]:

Read, compare, and optionally report two benchmark artifacts.

Args: reference_file: Pre-change benchmark JSON path. candidate_file: Post-change benchmark JSON path. report_file: Optional comparison-report JSON path. max_median_slowdown_percent: Largest accepted median slowdown.

Returns: JSON-compatible performance comparison report.

Raises: FileNotFoundError: Raised when either benchmark file is missing. OSError: Propagated when benchmark or report files cannot be read or written. json.JSONDecodeError: Raised when a benchmark file is not valid JSON. ValueError: Propagated when results are not comparable. KeyError: Propagated when a required result field is absent. TypeError: Propagated when required values have incompatible types.

def main(argv: Sequence[str] | None = None) -> int:

Run the performance-regression command-line interface.

Args: argv: Optional argument sequence. argparse reads sys.argv when omitted.

Returns: Process exit code. 0 means success, while 1 means comparison found a material slowdown.

Raises: SystemExit: Raised by argparse for invalid arguments. ValueError: Propagated for invalid or incompatible benchmark data. FileNotFoundError: Propagated for missing test or artifact files. RuntimeError: Propagated when a benchmark trial fails. OSError: Propagated for subprocess or artifact I/O failures. json.JSONDecodeError: Propagated for malformed benchmark JSON.

def benchmark_execution_environment() -> dict[str, str | None]:

Return execution-policy variables used by a benchmark trial.

Args: None.

Returns: Environment values that intentionally may differ between benchmark candidates, such as workspace caps and process-memory limits.

Raises: No deliberate exceptions are raised.