gcubed.benchmarking.comparator

This module contains the Comparator class that compares two Benchmarks instances.

class Comparator(gcubed.base.Base):

Overview

Create a set of benchmark results associated with a model and a version of Ox.

Comparator( old_benchmarks: gcubed.benchmarking.benchmarks.Benchmarks, new_benchmarks: gcubed.benchmarking.benchmarks.Benchmarks)

Overview

Analyses and reports on differences between the original and new benchmarks.

Arguments

old_benchmarks: The old benchmarks. Typically these have been run some time ago and are pickled (serialised to disk) and will need to be unpickled (restored to a Python object) before being passed to this Comparator for analysis.

new_benchmarks: The new benchmarks. Often these benchmarks are run and used immediately.

new_benchmarks: gcubed.benchmarking.benchmarks.Benchmarks

The new benchmarks.

old_benchmarks: gcubed.benchmarking.benchmarks.Benchmarks

The old benchmarks.

reporting_directory: str

The directory to save mismatch reports in.

save_reports: bool

True if reports are to be saved.

Defaults to False

FLOAT_FORMAT: str

The format to use for floating point values in CSV output.

Defaults to %.10f.

ABSOLUTE_TOLERANCE: float

The absolute allowable errors. See numpy.allclose for details.

Defaults to 0.00001.

RELATIVE_TOLERANCE: float

The relative allowable errors. See numpy.allclose for details.

Defaults to 0.00001.

DISCREPANCY_LIMIT: float

The discrepancy limit, used for determining which discrepancies to report.

Defaults to 0.00001.

def isclose(self, old: numpy.ndarray, new: numpy.ndarray) -> numpy.ndarray:

Arguments

old: The data frame of numbers from the old benchmarks.

new: The data frame of numbers from the new benchmarks.

Returns

A boolean array where two arrays are element-wise equal within the absolute and relative tolerances set for this Comparator.

Exceptions

Raises an exception if the two arrays are not the same shape.

def allclose( self, old: pandas.core.frame.DataFrame, new: pandas.core.frame.DataFrame) -> bool:

Arguments

old: The data frame of numbers from the old benchmarks.

new: The data frame of numbers from the new benchmarks.

Returns

True if arrays are element-wise equal within the absolute and relative tolerances set for this Comparator.

Exceptions

Raises an exception if the two arrays are not the same shape.

def compare_vectors( self, name: str, old: pandas.core.frame.DataFrame, new: pandas.core.frame.DataFrame, new_name: str = 'new', old_name: str = 'old') -> bool:

Overview

Compare two vectors.

Arguments

name: The name of the vector being compared.

old: The old vector

new: The new vector

old_name: The name of the old vector

new_name: The name of the new vector

Returns

True if the values in the vectors match.

def compare_matrices( self, name: str, old: pandas.core.frame.DataFrame, new: pandas.core.frame.DataFrame, new_name: str = 'new', old_name: str = 'old') -> bool:

Overview

Compare two matrices.

Arguments

name: The name of the matrix being compared.

old: The old matrix

new: The new matrix

new_name: The name of the new matrix

old_name: The name of the old matrix

Returns

True if the values in the matrices match.

def match_variables(self) -> bool:

Overview

Match the content in the SYM summaries of the model variables.

Checks to run:

  • The list of variables are the same.

Returns

True if the new and old benchmarks match.

def match_parameters(self) -> bool:

Overview

Ensure that there is exactly 1 new parameter for each old parameter and visa-versa.

Ensure that each parameter value is associated with the same parameter name and order and set members. and that the parameter values are identical (to within a reasonable level of numerical accuracy).

def match_linearisation_rhs_vector_values(self):

Overview

Compare RHS vector values used in initial model linearisation.

def match_nonlinear_model_lhs_vector_values(self):

Overview

Compare old and new LHS vector values calculated using the non-linear equations as part of model linearisation.

def match_linear_model_partial_derivatives(self):

Overview

Compare partial derivatives matrices for the model linearisation.

def match_ssf_matrices(self):

Overview

Compare state space form matrices computed from the linearised model partial derivative matrices.

def match_eigenvalues(self):

Overview

Check moduli of the state transition matrix eigenvalues.

def match_H1(self):

Overview

Check the H1 matrix in the stable manifold

def match_H2(self):

Overview

Check the H2 matrix in the stable manifold

def match_M1(self):

Overview

Check the M1 matrix in the stable manifold

def match_M2(self):

Overview

Check the M2 matrix in the stable manifold

def match_mu1(self):

Overview

Check the mu1 matrix in the stable manifold

def match_mu2(self):

Overview

Check the M2 matrix in the stable manifold

def match_Anew(self):

Overview

Check the Anew matrix in the stable manifold

def match_Znew(self):

Overview

Check the Znew matrix in the stable manifold

def match_Gamma_st(self):

Compare the Gamma_st matrix in the stable manifold.

def match_Gamma_jt(self):

Compare the Gamma_jt matrix in the stable manifold.

def match_Gamma_rT(self):

Compare the Gamma_rt matrix in the stable manifold.

def match_psi_rj(self):

Compare the psi_rj matrix in the stable manifold.

def match_psi_rs(self):

Compare the psi_rs matrix in the stable manifold.

def match_psi_rx(self):

Compare the psi_rx matrix in the stable manifold.

def match_tau_sjt(self):

Compare the tau_sjt matrix in the stable manifold.

def match_common_factor(self):

Compare the common factor matrix in the stable manifold.

def match_population(self):

Compare the population growth projection adjustments.

def match_productivity(self):

Compare productivity adjustments to exogenous projections.

def match_effective_labour_productivity(self):

Compare effective labour productivity adjustments to exogenous projections.

def match_potential_output_growth(self):

Benchmarks the potential output growth adjustments to exogenous projections.

def match_energy_usage_efficiency_gains(self):

Compare sectoral energy usage efficiency gain projections.

def match_consumption_energy_usage_efficiency_gains(self):

Benchmarks the consumption energy usage efficiency gain projections.

def match_first_year_observed_values_of_variables_adjusted_by_intertemporal_constants(self):

Benchmarks the first year observed values of variables adjusted_by intertemporal constants.

def match_first_year_original_projections_of_variables_adjusted_by_intertemporal_constants(self):

Benchmarks the first year original projections of variables adjusted by intertemporal constants.

def match_first_year_ssf_lhs_vector_values(self):

Benchmarks the first projection year state space form evaluations of the LHS vectors.

def match_first_projection_year_data_difference_from_ssf(self):

Benchmarks the first projection year data difference from state space form evaluations of the LHS vectors.

def match_partial_derivatives_wrt_intertemporal_constants(self):

Benchmarks the partial derivatives matrix used to compute intertemporal constants.

def match_intertemporal_constants(self):

Check that intertemporal constant values are the same.

def match_baseline_exogenous_variable_projections(self):

Benchmarks the baseline exogenous variable projections.

def match_first_projection_year_original_projections(self):

Benchmarks the first projection year original (unadjusted by constants) projections of the LHS vectors.

def match_baseline_c4t(self):

Benchmarks the baseline c4t matrix.

def match_baseline_h3t(self):

Benchmarks the baseline h3t matrix.

def match_baseline_initial_state_vector(self):

Benchmarks the baseline initial state vector.

def match_baseline_constantBL(self):

Benchmarks the baseline constantBL matrix.

def match_all_projections(self):

Match the details for all projections

Inherited Members
gcubed.base.Base
zeros
load_data
get_year_labels
log_calling_stack