Smoothing and long-run continuation

Table of contents

The age-weighted regional totals produce raw annual percentage log growth rates. The data system applies several transformations to those rates, rather than smoothing population or NTA income levels. All stages below are part of the standard age-adjusted construction; describing it as just an exponentially weighted moving average (EWMA) omits important behavior.

Source endpoint and convergence window

Let $U$ be the last population-source year, and $T$ the model’s last projection year. The code extends country effective-labor levels by repeating their year-$U$ value far enough to calculate annual transitions through $T$. This initially gives zero raw growth after the source endpoint.

Growth columns are labelled by the transition’s starting year. Consequently, the last transition wholly within the source, $U-1$ to $U$, is labelled $U-1$. The code copies that rate into column $U$ as the continuation anchor. For the current source endpoint of 2100, raw column 2099 measures the 2099-to-2100 transition and raw column 2100 initially repeats it.

The convergence endpoint is

\[H=\min(T,U+50).\]

The smoothing calculation uses the construction’s available years from the data-configuration start year through $H$. Columns after $H$ are initialized to exact zero and excluded from smoothing. With the current source ending in 2100 and a model extending to 2200, smoothing ends in 2150; 2151–2200 are zero. The endpoint itself is also explicitly set to zero below.

This is a source-based convergence window, independent of the publication end year. Once $T\ge U+50$, further extensions of the numerical model horizon append zeros without changing the earlier path, provided the source, construction start year, regional mapping and other inputs are unchanged. Extending a shorter horizon changes $H$ and can change earlier smoothed rates, even if both horizons already extend through the UN endpoint.

Numerical stages

The first five stages run separately for each region. They use the following defaults in the implementation; these are not exposed as ordinary model configuration YAML settings.

Control Default
EWMA smoothing factor alpha 0.05
Transition from EWMA toward raw rates 50 observations
First continuation decay rate 0.05 per year
Final exponential-tail length 30 observations
Cubic-spline resampling points 9
Source-to-convergence window 50 years

1. Locate the last nonzero rate and calculate EWMA

For each region, zeros are temporarily treated as missing only to locate the last nonzero observation $L$. If there is no nonzero observation, the entire result is zero. Actual zeros before $L$ remain observations in the smoothing calculation. The code does not use separate source-provenance flags to identify the end of available data.

From the first construction observation through $L$, it evaluates series.ewm(alpha=0.05, adjust=True).mean(). For a complete sequence indexed from zero, this is

\[m_i=\frac{\sum_{j=0}^{i}0.95^{i-j}g_j} {\sum_{j=0}^{i}0.95^{i-j}}.\]

The denominator adjusts for the shorter available history near the start. This is the normalized weighted-average form, not the unadjusted recursive initialization. See the pandas EWMA definition.

2. Blend back toward the raw endpoint

Over the final $M=\min(50,\text{number of observations through }L)$ observations, linearly spaced weights run from zero to one. At position $j$ within that window, for $M>1$,

\[b_i=(1-\lambda_j)m_i+\lambda_jg_i, \qquad \lambda_j=\frac{j}{M-1}.\]

Earlier observations retain their EWMA values. The last observation equals the raw endpoint at this stage. If the window contains only one observation, NumPy supplies a single zero blend weight and the EWMA already equals that observation. Here “raw” includes the UN projection years; it does not mean that those years are historical observations.

3. Continue with five-percent annual decay

After $L$, continue the final blended rate using

\[b_{L+k}=b_L\,0.95^k.\]

This preliminary continuation preserves a positive or negative endpoint’s sign while reducing its magnitude. It is subsequently modified by both the terminal-tail treatment and the spline.

4. Replace the final 30 observations and smooth the join

The terminal-tail stage replaces the last 30 observations of the through-$H$ series with an exponential path. If the series has $N$ observations, its starting position is $d=\max(0,N-30)$. Writing the value at that position as $v$, the replacement is

\[z_{d+k}=v\exp(-\kappa k), \qquad \kappa=\frac{\log(10^6)}{30-10}.\]

For a full 30-observation tail, the value is $10^{-6}$ of $v$ after 20 steps. It is very close to zero, rather than exactly zero, at the end. If $d>0$, this stage also smooths the join: it takes the slice from $d-w$ through $d+w-1$, where $w=\min(5,d)$, and replaces that slice with a centered rolling mean of width $w$, allowing one observation at the edges. For the normal horizon this is a five-point rolling mean over ten values.

There is an implementation edge case: the decay-rate expression is written as log(v / (v * 1e-6)) / 20. If $v=0$ on a path that was not entirely zero, this becomes undefined and can make the later spline fail. The all-zero early return above does not cover every path with a zero at this particular tail position.

5. Resample through nine cubic-spline points

The spline stage first sets the last value to zero. It then:

  1. Fits a CubicSpline through the whole annual sequence.
  2. Evaluates it at nine equally spaced positions spanning the sequence, including both endpoints. These positions need not be integer years.
  3. Fits a second cubic spline through those nine values and evaluates it at every original annual position.

Both splines use SciPy’s default not-a-knot boundary condition. This is interpolation through a sparse resampling of the path, not a fitted smoothing spline with a penalty parameter. See the SciPy CubicSpline reference. Finally, the caller explicitly assigns zero at $H$ to avoid a numerical remainder. All columns after $H$ stay exactly zero.

6. Apply the common regional tail rule

After selecting the model’s projection-year columns, the code scans interior years backward, looking for the latest year at which a strict majority of regions have a local minimum in absolute smoothed growth:

\[|\tilde g_{r,t}|<|\tilde g_{r,t-1}| \quad\text{and}\quad |\tilde g_{r,t}|<|\tilde g_{r,t+1}|.\]

The required count is $\lfloor R/2\rfloor+1$ for $R$ regions. If such a year exists, every region’s growth is set to zero from the following year onward. The local-minimum year itself is retained. If no year meets that test, there is no additional truncation beyond the endpoint zeros.

This test is neither population-weighted nor based on a fixed tolerance for “small enough” growth. It can cut off growth well before $H$, and it applies to all regions even when a particular region did not have a local minimum.