*! version 1.1.2 20feb2024

Title

tmpinv -- a non-iterated Transaction Matrix (TM)-specific
implementation of the LPLS estimator

Syntax

tmpinv
varlist|matname (rowsums first, missing skipped)
[if] [in]
[, options]

Constructing the LHS and RHS
slackvars(varlist|matname) the SLACK/SURPLUS VARIABLES part of
`a`, two columns, rowsums first

values(varname|colvector) the KNOWN VALUES part of `b`
the length of which should be equal to the one of the TM, missing
values included, colshape(TM, 1) = `v`

zerodiagonal set all diagonal elements of the TM to 0 in `a`

adjustment(strings:string) adjustment for extreme values to
match the RHS via shares of row/column sums, "row", "col",
or "ave" (mean of both), not specified ≠ no adjustment

SVD-based estimation
submatrix(#) maximum size of each contiguous submatrix, set
subm≤2 for (over)determination, OLS estimation, and an F-test from
linear regression (default, slow: higher quality of
individual estimates but potentially lower overall quality),
or a greater number (maximum is 50, faster: lower quality
of individual estimates but potentially higher overall
quality) for underdetermination, minimum-norm least-squares
generalized solution, and a t-test of mean NRMSE, based on a Monte
Carlo-simulated distribution

tolerance(real) roundoff error,
a number to determine when a number is small enough to be considered
zero (optional, not specifying tol is equivalent to specifying
tol=0)

level(#) confidence level (by default: c(level))

Monte-Carlo-based t-test
trace display regression/t-test output for each contiguous
submatrix

distribution display nine main percentiles of the Monte
Carlo pre-simulated distribution

Compensatory operations
iterate(#) number of iterations, set iter=0 to disable
completely, iter=1 to choose the first improvement in up to
c(maxiter) iterations (default), or a greater number to
minimize the NRMSE
(compensatory operations require non-empty values())

by, collect, fp, rolling, statsby, and xi
are allowed; see prefix.

weights are not allowed; see weights.

Description

The program implements the LPLS (linear programming through least squares)
estimator for Transaction Matrices (TM) with the help of the Moore-Penrose
inverse (pseudoinverse), calculated using singular value decomposition
(SVD). The pseudoinverse offers a unique minimum-norm least-squares solution,
which is the best linear unbiased estimator (BLUE); see Albert (1972, Chapter
VI). The estimation using 2x2 (by default) to 50x50 contiguous
submatrices, repeated with compensatory slack/surplus variables until NRMSE is
minimized in a given number of iterations (if values() are defined), is
followed by an F-test from linear regression/t-test of mean NRMSE from a
pre-simulated distribution (Monte-Carlo, 50,000 iterations with matrices
consisting of normal random variates, estimated with increased precision,
tol=c(epsdouble)). The result is adjusted for extreme values to match
`b` with the help of shares of estimated row sums/column sums/mean of both
if adjustment() is specified.

tmpinv is a sister program to lppinv 1) focusing on a single type
of LP problems (TM), 2) dividing the TM into contiguous submatrices
with the size of up to (49 + sum of the rest)x(49 + sum of the rest), 3)
being based on regress results (F-test) for subm(≤2) or on a
pre-simulated Monte Carlo distribution and ttesti for subm(>2), 4)
performing eventual "compensatory operations" by adding a slack/surplus variable
equal to residuals of KNOWN VALUES/their estimates from the previous step to
`a`, attempting to minimize NRMSE, and 5) adjusting the result to match
CONSTRAINTS in `b` (if enabled).

NB The rule of thumb is to use as many iterations in iter(#) as
possible since more iterations = lower NRMSE.

The ttesti tests the mean NRMSE against a no-values() 50,000
sample, which yielded the highest errors; ergo, poor test results indicate a
grossly misspecified model. Use the distribution option to compare
the NRMSE for each submatrix with the main percentiles of the sample (they are
sometimes easier to interprete than the t-test).

What is a TM?

Transaction Matrix (TM) of size (M x N) is a formal model of
interaction (allocation, assignment, etc.) between M and N elements
in any imaginable system, such as intercompany transactions (netting tables),
industries within/between economies (input-output tables), cross-border
trade/investment (trade/investment matrices), etc., where row and
column sums are known, but individual elements of the TM may not be:

• a netting table is a type of TM
where M = N and the elements are subsidiaries of a MNC;

• an input-output table (IOT) is a type of TM
where M = N and the elements are industries;

• a matrix of trade/investment is a type of TM
where M = N and the elements are countries or (macro)regions, where
diagonal elements may be equal to zero;

• a country-product matrix is a type of TM where
M ≠ N and the elements are of different types;

...

Example of a TM problem:

Estimate the matrix of trade/investment with/without zero diagonal
elements, the country shares in which are unknown. For a pre-LPLS approach
to this problem, see (Bolotov, 2015).

tmpinv clears estimation results and returns matrix r(solution),
matrix r(tests), scalar r(r2_c) (R-squared for CONSTRAINTS), and
scalar r(r2_v) (R-squared for KNOWN VALUES) (if available). In addition,
matrix r(nrmse_dist) is available with the help of the command:

. return list, all

Methods and formulas

The TM problem is written as a matrix equation `a @ x = b`, loosely
based on the structure of the Simplex tableau, where `a` consists of
coefficients for CONSTRAINTS (aka the "characteristic matrix" which depends on
M and N of the TM and is automatically generated by the algorithm) and
for SLACK/SURPLUS VARIABLES (the upper part) as well as for the identity matrix
I (the lower part) as illustrated in Figure 1. SLACK/SURPLUS VARIABLES can
be omitted.

Figure 1: Matrix equation `a @ x = b`

`a`
| `b`

+–––––––––––––––––––––––––––––––––––––––+–––––––––––––––––+–––––––––––––+

| CONSTRAINTS OF THE TRANSACTION MATRIX | SL/SU VARIABLES | CONSTRAINTS |

+–––––––––––––––––––––––––––––––––––––––+–––––––––––––––––+–––––––––––––+

| I
| COMPENSATORY S. | KNOWN V. |

+–––––––––––––––––––––––––––––––––––––––––––––––––––––––––+–––––––––––––+

Source: self-prepared

The solution of the equation, `x = pinv(a) @ b`, is estimated with the
help of SVD and is a minimum-norm least-squares
generalized solution if the rank of `a` is not full. To check if
`a` is within computational limits, its (maximum) dimensions can be
calculated using the formulas:

• (M + N) x (M * N) TM
without slack/surplus variables and known values;

• (M + N + M * N) x (M * N) TM without
slack/surplus variables but with known values;

• (M + N) x (M * N + 1) TM
with slack/surplus variables but without known values;

• (M + N + M * N) x (M * N + 1) TM with
slack/surplus variables and known values.

where M and N are the dimensions of the transaction matrix.

Examples

TM problem with Monte Carlo t-test based on the uniform distribution:
. clear
. set obs 30
. gen rowsum = rnormal(15, 100)
. gen colsum = rnormal(12, 196)
. tmpinv rowsum colsum, level(90)
. tmpinv rowsum colsum, zerod dist

TM problem with compensatory operations:
...
. mata: st_matrix("RHS", st_data(1::5,1..2))
. gen known = rnormal(18, 252) if _n <= 25
. tmpinv RHS in 1/25, v(known) subm(50) level(90)
. matlist r(solution)

Author

Ilya Bolotov

Prague University of Economics and Business

Prague, Czech Republic

ilya.bolotov@vse.cz

Thanks for citing this software and my works on the topic:

Bolotov, I. (2024). 'TMPINV': module providing a non-iterated Transaction
Matrix (TM)-specific implementation of the LPLS estimator. Available
from https://ideas.repec.org/c/boc/bocode/s459131.html.

References

Albert, A., 1972. Regression And The Moore-Penrose Pseudoinverse. New
York: Academic Press.

Bolotov, I. 2015. Modeling Bilateral Flows in Economics by Means of Exact
Mathematical Methods. [Paper presentation]. The 9th International Days of
Statistics and Economics: Prague.
https://msed.vse.cz/msed_2015/article/111-Bolotov-Ilya-paper.pdf

PS Please also check the Web of Science (WoS) for new research on LPLS and
TM in particular.