*! version 1.0.1 20feb2024

Title

tmpinvi -- an iterated (multistep) Transaction Matrix (TM)-specific
implementation of the LPLS estimator

Syntax

tmpinvi
varlist|matname (rowsums first, missing skipped)
[if] [in]
[, options]

Constructing the LHS and RHS
slackvars(varlist|matname) the SLACK/SURPLUS VARIABLES part of
`a`, two columns, rowsums first

values(varname|colvector) the KNOWN VALUES part of `b`
the length of which should be equal to the one of the TM, missing
values included, colshape(TM, 1) = `v`

zerodiagonal set all diagonal elements of the TM to 0 in `a`

adjustment(strings:string) adjustment for extreme values to
match the RHS via shares of row/column sums, "row", "col",
or "ave" (mean of both), not specified ≠ no adjustment

SVD-based estimation
submatrix(#) maximum size of each contiguous submatrix, set
subm≤2 for (over)determination, OLS estimation, and an F-test from
linear regression (default, slow: higher quality of
individual estimates but potentially lower overall quality),
or a greater number (maximum is 50, faster: lower quality
of individual estimates but potentially higher overall
quality) for underdetermination, minimum-norm least-squares
generalized solution, and a t-test of mean NRMSE, based on a Monte
Carlo-simulated distribution

tolerance(real) roundoff error,
a number to determine when a number is small enough to be considered
zero (optional, not specifying tol is equivalent to specifying
tol=0)

level(#) confidence level (by default: c(level))

Monte-Carlo-based t-test
trace display regression/t-test output for each contiguous
submatrix

distribution display nine main percentiles of the Monte
Carlo pre-simulated distribution

Compensatory operations
iterate(#) number of iterations, set iter=0 to disable
completely, iter=1 to choose the first improvement in up to
c(maxiter) iterations (default), or a greater number to
minimize the NRMSE
(compensatory operations require non-empty values())

Multistep estimation
lowerbound(real) lower bound for each element in the solution,
each value lower than which is replaced by a missing one (.)

upperbound(real) upper bound for each element in the solution,
each value higher than which is replaced by a missing one (.)

round(real) y in round(), in the units of which the
solution is rounded before comparing its elements with lowerbound()
and/or upperbound() (by default: 8e-307)

penalization add the number of missing values in rows and columns
of the solution as slackvars() in the model to compensate for the
use of lowerbound() and/or upperbound()

stepnumber(#) number of steps in addition to the initial
estimation (by default: 2)

by, collect, fp, rolling, statsby, and xi
are allowed; see prefix.

weights are not allowed; see weights.

Description

The program implements the LPLS (linear programming through least squares)
estimator for Transaction Matrices (TM) with the help of the Moore-Penrose
inverse (pseudoinverse), calculated using singular value decomposition
(SVD). The pseudoinverse offers a unique minimum-norm least-squares solution,
which is the best linear unbiased estimator (BLUE); see Albert (1972, Chapter
VI). The estimation using 2x2 (by default) to 50x50 contiguous
submatrices, repeated with compensatory slack/surplus variables until NRMSE is
minimized in a given number of iterations (if values() are defined), is
followed by an F-test from linear regression/t-test of mean NRMSE from a
pre-simulated distribution (Monte-Carlo, 50,000 iterations with matrices
consisting of normal random variates, estimated with increased precision,
tol=c(epsdouble)). The result is adjusted for extreme values to match
`b` with the help of shares of estimated row sums/column sums/mean of both
if adjustment() is specified.

tmpinv is a sister program to lppinv 1) focusing on a single type
of LP problems (TM), 2) dividing the TM into contiguous submatrices
with the size of up to (49 + sum of the rest)x(49 + sum of the rest), 3)
being based on regress results (F-test) for subm(≤2) or on a
pre-simulated Monte Carlo distribution and ttesti for subm(>2), 4)
performing eventual "compensatory operations" by adding a slack/surplus variable
equal to residuals of KNOWN VALUES/their estimates from the previous step to
`a`, attempting to minimize NRMSE, and 5) adjusting the result to match
CONSTRAINTS in `b` (if enabled).

NB The rule of thumb is to use as many iterations in iter(#) as
possible since more iterations = lower NRMSE.

The ttesti tests the mean NRMSE against a no-values() 50,000
sample, which yielded the highest errors; ergo, poor test results indicate a
grossly misspecified model. Use the distribution option to compare
the NRMSE for each submatrix with the main percentiles of the sample (they are
sometimes easier to interprete than the t-test).

tmpinvi is an iterated (multistep) version of tmpinv. Each further
step, the number of which is controlled in stepnumber(), uses the solution
obtained in the previous step, adjusted to eventual lowerbound() and
upperbound(), as values() in the model with/without penalization
of the number of missing values in rows and columns as slackvars(),
maximizing the R-squared for CONSTRAINTS. If R-squared for CONSTRAINTS is
out of bounds (below 0 or above 1), the command reports
non-convergence.

What is a TM?

Transaction Matrix (TM) of size (M x N) is a formal model of
interaction (allocation, assignment, etc.) between M and N elements
in any imaginable system, such as intercompany transactions (netting tables),
industries within/between economies (input-output tables), cross-border
trade/investment (trade/investment matrices), etc., where row and
column sums are known, but individual elements of the TM may not be:

• a netting table is a type of TM
where M = N and the elements are subsidiaries of a MNC;

• an input-output table (IOT) is a type of TM
where M = N and the elements are industries;

• a matrix of trade/investment is a type of TM
where M = N and the elements are countries or (macro)regions, where
diagonal elements may be equal to zero;

• a country-product matrix is a type of TM where
M ≠ N and the elements are of different types;

...

Example of a TM problem:

Estimate the matrix of trade/investment with/without zero diagonal
elements, the country shares in which are unknown. For a pre-LPLS approach
to this problem, see (Bolotov, 2015).

tmpinvi clears estimation results and returns matrix r(solution),
matrix r(tests), scalar r(r2_c) (R-squared for CONSTRAINTS), and
scalar r(r2_v) (R-squared for KNOWN VALUES) (if available). In addition,
matrix r(nrmse_dist) is available with the help of the command:

. return list, all

Methods and formulas

The TM problem is written as a matrix equation `a @ x = b`, loosely
based on the structure of the Simplex tableau, where `a` consists of
coefficients for CONSTRAINTS (aka the "characteristic matrix" which depends on
M and N of the TM and is automatically generated by the algorithm) and
for SLACK/SURPLUS VARIABLES (the upper part) as well as for the identity matrix
I (the lower part) as illustrated in Figure 1. SLACK/SURPLUS VARIABLES can
be omitted.

Figure 1: Matrix equation `a @ x = b`

`a`
| `b`

+–––––––––––––––––––––––––––––––––––––––+–––––––––––––––––+–––––––––––––+

| CONSTRAINTS OF THE TRANSACTION MATRIX | SL/SU VARIABLES | CONSTRAINTS |

+–––––––––––––––––––––––––––––––––––––––+–––––––––––––––––+–––––––––––––+

| I
| COMPENSATORY S. | KNOWN V. |

+–––––––––––––––––––––––––––––––––––––––––––––––––––––––––+–––––––––––––+

Source: self-prepared

The solution of the equation, `x = pinv(a) @ b`, is estimated with the
help of SVD and is a minimum-norm least-squares
generalized solution if the rank of `a` is not full. To check if
`a` is within computational limits, its (maximum) dimensions can be
calculated using the formulas:

• (M + N) x (M * N) TM
without slack/surplus variables and known values;

• (M + N + M * N) x (M * N) TM without
slack/surplus variables but with known values;

• (M + N) x (M * N + 1) TM
with slack/surplus variables but without known values;

• (M + N + M * N) x (M * N + 1) TM with
slack/surplus variables and known values.

where M and N are the dimensions of the transaction matrix.

Examples

TM problem with F-test from linear regression:
. clear
. set obs 10
. gen str13 country_id = ""
. mata: st_local("b", "Country #"); st_local("e", "Rest of World")
. mata: st_sstore(., "country_id", ("`b'":+strofreal(1::9)\"`e'"))
. gen float exports = runiform(0, 1000)
. gen float imports = runiform(0, 1000)
. list
. tmpinvi exports imports, zerod adj(ave) l(0) u(1000)
. matlist r(solution)
. return list

TM problem with Monte Carlo t-test based on the uniform distribution:
...
. tmpinvi exports imports, zerod adj(ave) subm(10) l(0) u(1000)
. matlist r(solution)
. return list

Author

Ilya Bolotov

Prague University of Economics and Business

Prague, Czech Republic

ilya.bolotov@vse.cz

Thanks for citing this software and my works on the topic:

Bolotov, I. (2024). 'TMPINVI': module providing an iterated (multistep)
Transaction Matrix (TM)-specificimplementation of the LPLS estimator. Available
from https://ideas.repec.org/c/boc/bocode/s459294.html.

References

Albert, A., 1972. Regression And The Moore-Penrose Pseudoinverse. New
York: Academic Press.

Bolotov, I. 2015. Modeling Bilateral Flows in Economics by Means of Exact
Mathematical Methods. [Paper presentation]. The 9th International Days of
Statistics and Economics: Prague.
https://msed.vse.cz/msed_2015/article/111-Bolotov-Ilya-paper.pdf

PS Please also check the Web of Science (WoS) for new research on LPLS and
TM in particular.