*! enforce v1.0 Thomas Blanchet 20apr2020

Title

enforce -- Force variables to satisfy a set of accounting identities

Syntax

enforce (identity) [(identity) ...], {replace|suffix(string)|prefix(string)}
[options]

where identity is:

[+|-] {varname|0} [{+|-}{varname|0} ...] = [+|-] {varname|0} [{+|-}{varname|0} ...]

* replace replace existing variables

* suffix(string) create new variables using existing variables names and the suffix string

* prefix(string) create new variables using existing variables names and the prefix string

fixedvars(varlist) variables to keep unchanged in the adjustment

nocheck do not check the feasibility of the adjustment

nofillmissing do not attempt to use accounting identities to fill in missing values

noenforce do not adjust the value of variables

diagnostic generate a variable indicating problematic observations

force attempt to do the adjustment even if it is not feasible

tolerance(real) numerical tolerance parameter

zero(real) absolute tolerance to edit results to zero

* Either replace, suffix(string) or prefix(string) is required.

Description

enforce forces variables to satisfy an arbitrary set of accounting identities.
It also checks that the system of accounting identities is feasible and plausible, and
uses the identities to fill in missing values when possible.

Options

replace replaces original variables in the identities by the new, adjusted ones.

suffix(string) creates new variables containing adjusted values using the name of the
original variables with the suffix string.

prefix(string) creates new variables containing adjusted values using the name of the
original variables with the prefix string.

fixedvars(varlist) specifies a set of variables whose value should not be changed.
Fixed variables can make the system of identities unsolvable, which the command will check.

nocheck asks not to check the feasibility or plausibility of the system of identities.

nofillmissing asks not to try to fill in missing values using the identities.
Note that if you use that option, the final adjustment might be incomplete, i.e.there might
still be ways to arrange identities so that they contradict the results.

noenforce asks not to perform the adjustment to the variables.

force asks to perform the adjustment even if the system is found to be infeasible: see Details.

tolerance(real) is a numerical tolerance parameter (amount by which the default tolerance is multiplied).

zero(real) results below this value are edited to zero.
Default is 1e-7.

Details

Note: Stata does not allow good typesetting of math equation in help files. Visit the commands' GitHub page for a better rendering of the text below.

This Stata command is designed to enforce an arbitrary set of accounting identities intelligently, while performing a series of auxiliary checks and adjustments.
To understand the command, first assume an identity a=b+c.
The simplest, naïve way of enforcing it is to multiply both b and c by the same constant, a/(b+c).
The main problem with that approach is that it only works well with positive variables.
If a=0, it will set both b and c to zero, instead of enforcing the more general condition b=-c.
If b+c=0, then it will not work at all, and more generally if b+c≪a (because b<0 or c<0), it can easily lead to absurd adjustments.

One way to fix that first issue is to calculate the discrepancy ε=a-b-c, and redistribute it proportionally to the absolute value of b and c.
That is, we redefine b as b+λε and c as c+(1-λ)ε, where λ=|b|/(|b|+|c|).
If b and c are both positive, this is equivalent to the naïve approach, but otherwise it behaves much more reasonably.

But there are still other problems.
First, if forces us to define a reference variable (in this case a) that will remain unchanged, which may or may not be desirable.
Second, it is not clear how to generalize this adjustment to more complex settings.
In practice, we must simultaneously satisfy dozens of accounting identities, with variables presents in several of them, so that adjustments must be performed across several dimensions.
We formalize that problem as follows.
Assume that we have a vector X=(x_1,…,x_n)' of variables to be adjusted, and we seek an adjusted vector Y=(y_1,…,y_n)' that must satisfy a set of accounting identities.
We will minimize the sum of (y_i - x_i)^2/|x_i| for i from 1 to n, subject to the accounting identities.
The convex cost function (y_i - x_i)^2 at the numerator ensure that the differences between raw and adjusted variables are as low as possible, and that they are spread equitably across all the variables.
The |x_i| at the denominator ensures that adjustments are proportional to the initial value of the variable.
In simple cases, this is equivalent to the procedure explained above.

Assume that the set of accounting identities can be written as a linear system AX=B.
The problem can be written in matrix form as:

minimize 1/2 X'QX + C'X subject to AX=B

where Q=diag(1/|x_1|,…,1/|x_n|) and C=(sign(x_1),…,sign(x_n))'.
This is a standard, quadratic programming problem with equality constraints and a positive definite matrix Q.
(Variables that are set fixed or equal to zero are removed from the vector X and included in B.)
The result can be written as the solution of a standard linear system (see Wikipedia for details).

This problem is solved via QR decomposition, so it will return an optimal solution in the least-squares sense if the system of identities is technically infeasible.
The command checks wether this is the case, and stops by default if the system of equalities is found to be infeasible for some observations.
You can override this behavior with the force option.
Note that variables initially equal to zero are implicitly fixed, so sometimes they can be the reason behind infeasibility.

This is the main task performed by the command enforce, although it also has several additional functionalities.
First, it takes advantage of accounting identities to fill in any missing value that can technically be calculated from nonmissing variables, even though it was initially absent from the raw data.

Second, it pays specific attention to the way constraints are defined to overcome missing value problems.
Indeed, assume for example that we have the constraints a=b+c, α=a+x, β=b+y, γ=c+z, and x=y+z.
Clearly, this implies that α=β+γ.
However, if the data were to only contain nonmissing values for α, β and γ, a naïve treatment of missing values would lead us to dismiss all the constraints as irrelevant to our data.
The command is designed to be aware of the fact that the system of identities implicitly imposes α=β+γ.

Third, the command analyzes the system of identities to find any implausibility (e.g. variable always equal to zero) or incompatibility with the data (in case of fixed variables).

Fourth, it provides extensive reporting on the magnitude of the discrepancies and the adjustments.

Example

set obs 100
drawnorm a1 b1 c1 a2 b2 c2 x y z
foreach v in a1 b1 c1 x y z {
// Create missing values at random
replace `v' = . if (uniform() < 0.1)
}
enforce (a1 = b1 + c1) (a2 = a1 + x) (b2 = b1 + y) (c2 = c1 + z) (x = y + z), fixed(a2) replace

Contact

If you have comments, suggestions, or experience any problem with this command, please contact <thomas.blanchet@wid.world>.