*! version 13.0 31dec2020

Title

scale_transformation - Finds a 6th-degree monotonic polynomial transformation
for a test score scale, which optimizes a specific objective function (see Description section for details).
The optimization object includes scores at two points of time (e.g. year 1 and 2) and two comparison groups (e.g. male vs female).
This program uses the Stata mata function [M-5] optimize to perform a grid search from multiple random initial
parameters to find the desired values. The main use of this program is to estimate the maximum and minimum gap difference for an (ordinal) test score scale
between two groups across two points of time. Other optimization objects (i.e. correlation, R-squared and controls) are available to run robustness checks.

Syntax

scale_transformation
, options

Main (Required)

type(integer) indicates the type of optimization (i.e. max or min)
to be performed as well as the objective function (i.e. Gap Growth,
Correlation, R-squared, or Controls Explanation - see details below).

score1(varname) numeric variable that contains test scores at (initial) period 1.

score2(varname) numeric variable that contains test scores at (final) period 2.

Secondary (Optional)

compgroup(varname) numeric variable that contains a group classification; required if type
is 1, 2 or 7 only.

controls(varlist) include varlist as controls to the baseline model at BOTH periods 1 and 2.

controls1(varlist) include varlist as controls at period 1 ONLY (i.e. period-specific controls).

controls2(varlist) include varlist as controls at period 2 ONLY (i.e. period-specific controls).

weights(varname) uses varname as inverse probability weights.

iterations(integer) specifies number of times that the program will find optimal values
using unique random-generated initial parameters;
default is iterations(1000).

maxoptiterations(integer) specifies maximum number of iterations before
each optimization stops and return results as if convergence was achieved;
default is maxoptiterations(25); see [M-5] mf_optimize##i_maxiter.

singhmethod(integer) specifies what the optimizer should do when, at an iteration
step, it finds that H is singular; default is singhmethod(1) for "hybrid";
see [M-5] mf_optimize##i_singularH.

bounddown(integer) specifies lower bound for the random-generated initial
parameters for the optimization; default is bounddown(-1500).

boundup(integer) specifies upper bound for the random-generated initial
parameters for the optimization; default is boundup(1500).

monotonicity(integer) specifies the type of monotonicity check to be
applied; default is monotonicity(1) for "Standard" - see below for details.

monofile(filename) specifies the file to be used when monotonicity
type is set to "External"; should only be used with monotonicity(3).

timeroff turn off the timer; default is set to include timer.

save(filename) save optimization results using filename once program is done.

seed(integer) set seed for replication.

robust(integer) activates option to use first K iterations (in addition to
N specified by iterations) to check for a possible change of sign in Max/Min Gap Growth optimizations.

Description

scale_transformation finds a 6th degree polynomial monotonic transformation for a test score scale
that optimizes one of the following:

a) Gap Growth: gap difference between the bottom and top groups from the compgroup variable in period 2, minus the same gap difference in period 1. In other words,
it is the difference between the regression coefficient of the top compgroup indicator (controlling for groups in the middle) in period 2, MINUS the same regression coefficient
of the top compgroup indicator (also controlling for other groups in the middle) in period 1. In both, the coefficient of the top compgroup indicator measures the difference between the top and bottom groups.
Note that the Gap Growth is positive when the Gap widens and negative when the Gap shrinks. Thus, the Gap Growth will always be negative when the coefficient changes
signs between period 1 and 2 (regardless of whether it is from + to -, or from - to +). Conversely, it will be positive when the gap widens, for instance, from -.1 to -.3;

b) Correlation: coefficient from regressing test scores from period 1 on test scores from period 2 (no controls allowed);

c) R-squared: from regressing test scores from period 1 on test scores from period 2 (no controls allowed); or

d) Controls Explanation: measures how much Gap Growth coefficients change when including controls in
controls1(varlist) and controls2(varlist) - NOT in controls(varlist). This allows to test the explanatory
power of only a subset of desired control variables. That means that, if you want to measure how much Gap Growth coefficients change by
introducing time-invariant controls, you must include these variables in both controls1(varlist) and controls2(varlist).
The Controls Explanation term is estimated by taking the distance (i.e. absolute value of the difference) between the Gap Growth calculated without variables included in
controls1 and constrols2, minus the Gap Growth calculated with these two controls varlists. To make the interpretation easier, both Gap Growths coefficients - with and without
controls - are included in the results.

The scale_transformation command is designed to verify the robustness of your results by finding plausible
test score scale transformations that maximize or minimize one of the above using a dataset in wide-format (i.e. one observation per row).
All the transformations found by this command are theoretically possible given the ordinal nature of test scores. Thus, the command helps to
verify that any results you obtained by comparing test scores using panel data (and potentially across different groups)
are robust to any arbitrary choice of scale by finding bounds for your estimations.

The methodology was originally developed in the Bond and Lang (2013) RESTAT paper, which also explains more in detail
the challenges posed by the ordinal nature of test scores and choice of scale. The current program builds on this
methodology, adjusts coefficient signs to allow for more flexibility, includes one more coefficient in the calculations and makes the optimization process more efficient,
precise and accessible.

A 6th-degree polynomial monotonic transformation is used because it provides flexibility to approach numerous continuous functions.
Nevertheless, a monotonicity checks needs to be applied to these transformations. These two characteristics yield a very complex optimization
problem that does not have a closed form solution. Furthermore, the monotonicity restriction introduces a discontinuity into the objective
function that creates multiple local maxima and minima. Considering all the above, this program takes advantage of the mata [M-5] optimize function and
performs a grid search to find multiple solutions from several different initial parameters. The results are reported in a dataset format that
includes the values of the evaluated objective functions, the resulting parameters of each optimization iteration and the initial parameters
that yielded that result.

Options

Main (Required)

type(integer) indicates the type of optimization and the objective function to be performed. The options are as follows:

1. Gap Growth Maximization
2. Gap Growth Minimization
3. Correlation Maximization
4. Correlation Minimization
5. R-squared Maximization
6. R-squared Minimization
7. Controls Explanation Maximization

score1(varname) specifies the variable that contains scores at (initial) period 1. In order to
improve processing speed, this variable is automatically scaled to be between 0 and 1 prior to running the program.
The transformation used for this process is the same for variables score1 and score2, and does not affect
the resulting estimations in any way.

score2(varname) specifies the variable that contains scores at (final) period 2. In order to
improve processing speed, this variable is automatically scaled to be between 0 and 1 prior to running the program.
The transformation used for this process is the same for variables score1 and score2, and does not affect
the resulting estimations in any way.

Secondary (Optional)

compgroup(varname) specifies the variable that contains the group classification to be analyzed
and from which the program will compare the top and bottom groups. This variable
needs to be numeric. The program will take the lowest and highest values (i.e. groups) and compare them,
while controlling for all other groups in the middle at the same time. Note that the variable compgroup
should only take on integers greater or equal to 0 (all missings will be ignored). Only allowed
when optimizing Gap Growth or Controls Explanation (i.e. type 1, 2 or 7).

controls(varlist) include varlist as controls at BOTH period 1 and 2. Variables should be numeric as they
will be used directly in regressions. Only allowed when optimizing Gap Growth or Controls Explanation (i.e. type 1, 2 or 7).

controls1(varlist) include varlist as controls at period 1 ONLY. Variables should be numeric as they
will be used directly in regressions. Only allowed when optimizing Gap Growth or Controls Explanation (i.e. type 1, 2 or 7).

controls2(varlist) include varlist as controls at period 2 ONLY. Variables should be numeric as they
will be used directly in regressions. Only allowed when optimizing Gap Growth or Controls Explanation (i.e. type 1, 2 or 7).

weights(varname) uses varname as inverse probability weights. If not specified, weights are set to 1.

iterations(integer) specifies number of times that the program will find optimal values using unique
random-generated initial parameters. The default is set to iterations(1000). The more iterations,
the longer it will take the program to run and finish. Note that by default, if the program is stopped before
ending, the results will not be stored. It is recommended that you test your program with a low number of iterations.

maxoptiterations(integer) specifies maximum number of iterations before the optimization program stops
and return results as if convergence was achieved. The default is set to maxoptiterations(25), which in
practice should be enough to achieve 'convergence' with a fair degree of accuracy. This limit is necessary as
the monotonicity restriction and complexity of the objective functions cause the optimization program to run indefinitely otherwise.
See [M-5] mf_optimize##i_maxiter for more details.

singhmethod(integer) specifies what the optimizer should do when, at an iteration step, it finds that H is singular.
The default is set to singhmethod(1) for "hybrid" but can be changed to 2 for "modified Marquardt algorithm".
See [M-5] mf_optimize##i_singularH for more details.

bounddown(integer) specifies lower bound for the random-generated initial parameters for the optimization. This program
creates random initial values and then uses them to find all the different local maxima or minima, accordingly. The default is
set to bounddown(-1500).

boundup(integer) specifies upper bound for the random-generated initial parameters for the optimization. This program
creates random initial values and then uses them to find all the different local maxima or minima accordingly. The default is
set to boundup(1500).

monotonicity(integer) specifies the type of monotonicity check to be applied. The default is set to monotonicity(1)
for "Standard", which checks for monotonicity at every possible score value (up to 4 decimals within the plausible range) in the original scale that was
transformed to be between 0 and 1. When less precision is needed, monotonicity can be set to 2 for "Sample", which only checks for monotonicity at the unique
scores values present in the sample data. If the test scores come from a larger dataset that have scores for multiple years, it is possible to do a more
thorough check (but more time consuming) by putting together an external file with all theoretical or observed plausible scores where you want the
program to check for monotonicity. For the latter, you must set monotonicity to 3 for "External" and include the filename in monofile(filename). The
plausible test scores in the "External" file should be in the original scale since the program will automatically transform them to the new scale and check
for monotonicity at each of these transformed scores.

monofile(filename) specifies the file to be used when monotonicity type is set to 3 for "External". This
option should only be used with monotonicity(3). You can use .dta, .csv, .xls or .xlsx files. Make sure that .csv,
.xls and .xlsx files have the name of the variable on the first row.

timeroff turn off the timer. The default is set to include the time (in minutes) that the program took to run, which
is reported only if the program finishes.

save(filename) save optimization results using filename once program is done.

seed(integer) set seed for replication, where 0 <= integer <= 2,147,483,647. If missing, the seed is randomly generated and included in the header of the program.
Always save the log file to ensure you are able to recover the seed number for replication.

robust(integer) Since the optimization problem for the Max/Min Gap Growth options (i.e. type 1 or 2) is complex and does not have a closed-solution,
in some specific cases the maximization could yield the solutions for the minimization problem (and vice versa). In order to address this issue, the user could
activate the robust option to use the first K=integer (where 20<=K<=300; K=even) iterations (in addition to N specified by the iterations option)
to check for a possible change of sign in Max/Min Gap Growth optimization results. Half of the K iterations are used to find the maximization solution while
the other half K iterations are used for the minimization problem. Once the correct approach is selected, the program keeps the solutions for the correct specification
and runs the remaining N iterations indicated in the iterations() option.

References

Bond, T. N., & Lang, K. (2013). The evolution of the Black-White test score gap in grades K–3: The fragility of results. Review of Economics and Statistics, 95, 1468–1479. doi:10.1162/REST_a_00370

Examples

Gap Growth maximization between scores in year 1 and 2 by comparing males vs females:

. sysuse timss_testscores.dta, clear

. scale_transformation, type(1) score1(score1) score2(score2) ///

compgroup(sex) iterations(20) maxoptiterations(15) mono(2) ///

seed(562) robust(20)

Note that, given the above command, the program will run 40 times from different initial parameters and each time, the program will
report convergence after 15 iterations, performing a "Sample" monotonicity check. Given the option robust(20), the program will
use the first 20 simulations to check that the sign of the gap is correct by carrying out 10 maximizations and 10 minimizations.
Once the correct direction is confirmed, the correct half is kept and the other discarded. Thus, in this example, the resulting dataset
will contain only 30 observations.

Author

Andres Yi Chang

andresyichang@gmail.com

Version

Last Updated on: December 31, 2020

This commnad was build with Stata Version 13