*! version 1.3.1 Matthew White 21mar2014

Title

bcstats -- Compare survey and back check data,
producing a data set of comparisons

Syntax

bcstats,
surveydata(filename) bcdata(filename) id(varlist)
[options]

Main
* surveydata(filename) the survey data

* bcdata(filename) the back check data

* id(varlist) the unique ID

Comparison variables
+ t1vars(varlist) the list of
type 1 variables

+ t2vars(varlist) the list of
type 2 variables

+ t3vars(varlist) the list of
type 3 variables

Enumerator checks
enumerator(varname) display enumerators with
high error rates and variables with high error rates for those enumerators;
varname in survey data is used

backchecker(varname) display the error rates of
all back checkers; varname in back check data is used

enumteam(varname) display the overall error rates of
all enumerator teams; varname in survey data is used

bcteam(varname) display the overall error rates of
all back check teams; varname in back check data is used

showid(integer[%]) display unique IDs with
at least integer differences or at least an integer% error rate;
default is showid(30%)

showall display the error rates of all enumerators, all variables,
and all variables for all enumerators

Stability checks
ttest(varlist) run paired two-sample mean-comparison tests for
varlist in the back check and survey data using ttest

level(#) set confidence level for ttest;
default is level(95)

signrank(varlist) run
Wilcoxon matched-pairs signed-ranks tests for varlist in
the back check and survey data using signrank

Comparisons data set
keepsurvey(varlist) include varlist in the survey data in
the comparisons data set

keepbc(varlist) include varlist in the back check data in
the comparisons data set

full include all comparisons, not just differences

nolabel do not use value labels

filename(filename) save as filename;
default is filename(bc_diffs.csv) or
filename(bc_diffs.dta) if dta is specified

replace overwrite existing file

dta save data set as .dta file; default is .csv

Options
okrate(#) the acceptable error rate;
default is okrate(0.1)

okrange(varname range [, varname range ...]) do
not count a value of varname in the back check data as a difference if
it falls within range of the survey data

nodiff(# string [, # string ...]) do not count
back check responses that equal # (for numeric variables) or
string (for string variables) as differences

exclude(# string) do not compare back check responses that
equal # (for numeric variables) or
string (for string variables)

lower convert all string variables to lower case before
comparing

upper convert all string variables to upper case before
comparing

nosymbol replace symbols with spaces in string variables before
comparing

trim remove leading or trailing blanks and
multiple, consecutive internal blanks in string variables before
comparing

* surveydata(), bcdata(), and id() are
required.

* t1vars(), t2vars(), or t3vars() is required.

Description

bcstats compares back check data and survey data,
producing a data set of comparisons.
It completes enumerator checks for type 1 and type 2 variables and
stability checks for type 2 and type 3 variables.

Remarks

The GitHub repository for bcstats is
here.
Previous versions may be found there: see the tags.

Options

Comparison variables

t1vars(varlist) specifies the list of type 1 variables.
Type 1 variables are expected to stay constant between
the survey and back check, and differences may result in action against
the enumerator. Display variables with high error rates and
complete enumerator checks.
See the Innovations for Poverty Action
back check manual for
more on the three types.

t2vars(varlist) specifies the list of type 2 variables.
Type 2 variables may be difficult for enumerators to administer.
For instance, they may involve complicated skip patterns or many examples.
Differences may indicate the need for further training,
but will not result in action against the enumerator.
Display the error rates of all variables and
complete enumerator and stability checks.
See the Innovations for Poverty Action
back check manual for
more on the three types.

t3vars(varlist) specifies the list of type 3 variables.
Type 3 variables are variables whose stability between
the survey and back check is of interest.
Differences will not result in action against the enumerator.
Display the error rates of all variables and complete stability checks.
See the Innovations for Poverty Action
back check manual for
more on the three types.

Stability checks

level(#) specifies the confidence level, as a percentage, for
confidence intervals calculated by ttest.
The default is level(95) or as set by set level.

Comparisons data set

keepbc(varlist) specifies that variables in varlist in
the back check data are to be included in the comparisons data set.
Variables in varlist are renamed with the prefix bc_ in
the comparisons data set.

nolabel specifies that survey and back check responses are
not to be value-labeled in the comparisons data set.
Variables specified through keepsurvey or keepbc are
also not value-labeled.

Options

okrange(varname range [, varname range ...])
specifies that a value of varname in the back check data will not
be counted as a difference if it falls within range of the survey data.
range may be of the form [-x, y] (absolute) or
[-x%, y%] (relative).

nodiff(# string [, # string ...]) specifies that
back check responses that equal # (for numeric variables) or
string (for string variables) will not be counted as differences,
regardless of what the survey response is.

exclude(# string) specifies that
back check responses that equal # (for numeric variables) or
string (for string variables) will not be compared.
These responses will not affect error rates and
will not appear in the comparisons data set.
Used when the back check data set contains data for
multiple back check survey versions.

nosymbol replaces the following characters in string variables with
a space before comparing:
. , ! ? ' / ; : ( ) ` ~ @ # $ % ^ & * - _ + = [ ] { } | \ " < >

trim removes leading or trailing blanks and
multiple, consecutive internal blanks before comparing.
If nosymbol is specified,
this occurs after symbols are replaced with a space.

Examples

Assume that missing values were not asked in
the back check survey version.

bcstats, surveydata(bcstats_survey) bcdata(bcstats_bc) id(id) ///

okrate(0.09) okrange(gameresult [-1, 1], itemssold [-5%, 5%]) exclude(. "") ///

t1vars(gender) enumerator(enum) enumteam(enumteam) backchecker(bcer) ///

t2vars(gameresult) signrank(gameresult) ///

t3vars(itemssold) ttest(itemssold) ///

keepbc(date) keepsurvey(date) full replace

Stored results

bcstats saves the following in r():

Scalars

r(showid1) 1 if showid() displayed
unique IDs for type 1 variables and 0 otherwise

r(showid2) 1 if showid() displayed
unique IDs for type 2 variables and 0 otherwise

Matrices

r(enum1) the type 1 variable error rates of all enumerators

r(enum2) the type 2 variable error rates of all enumerators

r(backchecker1) the type 1 variable error rates of
the back checkers

r(backchecker2) the type 2 variable error rates of
the back checkers

r(enumteam1) the type 1 variable error rates of
the enumerator teams

r(enumteam2) the type 2 variable error rates of
the enumerator teams

r(bcteam1) the type 1 variable error rates of
the back checker teams

r(bcteam2) the type 2 variable error rates of
the back checker teams

r(var1) the error rates of all type 1 variables

r(var2) the error rates of all type 2 variables

r(var3) the error rates of all type 3 variables

r(ttest2) the results of ttest for type 2 variables

r(ttest3) the results of ttest for type 3 variables

r(signrank2) the results of signrank for
type 2 variables

r(signrank3) the results of signrank for
type 3 variables

References

Innovations for Poverty Action Back Check Manual

Acknowledgements

Hana Scheetz Freymiller of Innovations for Poverty Action conceived of
the three variable types and
collaborated on the structure of the program.

Author

Matthew White

For questions or suggestions, submit a
GitHub issue
or e-mail researchsupport@poverty-action.org.

Also see

Help: [R] ttest, [R] signrank

User-written: cfout