*! version 0.9 18mar2015

Title

percentmatch -- Calculate the highest percentage match (near duplicates) between observations

Syntax

percentmatch
[varlist]

[, options]

Main
generate(newvar) Create variable newvar highest percent match

idvar Uniquely identifying variable in the dataset

matchedid(newvar) Create variable newvar highest match observation's idvar

Description

percentmatch calculates the highest percent match between observation across
the variables in varlist (or across all variables if varlist is not specified).
Similar to , percentmatch, compares observations to identify
identical values. The match percentage is given by the number of identical values
divided by the number of variables. percentmatch returns the highest match percentage
for each observation.

Options

Main

generate(newvar) creates newvar containing the highest match percentage.

idvar specifies the uniquely identifying id variable in the dataset. If
the variable doesn't exist in the dataset, it must be created before using
percentmatch.

matchedid(newvar) creates newvar with the corresponding highest percentage
match value for idvar for each observation. i.e. observation a's highest match was
with observation b.

Remarks

This command was developed to detect near duplicates in survey data. See Kuriakose and
Robbins 2015, Detecting Falsification in Survey Data for more details.

Examples

. sysuse nlsw88, clear

. percentmatch, generate(pmatch) idvar(idcode) matchedid(m_id)

. sysuse nlsw88, clear

. percentmatch age - wage, gen(pmatch) id(idcode) matchedid(m_id)

. sysuse bpwide, clear

. percentmatch, generate(pmatch) idvar(patient) matchedid(m_id)

Returned Results

Scalars:

r(p100) Number of observations with 100% match

r(p95) Number of observations with 95% match

r(p90) Number of observations with 90% match

r(vars) Number of variables over which match was calculated

r(N) Number of observations over which match was calculated

Macros:
r(varlist) Variables over which match was calculated

Author

Noble L. Kuriakose, SurveyMonkey, noblek@surveymonkey.com

Please cite this program by referencing the paper below:

Kuriakose, Noble and Robbins, Michael, Falsification in Surveys: Detecting Near Duplicate Observations (March 18, 2015). Available at SSRN: http://ssrn.com/abstract=2580502