*! version 1.0 9 Feb 2023

Title

cgssm -- The contrasting groups' standard setting method

Syntax

cgssm for datasets in wide format

cgssm varlist(min=2 max=2 numeric fv) [if] [in] [weight]
[, options]

cgssm for datasets in long format

cgssm varlist(min=1) [if] [in],
by(varname numeric fv) [options]

Immediate version of cgssm

cgssmi, msdrowmatrix(matrixname) [options]

cgssm, required

by(varname) Name for the categorising variable when the data are
in a long format. Factor variables are allowed.
Note that the missing values for a factor variable are set to zeroes

cgssmi, required

msdrowmatrix(string) A 2 by 2 matrix of (mean, sd)-rows as a string.
First row is the reference group. It can be either as text "m1, sd1 \ m2, sd2"
or the name of a matrix

cgssm and cgssmi, optional

refname(string) Alternative text for the reference group name

compname(string) Alternative text for the comparison group name

header(string) Alternative text for legend header

cpformat(string) Format the cut-off in the graph.
The default is %6.0f

pformat(string) Format the theoretical false positives and
the theoretical true negatives in the graph.
The default is %6.1f

graph Option for drawing a graph. Not necessary if some
twoway graph options are added

* Any twoway graph option. Implies that a graph is drawn

aweights, fweights, and iweights are allowed.

Description

Establishing a competency standard for when a certain level of expertise
is reached by identifying cut-off points on different performance measures based
on rating scores or simulator metrics is an important issue in competency-based
learning.

One method to set such standards, is the contrasting groups' standard
setting method.
It is a participant-based method where the performance of a certain procedure
is evaluated between participants of different expertise levels, e.g., novices
and experts.

Using the contrasting groups' method, the cut-off point is set by
identifying the intercept of two normally distributed curves that represent the
score distributions of the groups defined by their level of expertise.
After a pass/fail score is defined, the percentage of false positives and
negatives can be calculated to explore the consequences of the test.

Traditionally, these false positives and false negatives are calculated
based on the observed number of individuals who passes or fails a test.
However, validity studies often include only a small number of participants.
These small numbers make the rate of false positives and false negatives
sensitive to outliers.

Instead, using the normally distributed curves that represent the score
distributions of the groups defined by their level of expertise, the theoretical
false negatives, and theoretical false positives can be calculated.

What is new in cgssm and cgssmi is that the cut-off is found
using an exact formula based on solving a polynomial of first or second
order degree.

Examples

Suppose that variable bwt is a score and the variable smoke is the
grouping variable.

`"webuse lbw"'

With the mothers smoking during pregnanacy as a reference

`"cgssm bwt, by(smoke) name(smkref, replace)"'

With the mothers not smoking during pregnanacy as a reference using a
factor variable

`"cgssm bwt, by(0.smoke) refname(non-smoker) compname(smoker) name(nonsmkref, replace)"'

A typed matrix as argument

`"cgssmi, msd(50, 10 \ 20, 5) graph"'

A named matrix as argument

`"matrix a = 50, 5 \ 40, 5"'

`"cgssmi, msd(a) g"'

Stored results

cgssm and cgssmi stores the following in r():

Scalars

r(fp) The theoretical false positive rate.

r(fn) The theoretical false negative rate.

r(cutoff) The calculated cut-off.

Macros

r(graph_text) If a graph is requested, the code behind the graph.

Matrices

r(meansd) A 2 by 2 of the mean's and sd's used for the
calculations. Top row is the mean and sd for the reference group.

Authors and support

Author:

Niels Henrik Bruun,

Aalborg University Hospital

Support:

niels.henrik.bruun@gmail.com

References

Jørgensen, M., Konge, L. & Subhi, Y.

Contrasting groups' standard setting for consequences analysis in validity studies: reporting considerations.

Adv Simul 3, 5 (2018). https://doi.org/10.1186/s41077-018-0064-7