*! version 2.0.1 10october2023

chimchar -- Thoroughly clean string variables

Syntax

chimchar [varlist] , {numokay|numremove|numonly} [dpswitch]

chimchar_options

* numokay keeps letters and numeric characters

* numremove keeps only letters

* numonly keeps only numeric characters

dpswitch switches commas to periods and periods to commas before running the rest of the command

* Either numokay, numremove, or numonly is required.

Description

chimchar stands for CHanging IMpractical CHARacters. It removes characters from string variables that can impede functions like destring and reclink. You can choose to remove only special characters, remove both special characters and numeric characters, or remove all non-numeric characters altogether. If letters are not removed, special characters that are based on Latin letters are replaced with their closest ASCII (plain Latin) letter counterpart(s); for example, "Æ", "š" and "ĸ" would become "ae", "s", and "k", respectively.

If you do not specify varlist, the command will run over all variables in the currently loaded dataset.

Options

Exactly one of numokay, numremove, or numonly must be specified. If more than one or none of these are specified, the command returns an error.

numokay removes only special characters from the string variable(s) in varlist. It changes Latin letter-based characters to their simplest Latin equivalent, turning "Æ", "Ž" and "ĸ" into "ae", "z" and "k", respectively, while also removing special characters like quotes, parentheses, commas, and asterisks. It does not remove any numbers, the decimal point, or the minus sign. Therefore, using numokay would turn the string "-2šec..(8)" into "2sec..8". This could be useful for cleaning variables like usernames or addresses that could contain mixtures of numbers and letters.

numremove removes both special characters and numeric characters from the string variable(s) in varlist. It changes Latin letter-based characters to their simplest Latin equivalent, turning "Æ", "Ž" and "ĸ" into "ae", "z" and "k", respectively, while also removing special characters (like quotes, parentheses, commas, and asterisks), numbers, the decimal point, and the minus sign. Therefore, using numremove would turn the string "-2šec..(8)" into "sec". This could be useful for cleaning variables like people's names that should only contain letters.

numonly removes both special characters and letters from the string variable(s) in varlist. It removes all Latin letter-based characters and all special characters (like quotes, parentheses, commas, and asterisks), leaving only numbers, a single decimal point, and a single minus sign. Therefore, using numremove would turn the string "-2šec..(8)" into "-2.8". This could be useful for preparing variables that should be numeric for a future destring.

dpswitch changes all commas to decimal points and all decimal points to commas prior to running the rest of the command. It should be used if your dataset contains string variables that should be numeric that use the comma as a decimal marker.

Remarks

As of right now, I don't have the programming skills to write an ignore option that would let you specify specific characters to keep. If you'd like to collaborate on that as an addition to this command, email me at labhours@tmorg.org and we'll get in contact!