*! version 1.0.0 14feb2019

Title

chartabb -- Table of character byte code frequency counts

Syntax

chartabb [varlist] [if] [in] [, options]

files(filenames) list of file names (or URLs) to process

scalars(scalar:string_scalars) list of string scalars to process

literals(string_literals) list of string literals to process

Description

The chartabb command
(requires Stata version 10 or higher)
produces a table of character byte code frequency counts.
The companion chartab command
(requires Stata version 14 or higher)
produces a table of Unicode character frequency counts.

chartabb can process any combination of
string variables,
files,
string scalars, and
string literals in a single run.
The table produced reports the frequency count of byte codes
from all sources combined.

In all Stata versions up to 13,
a character is encoded using a single byte.
This allows for 256 distinct values.
The first 128 byte codes (values 0 to 127) are ASCII codes.
There is no standard for the remaining byte codes (values 128 to 255).

You may still use chartabb with Stata 14 or higher.
You will get
byte code frequencies instead of Unicode character frequencies and
the glyph for byte codes (128 to 255) will be that of the Unicode
Replacement Character as these are invalid UTF-8 sequences.

chartabb produces a table that includes,
for each unique byte code found in the source,
the decimal value of the byte code (which can be used
with Stata's char() function),
its hexadecimal string representation,
the glyph (the visual representation of the character),
and the frequency count.

Options

Options

files(filenames) specifies a list of files to process.
You can also specify a URL instead of a filename.

scalars(scalar:string_scalars) specifies a list of string
scalar names to process.

literals(string_literals) specifies a list of string
literals to process.

Examples

You can get byte code frequencies for one or more string variables, and you can
even use the standard [if] [in] qualifiers:

example do-file content

version 10

sysuse auto, clear
gen upmake = upper(make)
list make upmake if strpos(make,"Toyota")
chartabb make upmake if strpos(make,"Toyota")

(click to run)

chartabb can also process text from string scalars and string literals:

example do-file content

version 10

chartabb, literal("123123")

scalar s = char(9) + "abc" + char(200) + char(126) + char(127) + char(128)
chartabb, scalar(s)
ret list

(click to run)

chartabb can also process bytes from files.
In the following example, we store in a temporary tab-delimited file the
full content of the auto dataset:

example do-file content

version 10

sysuse auto, clear

tempfile f
outsheet using "`f'"

clear
chartabb, file("`f'")

(click to run)

Stored results

chartabb stores the following in r():

Scalars

r(Nprintable) number of printable ASCII characters (32-126)

r(Ncontrol) number of ASCII control characters (0-31,127)

r(Nextended) number of extended ASCII characters (128-255)

r(Ntotal) number of bytes in total (0-255)

and for byte codes found in the source:

Scalars

r(c0) number of char(0)

r(c1) number of char(1)

r(c2) number of char(2)

... ...

r(255) number of char(255)

macros

r(control) list of ASCII control codes observed

r(printable) list of printable ASCII codes observed

r(extended) list of extended ASCII codes observed

Author

Robert Picard

robertpicard@gmail.com

Also see

Help: [D] hexdump, chartab

SSC : listsome, charlist