Title
libjson: a mata class library for obtaining and parsing JSON strings into > object trees
Description The libjson class (library) is designed to be called from the Mata environement to perform tasks related to obtaining a JavaScript Object Notation (JSON) formatted response from the file or website URL (via a REST API). Most users will want to write an ado file that employs the libjson object to do all the heavy lifting, with the final tailoring of the output to be handled by the ado file. The first eight methods are the workhorse methods that most programmers will use (plus perhaps the four ulitily methods). The rest are included for exceptional cases. See the code examples below for more information.
Public Methods
+-----------------------------------------------+ ---| Methods used for calling JSON-based REST APIs |----------------------- +-----------------------------------------------+
static pointer (class libjson scalar) scalar webcall(string scalar url_base, string matrix args) Calls the REST API using the given URL and Args, parses the JSON response, and returns the root node of the libjson object tree. Arguments in the form of the given N x 2 string matrix are used to construct the full url with all values properly formatted("escaped") for use in a URL.
static string matrix webcall_flatten(string scalar url_base, string matrix args) Call the web REST API, and pass the results though flatten().
+--------------------+ ---| Sub-Tree selection |-------------------------------------------------- +--------------------+
pointer (class libjson scalar) scalar getNode(string rowvector selector) Returns the node branch address by given selector.
string scalar getString(string rowvector selector, string scalar missing_val) Returns the Scalar addressed by given selector.
real scalar getReal(string rowvector selector, real scalar missing_val) Returns the real number (converted from the Scalar addressed by given selector).
static string rowvector parseSelector(string scalar selstr) Converts a selector string into the vector format by breaking up the string at the colons.
+-----------------------------------+ ---| Working with flat key-value pairs |----------------------------------- +-----------------------------------+
string matrix flattenToKV() Return the entire libjson object as a list of flattened object names and values. Useful for processing small json responses from web servers.
static string scalar getFlattenedValue(string matrix flattened_results, string scalar key, string scalar value_if_missing) Scans the given flatten() resutls for the given key, and returns the scalar (string) value. A helper function for scanning the result of flatten() for particular key-value pairs
+----------------+ ---| Utility methods|------------------------------------------------------ +----------------+
static string scalar urlencode(string scalar s) Returns the given string with certain unsafe characters escaped as necessary for use in a URL.
static string scalar getrawcontents(string scalar url_base, string matrix args) Makes an http request using the given URL and extra arguments, and returns the results as a single string.
static real rowvector getVersion() Returns a vector with the library version number, in form of (major, minor, build).
static real rowvector checkVersion(real rowvector version_to_check) True if the library version number exceeds that of version_to_check.
+----------------------------------+ ---| Working with libjson trees/nodes |------------------------------------ +----------------------------------+
real scalar isArray() true if this libjson object is holding an Array.
real scalar isObject() true if this libjson object is holding an Object.
real scalar isAttribute() true if this libjson object is holding an Attribute (name/value pair).
real scalar isString() true if this libjson object is holding a string literal, or an attribute that holds a literal.
real scalar isScalar() true if this libjson object is holding a scalar value (=string).
real scalar arrayLength() returns the number of objects in an Array.
pointer (class libjson scalar) scalar getArrayValue() returns the Nth element of the array.
string scalar getAttributeScalar(string scalar key, string scalar missing_value) returns the named attribute as a scalar (string) value
pointer (class libjson scalar) scalar getAttribute(string scalar key) returns a pointer to the named attribute contained in this Object.
string scalar getAttributeName() returns the name of an Attribute.
string rowvector listAttributeNames(real scalar one_string_flag) Returns a list of the attributes of the Object.
string scalar bracketArrayScalarValues() Returns all scalar values in the array in the familiar bracket notation.
void prettyPrint() Print to the console the libjson object tree in a human-readible form that is JSON compliant.
string scalar toString() Reconstitutes the given libjson tree into a JSON string. Note that this will not be a perfect reproduction of the original input becuase all numbers and unquoted words get converted to strings (ie. 'null','true','false', etc.).
+------------------------+ ---| Building libjson trees |---------------------------------------------- +------------------------+
pointer (class libjson scalar) scalar parse(string scalar libjson_string) Parses the given JSON-formatted string into a libjson object tree, and returns the root node.
void addArrayValue(pointer (class libjson scalar) scalar p) Adds the libjson object to the end of the array.
void addArrayScalar(string scalar s) Adds the string to the end of the array as a Scalar object.
void addAttributeScalar(string scalar key, string scalar val) Adds the key/value(string) to the end of the array as a Scalar Attribute object.
void addAttribute(string scalar key, pointer (class libjson scalar) val) Adds the libjson object to the end of the array as a Scalar Attribute object.
void makeScalar() Force the libjson object to hold the given string.
-------------------------------------------------------------------------------
Selectors Selectors are a series of named (or implicitly named in the case of arrays, which start at index "1") branches to take, starting from the given node (usually the root node).
Given the following example JSON object: { "foo" : "1", "bar": { "bar2":"2" }, "foobar": [ "bar1","bar2"] } the results of the following selectors would be... ("foo") --> "1" ("bar","bar2") --> "2"
("foobar","2") --> "bar2"
("bar") --> Depends. If a node is expected, then the nod > e is selected. If a Scalar (string, real) is expected, then is NOT FOUND and > considered missing.
Flattened Selectors A "flattened" selector is a single string with a colon inserted between each selector. For example, ("foo":"bar") --> "foo:bar".
Code Examples In the following examples, it is assumed that the source is returns a JSON object with a status result in meta:result that we need to test.
Example with libjson object tree result: pointer (class libjson scalar) scalar root root = libjson::webcall("http://server.address/service_name/",("arg1","val1 > " \ "arg2", "val2")) if (root && root->getString(("meta", "result"),"") == "OK")
Example with "flattened" response (for those not comfortable with object trees > , or for trivial JSON responses): key_val_matrix = libjson::webcall_flatten("http://server.address/service_na > me/",("arg1","val1" \ "arg2", "val2")) if (libjson::getFlattenedValue(key_val_matrix, "meta:result")) == "OK")
Example for loading a JSON object from a local file: pointer (class libjson scalar) sclar root root = libjson::webcall("/path/to/local/JSON/file",.) if (root && root->getString(("meta", "result"),"") == "OK")
Note that to use these examples interactively from the command line (or when e > mbedded in an ado file), one must wrap them in a mata function body. For exam > ple,
string rowvector getExampleDataRow(url, string rowvecto > r selectors) { pointer (class libjson scalar) scalar root root = libjson::webcall(url,"") if (root) { string rowvector res res = J(1,cols(selectors),"") for (c=1; c<= cols(selectors); c++) { > res[c] = root->getString( libjs > on::parseSelector(selectors[c]) ,"") } return(res) } else return(J(1,0,"")); } data=getExampleDataRow("http://server.address/service_n > ame/",("meta:result","flattened_selector1","flattened_selector2")) will extract the selected scalars values from a valid JSON response and place > them in a string rowvector.
Latest Version The latest version is always kept on the SSC website. To install the latest version click on the following link
ssc install libjson, replace.
Recompiling If you get an error about the version of the .mlib file not being compatible (and you are running Stata 11 or newer), the library can be recompiled locally with:
do "libjson_source.mata"
However, you will likely then need to manually copy the new library over the old one.
Author
Erik Lindsley, Ph.D. ( ssc@holocron.org )