[Home] [Search] [D]
Last update 8th March 2005

Phobos: std.openrj

Open-RJ is an open-source library that implements readers of the Record-Jar structured text file format, designed and implemented by Matthew Wilson, as an exemplar project for his column Positive Integration in C/C++ Users Journal.

It is implemented in C & C++, with a C-API. The implementation of the basic library is platform-independent. Mappings are provided to several languages (including C++, D, Ruby and STL), and others (COM, Java, .NET, Perl, Python) are planned. In addition to platform-independence, the library focuses on small runtime costs - memory and speed - and the classic UNIX attributes of discoverability and visibility.

What is the Record-Jar format?

As described in the excellent book "The Art Of UNIX Programming", a Record-Jar structured format file consists of records and fields.

A field is a single line - optionally extended with trailing '\' - that contains a name, separated from an optional value by ':'.

A record is a list of fields, whose contents are arbitrary and can vary between records in the same database. Records are separated by a line that begins with "%%". The record separator also acts as a comment, so anything can come on a record separator line after the first two characters.

A database is a correctly parsed Record-Jar file. The Open-RJ API (and language mappings) provide access to all the records in the database and the complete set of fields. Hence, you may work with fields on a per-record basis, or treat the database as a single record and with all fields in the database.

A very simple Record-Jar file, representing a Pets Database, is shown below:

Name:       Elsa
Species:    Dog
Breed:      Mixed
%%
Name:       Fluffy Kitten
Species:    Cat
%%
Name:       Rebel
Species:    Dog
Breed:      German \
            Shepherd
%%
Name:       Pepper
Species:    Dog
Breed:      Border Collie
%%
Name:       Samson
Species:    Dog
Breed:      Ridgeback
%%
Name:       Sheltie
Species:    Dog
Breed:      Shetland \
            Sheepdog
%%
Name:       Sparky
Species:    Cat
%%

And that's pretty much all there is to it. There are no restrictions on what fields may be in a record, and no controls over whether all records have the same fields or not. That's the job of higher layers of application functionality. We keep Record-Jar simple so it's reliable, portable and fast, and it's those things in spades!

std.openrj

The D mapping of Open-RJ is packaged with Phobos in the std.openrj module. It consists of four classes:

Database
DatabaseException
Field
InvalidKeyException
OpenRJException
Record

and one enumeration:

ORJ_FLAG

The basic usage is as follows:

Create a database instance, on an existing Open-RJ database file, e.g.

    char[]   contents = cast(char[])(std.file.read("pets.orj"));
    Database db       = new Database(contents, ORJ_FLAG.ELIDE_BLANK_RECORDS);

Enumerate over the records in the database, by using foreach:

    foreach(Record r; db)
    {
      . . . // Process the record
    }

or by indexed lookup (which returns a Record instance):

    for(int i = 0; i < db.length; ++i)
    {
      Record r = db[i];

      . . . // Process the record
    }

Processing of the record may be done by either ...

... enumerating the fields in the record, by using foreach:

    foreach(Field f; r)
    {
      char[] name  = f.name;
      char[] value = f.value;

      printf("  %.*s=%.*s\n", name, value);  // e.g. "Breed=German Shepherd"
    }

or by indexed lookup (which returns a Field instance):

    for(int i = 0; i < r.length; ++i)
    {
      Field f = r[i];

      . . . // Process the field
    }

... or it might be by field lookup or by name (which returns a string):

    printf("  The value of the \"Species\" field is %.*s\n", r["Species"]);

std.openrj.Database

The Database class has the following methods:

this(in char[] contents, in uint flags = 0): Create a database object representing the given database contents; contents The contents (or full path) of the database, where lines are separated by carriage-returns; flags A combination of ORJ_FLAG flags which affect the processing of the Jar file; Note: throws a DatabaseException if the contents do not represent a correctly formed Open-RJ database
this(in char[][] lines, in uint flags = 0): Create a database object representing the given database contents; lines The database contents as an array of lines; flags A combination of ORJ_FLAG flags which affect the processing of the Jar file; Note: throws a DatabaseException if the contents do not represent a correctly formed Open-RJ database
Record opIndex(uint index): The record at the given index; Note: the function gives undefined behaviour if the index is invalid
Record[] getRecordsContainingField(char[] fieldName): Returns an array of all records that contain a Field with the given name; fieldName The name of the field
Record[] getRecordsContainingField(char[] fieldName, char[] fieldValue): Returns an array of all records that contain a Field with the given name and value; fieldName The name of the field; fieldValue The value of the field. May be null, in which case the semantics are identical to getRecordsContainingField

and the following attributes:

char[] jarName(): The name of the database Jar file
uint flags(): The flags specified in the constructor
Record[] records(): An array of all records in the database
Field[] fields(): An array of all fields in the database
uint numFields(): The number of fields in the database
uint numRecords(): The number of records in the database
uint length(): The number of records in the database

std.openrj.Record

The Record class has the following methods:

Field opIndex(uint index): Returns the field at the given index; Note: the function gives undefined behaviour if the index is invalid
char[] opIndex(char[] fieldName): Returns the value of the (first) field with the given name; fieldName The name of the field; Note: throws an InvalidKeyException if no field is found
Field getField(char[] fieldName): Returns the (first) field with the given name; fieldName The name of the field; Note: throws an InvalidKeyException if no field is found
Field findField(char[] fieldName): Returns the (first) field with the given name; fieldName The name of the field; Note: return null if no field is found
bool hasField(char[] fieldName): Returns true if the record contains one or more fields with the given name; fieldName The name of the field

and the following attributes:

Field[] fields(): An array of all fields in the record
uint numFields(): The number of fields in the record
uint length(): The number of fields in the record
Database database(): The database within which the record resides

std.openrj.Field

The Field class has the following properties: methods:

char[] name(): The name of the field
char[] value(): The value of the field
Record record(): The record within which the field resides. May be null

std.openrj.OpenRJException

Base exception class used by the openrj module.

std.openrj.DatabaseException

Exception class, derived from OpenRJException, thrown if the database Jar file cannot be opened, or its contents do not represent a correctly formed Open-RJ database.

The DatabaseException class has the following properties: methods:

ORJRC rc(): The general error code associated with the exception
ORJ_PARSE_ERROR parseError(): The parsing error code associated with the exception
int lineNum(): The lineNum associated with the exception

std.openrj.InvalidKeyException

Exception class, derived from OpenRJException, thrown when named fields are not found.

std.openrj.ORJ_FLAG

Enumeration, whose values control the loading behaviour of the Database class.

ORDER_FIELDS - Arranges the fields in alphabetical order
ELIDE_BLANK_RECORDS - Causes blank records to be ignored

std.openrj.ORJRC

Enumeration representing general database processing errors encountered in the loading of the Database class.

SUCCESS - Operation was successful
CANNOT_OPEN_JAR_FILE - The given file does not exist, or cannot be accessed
NO_RECORDS - The database file contained no records
OUT_OF_MEMORY - The API suffered memory exhaustion
BAD_FILE_READ - A read operation failed
PARSE_ERROR - Parsing of the database file failed due to a syntax error
INVALID_INDEX - An invalid index was specified
UNEXPECTED - An unexpected condition was encountered
INVALID_CONTENT - The database file contained invalid content

std.openrj.ORJ_PARSE_ERROR

Enumeration representing database parsing errors encountered in the loading of the Database class.

SUCCESS - Parsing was successful
RECORD_SEPARATOR_IN_CONTINUATION - A record separator was encountered during a content line continuation
UNFINISHED_LINE - The last line in the database was not terminated by a line-feed
UNFINISHED_FIELD - The last field in the database file was not terminated by a record separator
UNFINISHED_RECORD - The last record in the database file was not terminated by a record separator