ALPHAio: ALPHA dobs reader/writer

ALPHA dobs format is a file format to store observables from Monte Carlo simulations. It is a specification built on top on BDIO.

The ALPHA dobs format

The format is structured around BDIO records. There are three types of records:

• Fluctuation records: These store fluctuations of the observables with respect to different Monte Carlo ensembles. They are identified by having a BDIO user-info 2, and are binary records. The standard on how to read/write this fluctuations can be found in the documentation of the ADerrors package.
• Parameter records: These store information on the fluctuations. This information is essential/useful to interpret the observables that follow. These are ASCII records, that in fact store a TOML file, and are identified by having user-info 8.
• Info records: These are plain ASCII records with free format that explain the data that is contained in the file.

ALPHA dobs file structure

1. Info Record
2. Parameter record A
3. Fuctuation record A.1
4. Fuctuation record A.2
5. ...
6. Parameter record B
7. Fuctuation record B.1
8. Fuctuation record B.2
9. ...
10. Info record

An example will help in understanding the structure.

In this case the file has a single parameter record (with user info 8). As we can see by "dumping" the parameter record, it specifies that three observables follow. The parameter record also includes some additional information (the variables sigma, exact, N1 and N2) under the header "extra".

The file also starts and end with generic ASCII records with user info 1. These records are supposed to be only human readable, and should explain what the file contains. The format of these records is arbitrary. Finally, the file contains records with user info 7. These represent the checksum (md5) of the previous record. Checksum records are nor mandatory (the example above did not have checksum for the records with user info 1), but are highly recommended, specially for fluctuation and parameter records.

ALPHA dobs parameter records

There are several type of parameter records, depending on the structure of the observables that follow, but all parameter records must obey the following conventions

1. The following keys are mandatory:
   • IDHDR A header ID identifying the format. It must always be the integer 1709034443.
   • nobs An integer specifying the nomber of observables stored with this structure
   • dimensionsAn integer specifying the dimensions of the observables that follow. A 0 indicates a scalar observable, a 1 a vector of observables, and in general an integer d specifies a d-dimensional array of observables.
   • type A string that can be only "obs" or "dict". It specifies if the observables are plain observables or Dictionaries with strings as keys.
2. Moreover the following rules apply:
   • If dimensions > 0, there must be a key size that stores a vector of integers with dimension elements. They specify the length of the axes of the array.
   • If type is "dict", there must be a key keys that is a vector of strings that holds the keys of the dictionaries.
3. Finally all parameter records can contain arbitrary TOML data. The only condition is that it should not appear within a header.

The different type of parameter records are described below.

Scalar parameter records

These are identified by the values type being "obs" and dimension being 0. An example valid TOML data is

nobs = 3
dimensions = 0
type = "obs"
IDHDR = 1709034443

[extra]
sigma = 0.011095009552825661
exact = 0.0006287577745979422
N1 = 1097
N2 = 8103

That implies that 3 observables follow. After this record the file must contain 3 fluctuation records (with user info 2), before having any other parameter record.

Array parameter records

These are identified by the values type being "obs" and dimension being any integer larger than 0 (the dimension of the array). An example valid TOML data is

nobs = 2
dimensions = 2
size = [2,3]
type = "obs"
IDHDR = 1709034443

[other]
mhat = 0.12243487484576

This header specifies that 2 observables follow, each being a 2x3 matrix (two dimensional array). After this record the file must contain 12 fluctuation records (with user info 2), before having any other parameter record.

Dictionary parameter records

These are identified by the value type being "dict". An example valid TOML data is

nobs = 3
dimensions = 1
size = [192]
type = "dict"
keys = ["mpi", "mk", "md"]
IDHDR = 1709034443

[prm]
lambda = 0.12243487484576

This header specifies that 3 observables follow, each being a 192 element vector (one dimensional array). The 3 observables must be interpreted as a dictionary with keys ["mpi", "mk", "md"]. After this record the file must contain 576 (=192 x 3) fluctuation records (with user info 2), before having any other parameter record.