Tables on disk

You can save the table to disk as a "serialized table" or "splayed table". It is easy to store complex data in serialized tables, but such tables are entirely getting into memory, and there is no way to work with such tables larger than RAM.

o)t: +`simple`complex!(1 2; (`a`b`c!!3; `c`d!(1 2; 3 4)));
o)`:./tmp/stbl set t;
o)tbl: get `:./tmp/stbl
simple complex
1      `a`b`c!0 1 2
2      `c`d!(1 2;3 4)

"Splayed table" in O means just the same as in Q/KDB - saving each field vector in a separate continuous file capable of mmapping it.

"Mmapping" allows to avoid loading all table contents in memory in one go, but to work with a table off the disk (by using OS memory-mapped files technique).

Supported field types

Only following field types are supported in splayed tables:

  • Vectors excluding symbol vectors.
  • Compound lists - lists containing primitive scalars (all scalars except GUIDs) and/or vectors excluding symbols vectors. Nested lists are not supported.
Version 0.6.0+ supports scalar GUIDs in compound lists.


Convert symbol vectors into enum vectors first to be able to save them in splayed table. You can use .o.en function for that.

Saving on disk

Saving table vаlues to disk is pretty simple. Just choose directory to keep your table files and execute something similar to:

o)a:(+:)`a`b`c!(1 2 3;1 2 3;1 2 3); f:`:/tmp/o_table/; f set a;

Important! Please pay attention to trailing slash at the end of the path symbol. That defines saving table as a splayed table.

Doing that creates /tmp/o_table directory filled with files named ".d", "a", "b" and "c".These files corresponding to fields and the ".d" file which contains symbol vector `a`b`c to keep the field order of the table.

Please pay attention that only fields mentioned in ".d" file are considered as table fields. Thus even if files are in the table directory, but not in the ".d" file, they will be ignored.

Use "get" function to load table contents:

o)f:`:/tmp/o_table/; get f
a b c
1 1 1
2 2 2
3 3 3

Getting table contents results in a separate/full copy in memory. That is modifying loaded table does not influence the copy on disk in any way.

On-disk table modifications

Modifying table on disk can be done in two ways:

  • Amending via a path symbol.
  • Opening table in workspace.

Amending via a path symbol is useful e.g. for adding a new field.

o)// creating test table
o)a:(+:)`a`b`c!(1 2 3;1 2 3;1 2 3); f:`:/tmp/o_table2/; f set a;
o)// append two new fields
o)// modify .d file to record new fields

Appending records via path symbol is ok.

o)a:(+:)`a`b`c!(1 2 3;1 2 3;1 2 3); f:`:/tmp/o_table3/; f set a;
o)// appending one record
o)// appending two records
o).[f;();,;(10 10;20 20;30 30)];

Opening table in workspace works by binding global variable to table mmapped on disk.

o)// create new table on disk
o)a:(+:)`a`b`c!(1 2 3;1 2 3;1 2 3); f:`:/tmp/odb1/; f set a;
o)// load it to workspace. Now odb1 global variable contains a writable reference
o)load f;
o)// append new record
o)// ... or via short syntax
o)// see contents
a  b  c
1  1  1
2  2  2
3  3  3
10 20 30
10 20 30
o)// close table, saving pending changes