🔍

Additional functions for partitioned tables

Name Description / comments
.o.pnew[<mt>;<pd>] Creates new partitioned table or appends new partition
.o.pget[<dir>] Loads partition metadict from disk & converts into partitioned table
.o.pset[<dir>;<mt>] Stores partitions and metadict on disk
.o.pstore[<dir>;<mt>;<ids>] Stores partitions and metadict on disk
.o.psym[<mt>;<ids>] Re-assign domain symbol
.o.pdel[<mt>;<ids>] Deletes partitions from partitioned table
.o.pmd[<dir>] Rеturn partition metainfo dict filename relative to "dir"
.o.pmnt[<mt>;<ids>] Mounts specific partitions
.o.pumnt[<mt>;<ids>] Un-mounts specific partitions
.o.pmtvalid[<mt>] Checks if mt is proper metainfo dict
.o.pvalid[<mt>] Checks if mt is ptable or proper metainfo dict
.o.pidvalid[<ids>] Checks if partition ids are valid
.o.pid2ix[<mt>;<ids>] Translate partition ids into simple indices

whеre

mt - ptable or ptable metainfo dict or 0N0 (for new ptable creation)

pd - dict with `gf`info fields describing new partition

dir - ptable directory symbol

ids - partition identifiers

Creating ptable or adding new partition

Passing 0N0 to .o.pnew as existing partitioned table makes a new table, otherwise adds a new partition.

o) load "core";
o) gf:+`a`b!(1 2;10 20);
o) info:+`size`segId`refPath!(2 2; 0 0; (0N0;0N0));
o) p1: +`c`d!(100 100;200 200);
o) p2: +`c`d!(1000 1000;2000 2000);
o) mnt:+`mntval!(p1;p2);
o) pd: `gf`info`mnt!(gf;info;mnt);
o) pt:.o.pnew[0N0; pd];
o) pt
a b  c    d   
--------------
1 10 100  200 
1 10 100  200 
2 20 1000 2000
2 20 1000 2000

And creating new ptable with enum fields is done like:

o) load "core";
o) sym1::`1`2`3`4`100`1000;
o) sym2::sym1;
o) gf:+`a`b!(`sym1$`1`2;10 20);
o) info:+`size`segId`refPath!(2 2; 0 0; (0N0;0N0));
o) p1: +`c`d!(`sym1$`100`100;200 200);
o) p2: +`c`d!(`sym1$`1000`1000;2000 2000);
o) mnt:+`mntval!(p1;p2);
o) pd: `gf`info`mnt`sym!(gf;info;mnt;`sym2);
o) gf2:+`a`b!(`sym1$`3`4;10 20);
o) pd2: `gf`info`mnt!(gf2;info;mnt);
o) pt:.o.pnew[`ptable$pd; pd2];
o) pt`a
`sym2$`1`1`2`2`3`3`4`4
Note sym2 becomes domain for the entire ptable.

See following for adding new partition.

o) load "core";
o) gf:+`a`b!(1 2;10 20);
o) info:+`size`segId`refPath!(2 2; 0 0; (0N0;0N0));
o) p1: +`c`d!(100 100;200 200);
o) p2: +`c`d!(1000 1000;2000 2000);
o) mnt:+`mntval!(p1;p2);
o) pd: `gf`info`mnt!(gf;info;mnt);
o) gf2:+`a`b!(3 4;10 20);
o) pd2:`gf`info`mnt!(gf2;info;mnt);
o) pt:.o.pnew[`ptable$pd; pd2];
o) pt
a b  c    d   
--------------
1 10 100  200 
1 10 100  200 
2 20 1000 2000
2 20 1000 2000
3 10 100  200 
3 10 100  200 
4 20 1000 2000
4 20 1000 2000

Saving / loading ptable to/from disk

Ptables are saved into one directory by default. Sub-directories are named using random GUID vаlues unless manually named (use refPath).

.o.pset function is used for saving to disk, .o.pget - to get saved ptable meta from disk.

o) load "core";
o) gf:+`a`b!(1 2;10 20);
o) info:+`size`segId`refPath!(2 2; 0 0; (0N0;0N0));
o) p1: +`c`d!(100 100;200 200);
o) p2: +`c`d!(1000 1000;2000 2000);
o) mnt:+`mntval!(p1;p2);
o) pd: `gf`info`mnt!(gf;info;mnt);
o) pt:.o.pnew[0N0; pd];
o) .o.pset[`:/tmp/pset1/; pt];
o) pt2:.o.pget[`:/tmp/pset1/];

Important! Please pay attention to trailing slash at the end of the dir symbol.

Just as with ordinary table, it's user responsibility to assign domains to enums.Use .o.psym function to re-assign domain symbol after loading ptable.

o) load "core";
o) sym1::`1`2`100`1000;
o) sym2::sym1;
o) gf:+`a`b!(`sym1$`1`2;10 20);
o) info:+`size`segId`refPath!(2 2; 0 0; (0N0;0N0));
o) p1: +`c`d!(`sym1$`100`100;(200;"TEST"));
o) p2: +`c`d!(`sym1$`1000`1000;(2000;"TEST"));
o) mnt:+`mntval!(p1;p2);
o) pd: `gf`info`mnt!(gf;info;mnt);
o) pt:.o.pnew[0N0; pd];
o) .o.pset[`:/tmp/pset3/; pt];
o) pt2:.o.pget[`:/tmp/pset3/];
o) pt2:.o.psym[pt2; `sym2];
o) pt2`a
`sym2$`1`1`2`2

Partition ids

Several following functions accept partition ids as argument.Partition ids can be defined as:

Id value Example Meaning
0N0 0N0 all partitions
vector of longs ,0 only partitions with given indices starting from zero
table of partitioning fields +`x`y!(1 2; 2000.09.11 2022.02.24) only partitions with corresponding partitioning fields

Saving individual partitions to disk

.o.pstore function is just more general alternative to .o.pset. It can be used to save all or one or several partitions at a time along withmeta-dict. It expects target directory as first arg, ptable - as second one, and partition ids - as third one.

o) load "core";
o) gf:+`a`b!(1 2;10 20);
o) info:+`size`segId`refPath!(2 2; 0 0; (0N0;0N0));
o) p1: +`c`d!(100 100;200 200);
o) p2: +`c`d!(1000 1000;2000 2000);
o) mnt:+`mntval!(p1;p2);
o) pd: `gf`info`mnt!(gf;info;mnt);
o) pt:.o.pnew[0N0; pd];
o) .o.pstore[`:/tmp/pset1/; pt; ,0];       // just first partition + meta-dict will be saved
o) .o.pstore[`:/tmp/pset1/; pt; ,(#gf)-1]; // just last partition + meta-dict will be saved
o) .o.pstore[`:/tmp/pset1/; pt; 0N0];      // all partitions + meta-dict will be saved
o) .o.pstore[`:/tmp/pset1/; pt; gf];       // all partitions defined by "gf" + meta-dict will be saved

Deleting partitions

Removing partition is made using .o.pdel function. It rеturns partitioned table without deleted partition. Partition are specified by their partition ids.

o) load "core";
o) gf:+`a`b!(1 2;10 20);
o) info:+`size`segId`refPath!(2 2; 0 0; (0N0;0N0));
o) p1: +`c`d!(100 100;200 200);
o) p2: +`c`d!(1000 1000;2000 2000);
o) mnt:+`mntval!(p1;p2);
o) pd: `gf`info`mnt!(gf;info;mnt);
o) pt:.o.pnew[0N0; pd];
o) pt2:.o.pdel[pt; 0];
o) pt2
a b  c    d   
--------------
2 20 1000 2000
2 20 1000 2000

Mounting / unmounting partitions

To avoid quite heavy lazy mounting and immediately unmounting partitions, manual mounting is supported.

See example below. Note compound lists partitions.

o) // test with compound list in partitions
o) load "core";
o) gf:+`a`b!(1 2;10 20);
o) info:+`size`segId`refPath!(2 2; 0 0; (0N0;0N0));
o) .p1: +`c`d!(100 100;(200;"TEST"));
o) .p2: +`c`d!(1000 1000;(2000;"TEST"));
o) mnt:+`mntval!(.p1;.p2);
o) pd: `gf`info`mnt!(gf;info;mnt);
o) pt:.o.pnew[0N0; pd];
o)
o) // save to disk and load back
o) .o.pset[`:/tmp/pmnt_umnt1/; pt];
o) pt2:.o.pget[`:/tmp/pmnt_umnt1/];
o)
o) pt3:.o.pmnt[pt2; gf];
o) d3:`dict$pt3;
o) pt4:.o.pumnt[pt3; gf];
o) d4:`dict$pt4;
o) d3
gf  | +`a`b!(1 2;10 20)
info| +`size`segId`refPath!(2 2;0 0;(`:1c516dea-89b1-4182-9f1f-feecb3420efe/;`:3848a412-1c17-4935-92ae-fd6ce2dc4a40/))
mnt | +,`mntval!((+`c`d!(100 100;(200;"TEST"));+`c`d!(1000 1000;(2000;"TEST"))))
root| `:/tmp/pmnt_umnt1/
o) d4
gf  | +`a`b!(1 2;10 20)
info| +`size`segId`refPath!(2 2;0 0;(`:1c516dea-89b1-4182-9f1f-feecb3420efe/;`:3848a412-1c17-4935-92ae-fd6ce2dc4a40/))
mnt | +,`mntval!((`:1c516dea-89b1-4182-9f1f-feecb3420efe/;`:3848a412-1c17-4935-92ae-fd6ce2dc4a40/))
root| `:/tmp/pmnt_umnt1/

Note how mntval contents change in each stage.