🔍

Custom plugins

The platform allows creating two types of plugins:

stateless functions (monads, dyads, triads, tetrads, polyads);
stateful reagents (async streams).

Both types are written in Rust, compiled as dynamic libraries (.dylib on macOS, .so on Linux), and loaded at runtime.

Prerequisites

Building plugins requires:

Rust nightly toolchain
Platform source tree — the ext crate and its transitive dependencies are needed at compile time

The ext crate depends on several internal crates, so the full kernel source tree is required:

kernel/
  kernel/
    ext/              # plugin interface
    ops/              # operations
    vectorize/        # proc-macros (#[monad], #[dyad])
    o/ffi/            # FFI types (AST, Interpreter API)
    rt/               # runtime (basert, utilsrt, sync)

Without access to the platform source code, plugins cannot be compiled.

Stateless function plugin

Let's create a plugin that registers a custom add dyad function.

Project setup

Create a new Rust library:

cargo new --lib my_plugin

Configure Cargo.toml:

[package]
name = "my_plugin"
version = "0.1.0"
edition = "2021"

[dependencies]
ext = { path = "/kernel/ext" }

[lib]
crate-type = ["cdylib"]

The ext crate path should point to kernel/ext directory inside the platform source tree. For example, if the platform source is at /home/user/theplatform , then use ext = { path = "/home/user/theplatform/kernel/ext" } .

Plugin code

Create src/lib.rs:

#![feature(abi_vectorcall)]
#![feature(allocator_api)]
#![feature(c_unwind)]

extern crate ext;
use ext::*;

#[dyad]
fn add_fn(lhs: &AST, rhs: &AST, i: &Interpreter) -> AST {
    match (lhs.tp(), rhs.tp()) {
        (SC_LONG, SC_LONG) => {
            i.new_scalar_long(lhs.long() + rhs.long())
        }
        _ => panic!("nyi")
    }
}

declare_plugin!(
    // constructor: called when plugin is loaded
    |i: &Interpreter| {
        let sym = i.intern_string("add");
        i.insert_entity(sym, i.new_verb_dyad(add_fn));
        i.new_string("add registered")
    },
    // destructor: called when plugin is unloaded
    || {}
);

Key elements:

#[dyad] macro marks the function as a dyadic verb (2 arguments). Use #[monad] for monadic (1 argument).
declare_plugin! macro generates the plugin entry points. The constructor receives an Interpreter reference and registers functions.
i.intern_string("add") creates a symbol for the function name.
i.insert_entity(sym, i.new_verb_dyad(add_fn)) binds the symbol to the function.

Building

Requires Rust nightly toolchain:

cargo build --release

The compiled plugin will be at target/release/libmy_plugin.dylib (macOS) or target/release/libmy_plugin.so (Linux).

Usage

o)load "/path/to/libmy_plugin.dylib";
"add registered"
o)add[1;2]
3
o)add[10;20]
30
o)add[100;200]
300

Available function arities

Macro	Arity	Registration	Example call
`#[monad]`	1	`new_verb_monad(f)`	`f[x]`
`#[dyad]`	2	`new_verb_dyad(f)`	`f[x;y]`

Interpreter API

The Interpreter reference provides methods for creating and inspecting values:

Constructors:

Method	Description
`new_scalar_long(v)`	Create long scalar
`new_string(s)`	Create string
`new_vector_long(iter)`	Create long vector from iterator
`new_dict(keys, vals)`	Create dictionary
`new_signal(msg)`	Create error signal
`new_verb_monad(f)`	Wrap function as monad
`new_verb_dyad(f)`	Wrap function as dyad

Accessors:

Method	Description
`tp()`	Get value type id
`long()`	Extract long value
`float()`	Extract float value
`bool()`	Extract bool value
`string()`	Extract string slice

Type constants:

Constant	Type
`SC_LONG`	Scalar long
`SC_FLOAT`	Scalar float
`SC_BOOL`	Scalar bool
`VEC_LONG`	Vector of longs
`VEC_CHAR`	String (vector of chars)

Using built-in verbs

Plugins can call existing platform verbs:

#[dyad]
fn add_fn(lhs: &AST, rhs: &AST, i: &Interpreter) -> AST {
    match (lhs.tp(), rhs.tp()) {
        (SC_LONG, SC_LONG) => {
            let dyad = i.get_verb_dyad("+").unwrap().dyad();
            unsafe { dyad(lhs, rhs, i) }
        }
        _ => panic!("nyi")
    }
}

Stateful reagent plugin

Reagent plugins implement async I/O streams. They consist of a Sender (sink) and Receiver (stream).

Project setup

Same as function plugin, but also depends on rt crate:

[dependencies]
ext = { path = "/kernel/ext" }
rt = { path = "/kernel/o/rt" }

Plugin code

#![feature(allocator_api)]
#![feature(abi_vectorcall)]
#![feature(c_unwind)]

extern crate ext;
extern crate rt;

use core::pin::Pin;
use ext::*;
use std::io;
use std::task::{Context, Poll};

pub struct Receiver;

#[derive(Clone)]
pub struct Sender;

impl Sink for Sender {
    type Error = io::Error;
    fn poll_ready(self: Pin<&mut Self>, cx: &mut Context<'_>)
        -> Poll> { self.poll_flush(cx) }
    fn start_send(self: Pin<&mut Self>, item: AST)
        -> Result<(), Self::Error> {
        let interp = rt::task::local::interpreter::().unwrap();
        let mut s = String::new();
        interp.format(&mut s, &item);
        println!("reagent send: {}", s);
        Ok(())
    }
    fn poll_flush(self: Pin<&mut Self>, _: &mut Context<'_>)
        -> Poll> { Poll::Ready(Ok(())) }
    fn poll_close(self: Pin<&mut Self>, _: &mut Context<'_>)
        -> Poll> { Poll::Ready(Ok(())) }
}

impl Meta for Sender {
    fn meta(&self) -> Option {
        MetaInfo::builder().entry("type", "my-reagent").finish()
    }
}

impl SinkReagentExt for Sender {
    fn boxed(&self) -> Box + Send + 'static> {
        Box::new(self.clone())
    }
}

impl Stream for Receiver {
    type Item = Result;
    fn poll_next(self: Pin<&mut Self>, _cx: &mut Context<'_>)
        -> Poll> {
        let interp = rt::task::local::interpreter::().unwrap();
        Poll::Ready(Some(Ok(interp.new_string("hello from reagent"))))
    }
}

impl StreamReagentExt for Receiver {}

pub extern "C-unwind" fn new_reagent(
    _: &[AST], _: &Interpreter
) -> io::Result<(SinkReagent, StreamReagent)> {
    let rx = Box::new(Receiver);
    let tx = Box::new(Sender);
    Ok((tx, rx))
}

declare_plugin!(
    |i: &Interpreter| {
        i.register_reagent("myreagent", new_reagent);
        i.new_string("reagent registered")
    },
    || {}
);

Traits to implement

Trait	Purpose
`Sink`	Sending data to the reagent
`SinkReagentExt`	Cloning for the sink
`Stream`	Receiving data from the reagent
`StreamReagentExt`	Extension trait for the stream
`Meta`	Metadata for introspection

Usage

o)load "/path/to/libmy_reagent.dylib";
"reagent registered"
o)r: reagent[`myreagent];
o)r["test"];
reagent send: "test"
o)react {[x:r] println[x]};

load

reagent

react