Sempods - Bare GPN Model Template

NOT YET READY FOR USE (a lot more code incoming shortly)

SemPod is a light-hearted way to say the phrase Bare Gaussian Prototypical Network; and simply means a pod where semantic classification of different kinds of things can happen. SemSage is the tool used to create them.

Using one, you can accurately classify anything from allergens to zoology without using LLM inference or reasoning! A SemPod works strictly from text embeddings and the amazing geometry that binds our language together.

SemPods are quite a bit more precise than the kinds of semantic matching you might be familiar with from technologies like RAG servers and semantic search because they use a much more advanced set of calculations.

How They Work (High Level)

(coming shortly)

How They Work (SMTM - Show Me The Math!)

(coming shortly)

What's in here

util/           semsage lifecycle CLI, training helpers, installers
app/inf/        dece-semsaged -- the HTTP scoring gateway (C++)
app/exp/        Foreshock dashboard (Deno + ECharts)
train/          trainer template, Lua provisioner, tolerance harness

How Covariant Estimation classifiers work

A classifier defines sensors -- concepts expressed as sets of example phrases. Each phrase is embedded into a 768-dimensional vector (nomic-embed-text v1.5). The centroid (mean) and diagonal variance of those embeddings form an independent Gaussian per sensor. Incoming text is embedded and scored against every sensor centroid using standardized (variance-weighted) Euclidean distance -- the diagonal case of the Mahalanobis distance. Scoring is O(1) per sensor.

Quick start

1. Install the toolchain

util/install_splinter   # builds llama.cpp, libsplinter, installs Deno
util/install_nomic      # downloads nomic-embed-text v1.5 GGUF (~274 MB)

2. Write your classifier

Copy the trainer template and define your sensors:

cp train/_template train/my_classifier
chmod +x train/my_classifier
# Edit train/my_classifier -- add SENSOR / INTRINSIC_SHUNT / MONOLITHIC_SHUNT calls

See the comments inside _template for the full DSL reference. Key conventions:

Prefix	Meaning
|	Unidirectional axis pillar (always returned, never filtered)
++ / --	Graduated strong positive/negative
+ / -	Mild positive/negative
%	Structural / framing detection
@	Ambient / personality trait
(none)	Standard sensor

Each sensor needs phrases (caret ^-delimited) and a bloom label matching your .rc file.

3. Train

sh train/my_classifier

This embeds all phrases, computes centroids and variances, and writes the model to dece/my_classifier/my_classifier.dece. Training takes 15-45 minutes depending on phrase count and hardware.

4. Serve

semsage uplink my_classifier.dece
semsage start my_classifier.dece

The scorer listens on 127.0.0.1:3271 by default. Score text via:

curl -X POST http://127.0.0.1:3271/api/v1/score -d "Your text here"

5. Visualize

semsage visualize

Opens the Foreshock dashboard at http://localhost:8000.

6. Validate

Place .txt specimen files in train/tests/, configure tolerance bands in train/tests/tolerances.json, then:

cd train/tests && python3 check_sensors.py

This scores every specimen, checks each sensor falls within its declared band, computes adhesion (regression margin), and writes an HTML report. Non-zero exit on any breach -- CI-ready.

Tuning a single sensor

semsage retrain my_classifier.dece "+joy"
# Edit the generated dece/my_classifier/retrain-+joy.sh
sh dece/my_classifier/retrain-+joy.sh

External embedder

Set SEMSAGE_EMBEDDING_URL to embed via an OpenAI-compatible HTTP endpoint instead of the local splinference IPC bus:

export SEMSAGE_EMBEDDING_URL=http://gpu-host:8080/v1/embeddings
export SEMSAGE_EMBEDDING_AUTH=sk-...        # optional bearer token
export SEMSAGE_EMBEDDING_MODEL=nomic-embed  # optional model name

Architecture

                    +---------+
  text specimen --> | embedder | -- 768D vector via IPC bus or HTTP
                    +----+----+
                         |
                    +----v----+
                    | scorer  | -- cosine sim + standardized Euclidean distance
                    +----+----+   against every cached sensor centroid
                         |
                    JSON response: [{key, similarity, distance, dotproduct}, ...]

The embedder (splinferencep) and scorer (dece-semsaged) communicate over a shared-memory IPC bus provisioned by semsage uplink. Alternatively, an external HTTP embedder can replace the local embedder entirely.

Retuning tolerances

After training or tuning, benchmark your model:

1. Place representative specimens in train/tests/
2. Run the scorer and note each sensor's similarity/distance values
3. Set bands in tolerances.json with ~6pt similarity margin and ~5 distance margin
4. Run check_sensors.py -- the adhesion metric tells you how much headroom each sensor has

License

Apache-2.0. See LICENSE.txt.