Projects#

Evaluating educational materials with LLM-as-judge#

How do you tell whether an LLM’s output is any good when “good” is a pedagogical judgement rather than a string match? At Fab AI I led the design and deployment of an LLM-as-judge system that scores lesson plans, storybooks and textbooks against rubrics built with education specialists.

The useful result was methodological: framing criteria as binary checks with explicit fail conditions, rather than graded scales, reached 87% human–LLM agreement against a 91.8% human–human ceiling — consistent across GPT-5, Claude and Gemini. The rubrics and workflow are now used by external organisations to assess education technology.

Read the report for the full methodology and results.

Deep auditory encoding with self-attention to predict brain activity#

This library allows you to train a recurrent DNN (a GRU) and learn a self-attention mechanism that weighs hidden states - the resulting weighted tensor is used to predict brain activity (or whatever you choose as a target). It also contains many variations of this model type (shared attention between targets, multi-head attention etc) and some functions for visualizing the computed attention weights on a spectrogram.

The general idea is captured in this figure:

Schematic of the GRU self-attention encoding model

Read more in this blogpost!

A BIDS app implementing voxel-wise encoding models in fMRI using Docker#

The voxel-wise encoding BIDS app is a suite of Python tools for preprocessing fMRI and stimulus data, temporally aligning them, creating a lagged stimulus representation and training and validating voxel-wise encoding models using Ridge regression with hyperparameter search. If you don’t want to bother with a full Python installation, or want to run your analyses on a HPC cluster, you can easily use a Docker image for a smooth and reproducible workflow.

Schematic of the voxel-wise encoding BIDS app workflow

See this blogpost to learn more!

Open source contributions#

Alongside my own libraries I contribute to packages in the scientific Python ecosystem:

  • Nilearn — machine learning for neuroimaging.

  • skrub — machine learning with dirty categorical data, including the fuzzy-matching tools I wrote about in this post.