Show HN: I have trained 15 blood glucose forecasting models on my own data

Caution

Research and educational use only - not a medical device. T1DMAI is a research artifact. It is not clinically validated, has no regulatory clearance, and its output is a forecast of research signals, not medical advice. It must not be used to make medical, diagnostic, or treatment decisions, to calculate or adjust insulin doses, or to manage diabetes in any way. For medical guidance consult a qualified healthcare professional. The software is provided "as is", without warranty of any kind (see LICENSE).

Note

Most of the code in this repo was written by AI. If you don't like that, feel free to leave.

This repo is a zoo. Here I train & test many blood glucose forecasting models (including my own) on both my own data and metabonet (public).

My model is a small transformer that forecasts blood glucose 2 hours ahead. I've made two variants: D16 DILATE and D16 MSE which use DILATE and MSE loss for training, respectively. D16 refers to the width of the model (16).

They are both trained on synthetic traces, and then a LoRA adapter is added for light fine-tuning, but I also test them without the adapter.

Feast thine eyes on some of their forecasts (these have been trained on synthetic traces, then fine-tuned with a LoRA adapter on my training split):

Background

I have type 1 diabetes and I wanna know if I can train a BG forecasting model to help me with my daily management. To this end I've made some auxiliary tools:

• T1DMSIM creates synthetic CGM traces since UVA/Padova was too slow on my laptop.
• T1DMDROID runs the models on my phone, also has some neat functionality like reading BG from multiple CGMs at once.

For evaluation, I use the recommendations by Jacobs et al., IEEE Reviews in Biomedical Engineering, 2024 [1]. It's a nice paper that you should probably read.

Main results

All models suck. XGBoost is the best one but my own model comes close. More details below:

• On my data, D16 DILATE + LoRA comes 3rd of 26 (22.8), behind XGBoost (22.2) and LightGBM (22.4). Wilcoxon puts it level with both at 60 and 90 min, behind them at 30 and 120.
• On MetaboNet, D16 DILATE + LoRA comes 9th (34.6) but has the lowest RMSE of all models at 90 min. XGBoost leads at 30, 60 and 120.
• At 60 min a 3771 parameter LoRA adapter helps both models on both datasets: 26.1 to 22.8 and 27.1 to 23.5 on my data, 36.6 to 34.6 and 36.7 to 35.1 on MetaboNet. At 120 min on MetaboNet it makes D16 DILATE worse (48.0 to 50.4).
• On MetaboNet, LoRA also fixes the bands. The 90% band holds the truth 75-78% of the time at 60 min with sim only and 81-85% with LoRA. On my data both are 83-85%.
• T1DMAI + LoRA reacts earliest on my data: 40-50 min time gain at 60 min, against 30 for XGBoost.
• It also catches the most lows on my data: 40-47% at 60 min, against 29% for XGBoost. On MetaboNet every low-RMSE model misses nearly all lows (XGBoost 0%, T1DMAI 6-8%), while linear extrapolation catches 58%.
• LightGBM matches XGBoost within 0.2 mg/dL everywhere. An LSTM with 15 hours of history comes 3rd on MetaboNet (33.6 at 60 min), against 35.9 for the 2 hour one.
• Zero-shot Chronos-2 and TimesFM 2.5 beat zero-shot Chronos-Bolt by 1.4-2.7 mg/dL at 60 min on both datasets, but trail fine-tuned Chronos-Bolt.

Models

Numbers in brackets point to References.

Everything except the baselines and the foundation models was trained from scratch on each dataset's training split. T1DMAI comes in two versions: sim only is the checkpoint in models/, and sim + LoRA adds a frozen-base rank 1 adapter [23] (3771 parameters) trained on the real training split.

T1DMAI details:

| Architecture | encoder-only transformer, 30 min patches, one attention head, RoPE [26], SwiGLU |
| Size | 16 layers of width 16, 31k parameters |
| Loss | 7 quantile pinball loss plus a median term (DILATE [25] on soft-DTW [24], or MSE) |
| Inputs | CGM in a risk-scaled space, carb absorption curve, insulin action curve, mask bit |
| Output | 7 quantiles (5% to 95%) for every 5 minute step of the next 2 hours |
| History | up to 7 days |
| Simulator training | 1M simulated people, 7 days each, 10000 steps of batch 512 |
| Runtime | ExecuTorch or LiteRT, on the phone in T1DMDROID |

More in docs/USAGE.md and ARCHITECTURE.md.

Data

My data. 115 days from T1DMDROID: days 1 to 80 for training, 81 to 115 for testing. Note that the model sees future inputs like insulin & carbs. I should probably also train the adapters while withholding those inputs.

MetaboNet [2]. Official split, 279 test people from 9 studies. Blind, because most of them use closed-loop pumps whose future insulin reacts to glucose the model hasn't seen.

Method

| My data | MetaboNet |
|---|---|
| Forecasts | every 5 min of the test days | every 10th row of the leaderboard template |
| Horizons | 30, 60, 90, 120 min | 30, 60, 90, 120 min |
| Training windows, other models | 60000 | 351000 (200000 for fine-tuned Chronos) |
| LoRA steps | 2000 | 6000 |

• LoRA checkpoints are picked by mean RMSE on the last 10 days of each subject's training data.
• All models forecast at the same moments.
• 95% intervals: bootstrap over people (MetaboNet) or days (my data).
• Significance: Wilcoxon signed-rank on paired absolute errors against the best model, Bonferroni corrected.
• Clinical error grids: DTS [20], Clarke [22], CG-EGA [21].

Limitations

• My data is N=1 over 35 days.
• The test sets differ a lot (in range 89% vs 75%, CV 28% vs 40%), so don't compare numbers across them.
• Lows are rare: 292 forecasts on my data, 8347 on MetaboNet.
• T1DMAI fell back to the last reading on 0.7% of my forecasts and 0.4% on MetaboNet, including all 28 ShanghaiT1DM ones. The other models never fall back.
• The bands are raw quantiles, no conformal correction.
• The sparse GP's own uncertainty isn't shown. Its noise fit hit the bottom of its grid, giving a band about 1 mg/dL wide.
• Time gain is only computed for my data.
• Unlogged meals count as "no meal".
• CPU timings are single runs on one laptop core and varied by up to 45% between runs.
• LightGBM, LSTM (15 h), Chronos-2 and TimesFM 2.5 have no CPU timing yet.

Terms

Full tables

A * marks the lowest RMSE and the models not significantly worse than it.

My data:

| Sparse GP | 89