Research Seminar on AI: Tabular foundation models (TFMs)

Tabular foundation models (TFMs) have recently achieved state-of-the-art performance on small tabular datasets by performing prediction in context, without task-specific training or hyperparameter optimization. This seminar is divided into two parts. The first part provides a primer on TFMs, explaining their connection to Bayesian inference, how they are pre-trained, and why they express strong generalization in small-data settings. The second part presents our evaluation of TFMs for molecular property prediction across pharmaceutical and chemical-engineering benchmarks. Following a rigorous benchmarking methodology, we find that, when choosing a suitable representation, TFMs outperform the current state of the art, including specialized pre-trained molecular foundation models, and strong classical baselines. These results are achieved without any hyperparameter optimization or task-specific fine-tuning, using only a single forward pass.