Generalized Transformers from Applicative Functors

Transformers are a machine-learning model at the foundation of many state-of-the-art systems in modern AI, originally proposed in [arXiv:1706.03762]. In this post, we are going to build a generalization of Transformer models that can operate on (almost) arbitrary structures such as functions, graphs, probability distributions, not just matrices and vectors.

[...]

This work is part of a series of similar ideas exploring machine learning through abstract diagrammatical means.

https://cybercat.institute/2025/02/12/transformers-applicati...