A linear SVM can in turn be expressed as a very shallow neutral network. The main difference is that with SVMs you put all your effort into transforming inputs for the model (e.g. all the popular kernels) while with neural networks usually most of the effort goes into clever model architectures.