Modern Hopfield networks have enjoyed recent
interest due to their connection to attention in
transformers. Our paper provides a unified framework for sparse Hopfield networks by establishing
a link with Fenchel-Young losses. The result is a
new family of Hopfield-Fenchel-Young energies
whose update rules are end-to-end differentiable
sparse transformations. We reveal a connection
between loss margins, sparsity, and exact memory
retrieval. We further extend this framework to
structured Hopfield networks via the SparseMAP
transformation, which can retrieve pattern associations instead of a single pattern. Experiments on
multiple instance learning and text rationalization
demonstrate the usefulness of our approach.