Typically, in quantum mechanics classes, we're taught that position eigenstates "aren't real states" because they're not square integrable, so not in the Hilbert space.
In the more general algebraic quantum mechanics though, states aren't specifically identified with vectors in a Hilbert space, but rather positive, normalized linear functionals on the C*-algebra of observables.
The delta "state" fits this definition. You can, for example, take a subsequential weak* limit of well-defined Hilbert space states that are getting skinnier and taller. This is guaranteed to exist by the compactness of the state space in the weak* topology. This resulting state would behave exactly like the delta state.
Of course, it wouldn't be a normal state with respect to the standard representation, but it would still be a valid state.
On a kind of related note, if this is true, does this interpretation kind of follow to type III von Neumann algebras?
I've read that the interpretation of mixed states as probability measures over pure states breaks down in type III von Neumann algebras, since the pure states making up the mixed states are not normal in the mixed state's GNS representation.
However, if the non-normal pure states behave like deltas, I don't really see the issue with still interpreting the mixed state as a probability distribution over the pure states. We essentially do that all the time in classical mechanics.