I think the "would" is often overlooked in the whole "Mother I WOULD like to fuck". It implies that whoever is speaking wouldn't like to fuck the majority of mothers, but would make an exception for that specific one.
That implies a lot more stuff just "motherhood". When I think of "MILF", I think of a traditional mother: a woman who's had one, possibly multiple babies, has been married before, is not in her youth, and doesn't have much time to take care for herself. Despite all of this, she's still attractive the point they make people go, "wow, now, that's a mother I would like to fuck".
A character like Mavis (who's been mentioned before) doesn't really count as a MILF despite being a mother because she's basically just a young adult woman, while a character like Aunt Cass, who's not a mother at all, strikes me more like a MILF because of her protective attitude, slightly more advanced age, and her simple looks.