I know this is a dumb question, but what are the disadvantages and risks of eating food you're mildly allergic to?

There're some foods I really like but which I'm lightly allergic to. Symptoms usually include lips puffing up slightly, slight itchiness in mouth and throat, and minor gastrointestinal symptoms. They usually pass within 20 - 60 minutes.

Obviously the smartest thing to do is just avoid those foods no matter what. But if I do consume them, am I potentially experiencing additional negative symptoms I don't notice?

For example, could there be short-term negative neurological effects? Does histamine negatively affect cognition at all? It seems like it could increase wakefulness, which actually seems helpful, but I'm wondering what harm it might also be doing