-Sporadic airing schedule in season 4.
-It is no longer the only show with its brand of humor on TV. There are rivals now.
-It has failed to keep pace with rival shows and no longer holds the same kind of interest it once had.
-Completely abandoned worldbuilding and genuine plot or continuity in favor of shitting on tropes common to modern dramedies or adult animation, but beyond some clever visual gags it does little with its parody status and also dips into referencing previous events despite there also being a status quo.
-Now that R&M's style of humor is commonplace people are getting tired of it, and are turning off to look for something new. Jaded, cynical humor is harder to appreciate at the present moment (going outside itself is becoming a nightmare).
There are other factors too but I think these are some of them.