Yep. During its second season, Family Guy had weak ratings against Frasier and Who Wants to Be a Millionaire. That led to its first cancellation.
After FOX picked up the series for 13 more episodes at the very last second, the show still suffered against Survivor and Friends. That was the second cancellation.
As a raunchy Simpsons rip-off, FG had always been on death row until [adult swim] reruns with high viewership and successful DVD sales convinced FOX to give the series one final chance.
Family Guy's early history is pretty shocking considering how big of a name Seth MacFarlane is now, as well as the fact the show just reached its 20th anniversary on the air.