There's not being perfect like cussing in front of your kids or missing out on birthdays because you're busy.
Then there's leaving you're kid alone with emotionally scarred womenchildren from space with superpowers that had previously taken your child away without consent, almost getting him killed, putting their emotional pressure on him, have had numerous dubious attitudes towards humanity, and are also wanted fugitives. To SU's credit, Steven's mental situation in SUF was always the inevitable result.
The cherry on the top is abandoning your son with women that look and act like his mom. If it was hard on Greg, he should've known it would've been 2x harder on Steven. I still love Greg, tho.