I can't really agree with that. I feel like you're looking at it a bit too closely, like the story was a two-hours movie with main characters being the only one with actual agency to move the plot forward.
It doesn't work like that with serialized comics in general and especially Ichikawa's manga. When reading her works there is always this eerily lingering aura of omen, which is partly due to her inhumanly uncanny valley characters, but also based on how the story has an unescapable destiny, a story going in circles or curses going on.
Moreover, explanation, lore, foreshadowing and everything of the sort ARE plot in this kind of manga. How many times did we see apparently minor remarks that turned out to be important in hindsight? The earth gems have Sensei who can drop bits of lore anytime.
To get to the actual answer and point of disagreement, I don't think that Ichikawa's treating the two groups of gems as separated characters. They're still a single group that had their interpersonal dynamics fucked up by the trauma of willing departure and infighting, but the agency isn't just on Phos' part. The biggest change actually happened on the earth gems' part, with Adamant stepping down and basically reprogramming himself. Euclase and Phos/Parappa's plans were each inextricably tied by the assumptions on the other part's plan. Chapter 71 is deliberately written with a mirroring structure between the two groups to embody that.
Houseki no kuni is about Phos' journey, but a central theme is that enlightenment does not come from agency, with the main characters being the ones that can move the plot around the most: Adamant is a central character and he barely "did" anything out of volition. How you react to things and work towards reaching piece of mind, that should be the point of the struggle. Phos is the most frantic character in the manga, but everything he did so far ended up in disappointment, except the parts where he just showed sheer kindness.