Don't remove Ashi. Just instead of Jack dragging her around and showing what a good guy he is have fourth episode with Ashi trying to kill Jack through the episode. Why doesn't Jack kill her? Well, let's assume that Jack has already been in the worm they were in and change things so that he knew only way out would work if two people have to work together. He explains and she reluctantly cooperates. Make it so that something goes wrong and Jack decides to save her life in the last moment. Jack tells her the same thing he told her before he killed her sisters and instead of attacking Ashi escapes, she can't beat him and she owes him her life.
Next episode High Priestess beats the shit out of her reminding that her life belongs to Aku, she decimates villages and makes Ashi kidnap children to mind control them. Ashi isn't comfortable with this, but can't refuse. High Priestess tells Jack that if he fights her back she will make Ashi kill the children and Ashi does have a button to kill them, seeing Jack getting killed for children by her mother who used to beat her up was her last straw and she breaks mind control device, but makes Jack snap thinking he failed children.
Next episode she leaves children at an orphanage, but takes one of them with her as a proof that Jack didn't fail. While Ashi learns about Jack's exploits by meeting different characters he helped for fanservice, we also hear couple of legends from mouth of a child about what a great hero Jack is, possibly told in a flashback. Along the way Ashi meets Scotsman's daughters and Scotsman's Specter, he joins Ashi. When Ashi, Scotsman and child see Jack trying to commit seppuku, Scotsman beats the hell out of the spirit who was accusing him and Ashi shows him the child and Ashi tells him she wants to help him.
I am running out of space, but I don't think you actually need to change a lot of things to fix 5th season, it just a little too much on tell, not show side.