What's the limit on drawing on a layer of someone else's art and making your own edits before it becomes plagiarism?

I've saved a huge HD of manga/anime to trace from for my comic, and it's saving a lot of time rather than constructing the panels myself. But I don't want some autists to accuse me of plagiarism.