People who discount power levels are morons who don't possess the adequate intellect and keen mind to discern the series' core values. Power levels are not meant to discredit or undermine the whimsical martial-arts-and-adventure-focal point of Dragon Ball, but to enhance it. Consider, for those that bear the cognitive ability, how simple it is for one to pit characters against each other in contests of skill and strength who otherwise would never have met nor have any reason to meet on the battlefield. Without such intricate and solid guidelines, such matchups would fester and ultimate stagnate in perpetuity, as neither side would have any sort of concrete evidence against the other, leading to an infinite and ostentatious barrage of insults, ad hominem, and headcanon. Yet, with the implementation of power levels, we are easily able to determine which character would hold the advantage in these "what-if" matchups, and as such, utilize them as the first source of evidence, with the rest of the debate then naturally leading into talks of feats, martial arts ability, and tactics.
Toriyama's dismissal of power levels was only meant as a dismissal of the fools who sought to replace the story's roots with them, not of the concept as a whole. Indeed, power levels make up the backbone of the franchise, and are arguably the most integral part, as they facilitate healthy intellectual debate and force proponents of hypotheses to back up their proposals with factual statistics and numbers, lest their arguments be immediately decried as headcanon.
Only those who do not possess an adequately refined mind--most likely the same "people" who criticize Jiren the Gray and insist that he is "boring" with a "bland" backstory--would argue against this most irrefutable of statements.