I mean the human equivalent is "race". There is genetic backing to this because ancestry can be traced, and there are different phenotypes.
One standard that evolutionary biologists consider to be defining of a single species is that ability to naturally mate (not forced by humans), and produce grand children.
However, Neanderthals bread with current Homo sapiens are considered a subspecies of Homo sapien (Homo sapiens neanderthalensis).
I think that society pushes back about considered humans to be subspecies because it can rationalize hate/war etc.
This is just my opinion.