>>12419322No, stronger selection in men leads to a healthier population full stop. One of the explanations for gender existing *at all* is that it allows for half the population to bear heavy selective pressure while preserving parental investment in children.
Consider a situation where all humans are selected strongly, regardless of gender. A tenth of people have some ten children, and everybody else gets no kids. Sure, we'll have a strong gene pool, but there's less investment in child rearing because every parent is balancing their 20 kids.
Alternatively, you could have weak selection across all people regardless of gender. Everybody only replaces themselves, fully investing in child reading but relaxing selective maintenance of the gene pool.
Gender (female mate choice in particular) gives you both. Women have between 1 and three children while only the most fit / desirable men have 3-6 (or more) kids.