There is an objective way to measure the fitness of a trait in a population: If its frequency increases over time, that’s evidence of fitness. There are caveats like genetic drift, but overall it’s a reasonable definition. We can also define the fitness of an organism: The probability that the organism, directly or indirectly, passes on its genes. This can happen by having more offspring or by helping related organisms to have more offspring.
There is no equivalent definition at the population or species level. Even population size isn’t a good proxy for fitness, because that’d imply that krill or the domestic chicken are among the world’s fittest species. It’s also imply that Giant Redwood are less fit than Bishop pines. It’s a silly conclusion.
If there is any meaningful definition of population and species fitness, that’s not going extinct. Everything else, including fluctuations in population size that don’t lead to extinction, cannot count as a measure of fitness.