Most DNA, and the variation we observe in it, is non-adaptive. The majority of mutations don’t cause any change to the organism’s functioning. Only very rarely, a mutation will cause a problem, and even more rarely, it will be advantageous.
It’s uncertain how often changes to traits and structures downstream from DNA are neutral. Can these complex structures arise without adaptation, and how much can they vary without causing problems to the individual? Many genes can be deleted without any observable negative impact. This suggests that changes at higher levels of complexity are often also neutral. There’s less data on how finely tuned the size, shape, composition and function of more complex structures like organelles are. The same applies to larger and more complex entities like cells, organs and the entire organism.
As an aside, there are examples of traits like the coloring of birds that aren’t due to natural selection. That’s not the same than saying they aren’t adaptive: Sexual selection is often a superior explanation. The traits are adaptive, but the adaptation is to mate choice, rather than other aspects of the organism’s environment.
Inspired by Arlin Stolzfus’ book Mutation, Randomness, and Evolution, Larry Moran has recently written about different kinds of adaptationism on his blog Sandwalk. Contrary to Moran, I think that the stance taken by evolutionary biologists like Ernst Mayr and Richard Dawkins is hard to dismiss: For highly visible traits, adaptation remains the best explanation. While it’s easy to refute adapatationism on the DNA level, as Moran often and convincingly does, it’s harder to argue for the same for downstream traits.