Some coffee makers allow the use of permanent filters, such as a finely meshed gold tone basket. You can make a good cup of coffee using one of these, but it's hard to make a great one in my opinion.

A paper filter will do a better job of filtering out the size of coffee particles that produce a silt at the bottom of the cup after drinking.

That being said, the kind of paper matters as well. Some drinkers will opt for brown-colored filters over white due to concerns over how the brown coloring was removed. It's true that some white filters are bleached with chlorine or other chemicals you wouldn't want in your body, but the better white filters are "bleached" with oxygen. These are not only safe to use but can actually remove the metallic notes in a brown filter.

It's hard to really know what else is in your filter even after reading the box and doing some research. The paper taste itself is natural but not desirable. For this reason, brewers will often do an empty brew, if you will, by pouring hot water into an empty filter and discarding the water before the actual brew.

I'm not sure how much of the taste difference is a placebo affect versus reality, but it certainly doesn't hurt to do this just to be sure.

And of course, use the right filter for your equipment. Filters can look similar and can be made to work in systems they were not designed for, but this can have surprisingly negative results. Filters are designed not just to fit the mold they are placed in but to control the pace and direction of the water in concert with the system. Slowing down or speeding up this process alters the ratio of dissolved solids and extraction of the bean.

Also see ../heat-light-and-air regarding materials used to hold the filter.