If rust is the main problem you are dealing with, there are as many opinions as to how to remove the rust as there are rusty knives. For a knife, what you do depends on how rusted it is, and what the knife is. A 50+ year old slip-joint, for example, you would want to lightly restore. The patina left under the rust adds character, so you generally don't want to just sand it down. A bit of very fine steel wool to remove brown rust, followed by a scrubbing with a toothbrush and some toothpaste over the whole knife would probably be all it needs. If it's a kitchen knife, I'd treat it the same as an axe head. Strip it down to the base shiny metal layer with the wire wheel, then polish it back up and put a new edge on it.
For a general purpose fixed blade like a Mora or something, just use the steel wool to get the dark rust off, and then oil it every now and then to keep it from re-rusting. Or, you can let it sit in vinegar over night to give it a nice dark grey patina that will also resist rust.
To clean rust with a potato, cut it in half lengthwise or crosswise, depending on how large a surface area you want. Dip the cut end in dish soap or baking soda and firmly rub it over the rusted area. If the end of the potato gets slick, slice it off and dip the newly cut end.