The goal is to lightly polish the shaft so that it rotates freely and smoothly in the base tower. The only real solution is to use 120 grit or finer sandpaper (preferably emery cloth from the hardware store) and lightly polish the clamp shaft using the penetrating oil as a lubricant. Emery cloth is best since it will last longer when coated with oil. It would be difficult to remove more than 0.0001" of the shaft diameter when polishing by hand. Any small amount of metal that is removed will not negatively affect the performance of the clamp in any way. Clean the shaft before testing it in the clamp tower. Repeat the polishing and test fit steps until you are satisfied with the fit.
As noted in the previous post, it is possible to make an inexpensive tool to polish (hone) the inside diameter of the base tower if needed.
Irvin, please state specifically why you think this process will not work. Every machinist or mechanic uses emery cloth to solve similar problems on all types of equipment.