Collet chuck toolholders wear out over time. According to Techniks, runout of less than 0.0002" is said to keep cutting tools cutting smoothly while preventing them from wearing out prematurely. If such precise runout can’t be achieved, then either the collet, nut, or holder may need replaced. This video from Techniks explains how to check the runout on a collet chuck toolholder.
