Idler gears don't give up...they can cut into the casing thrust face.
It tends to be more the 1st motion shaft the whines, as it sits in the big bearing, has a pilot bearing on the nose and the mainshaft end bearing inside, with the input gear (bottom drop gear) on the outside and 4th on the inside.
As the bearings wear it starts to misalign the input gear and idler on the outside and on the inside end 4th gear which is in constant mesh with the bigger 4th on the laygear, hence the whine.
The whole power and drive of the car has to go through....primary, idler, 1st motion, then laygear then 1st 2nd 3rd and then the diff pinion/ crown wheel.
In 4th gear its a direct top so the drive all goes through the mainshaft by engaging 4th directly onto the 1st motion shaft end. This makes the drive a straight line from the input gear to the diff pinion.
With less side-loads and gears under power, this makes the whine less in top gear and more noticeable in lower gears.
Anyone notice the difference between an A series and A plus whine ?