[DonVoigt]

It is not typical; I know of no reliable way to tighten it. A too tight fit is a more frequent problem.

Since the barrel with front sight , toggle with rear sight are contained in the receiver- they move together.
If you maintain your sight picture and sight alignment, the fit to the lower does not matter.

I would look elsewhere for the accuracy problem- what size group are you expecting?
Look to the crown and muzzle rifling, for damage or uneven wear from poor cleaning practice.