AGE, I was just thinking ( a dangerous thing). Could your accuracy problem be the muzzle crown? I would hate to see you file and grind to chase the ball. When the ball could be sent on its way properly. I am no gunsmith , just a thought. It might settle down and center if the gun has not been fired. The people that owned it previous could have used a steel cleaning rod and in thier haste could have damaged the crown. It just seems to be way off for so close. Again, I am no gunsmith. I drink to much to play with pressures that could hurt me. I hope you run another hundred round through it before you make your decision.
P.S. After I retired and realized I could not afford new toys, I decided to shoot my collection as if they were shooters. My Colts shot straight. Very straight. My most recent one was made in 1863. Even the 49's shot straight. They were right on or within an inch at 25 yds with a rest.