I wouldn’t consider a minty 1939 Mauser 42 code Luger with just an unnumbered toggle pin to be "just a shooter". No way. This is irregardless if the part is a wartime replacement or not. If you showed the firearm to your friends or family, none of them would even come close to noticing this innocuous detail. And most collectors wouldn’t either, just the sharp-eyed hawks on this board.
Of course, if this little detail on your mental landscape would be a permanent irritant, pass on it for sure. Only keep the stuff that makes you happy. Why else collect?