You are not far from the truth. Luger wasn't a design genius. He was a patenting genius. There is a difference
Paul Mauser, Isidor Loewe and Paul von Gontard thought he was a bit of an **shole, and although he remained on friendly terms with Hugo Borchardt, they had their fair share of professional disputes as well.
The reason that he died broke was mainly his own fault. He made too many enemies in the wrong places so when there was a chance to get rid of him, they did kick him out of the company.