It is your hand(and the grip safety).
I have the same problem; I find that if I put a layer or two of the soft/fuzzy "loop" side of Velcro on the safety,
it helps prevent pinching and also gives a more secure depress to relieve the sear bar for movement.
And it is removable without damaging the finish.