Hi Bernhard, you have more to do! You stopped to early? After the 180 grit, or even 150 grit, you go to the next successive higher grit number in increments of 100, or less, maybe 80? Use a closed grit paper (3M Wet/Dry works for me) so random grains don't come loose and re-scratch the surface you just sanded? Example, start with 180, then got to 220, then 320, usually this is enough, and finish with a fine blending cloth. Originally it was all done, from machining to high polish finish, on grit charged cloth wheels...
The rule of thumb is sand with the next size grit until you can't see the lines from the previous grit? Then the next, and the next, so on and so on... The trick is removing the blemishes, and still keep all the lines sharp and the flat surfaces flat! Light is important, check your work often in bright sunlight... there is no substitute for that?
Good luck is needed as well, and I can offer plenty of that! The rest is up to you... best, til....lat'r.....GT