This is in fact a security feature, related to assembly deployment and clickonce stuff. I think you just need to add a strong name to the main assembly to overcome this issue.

If you don't have a strong name, the name will be something like this and it will change if the application path change (see the Url token):

%appdata%\WindowsFormsApplication1\WindowsFormsApplication1._Url_3pei3cdnq3srqpjiwh1qnf12ncsp5c2w\1.0.0.0\user.config

If you have a strong name it will be something like this and it will not change because your app origin is now identified, whatever its current location is (see the StrongName token):

%appdata%\WindowsFormsApplication1\WindowsFormsApplication1._StrongName_fe0ndyau2vlgeac4gmbg13u3q4jtyrqv\1.0.0.0\user.config