So just to clarify.
Zurtan said:
The client side app is not necessarily running.
Are you only interested on one application on the client side? You don't need a full featured remote administration method, you just want this one application?
If so I really think just have that application run on the client side all the time (like the SSH daemon etc. has to anyway). You can make it run on startup, and it doesn't need to make a visible window or anything until needed. And it can connect to the server (over say TLS!) avoiding some of the networking problems with server→client connections, especially if going outside a LAN environment.
Remote access has a lot of security concerns, especially if those will be other peoples computers, so if it's a single app I don't see the need.
Setting up remote access with generally mean doing something on the client computers / domain (e.g. creating an account with a known password, enabling remote access, etc.) so “install the client app as a start-on-login” seems about the same.
Zurtan said:
Does PowerShell allow me to store the credentials on the server as a file? Or does the User have to enter a password every time he wants to connect via powershell?
Yes you can store credentials. You should never ask for another users personal credentials, so this will generally be for another account/login you create on the domain (similar for Linux) with the relevant permissions.