Most games with physics use the same code on client and server. Thus, most games would use unity3D on the server, too.

If you don't want to do that (you have some reasons not to,) you have to build a physics simulation engine that works in the server environment you choose. For Go, I don't know of any direct physics engines you can download, but you can use cgo to call into an existing physics engine written in C. The "ODE" physics engine has a fully C-callable API, so that might be a good first option.

If that doesn't work out, then you may be able to write your own from scratch in go, too. It's a lot of work, for sure, but it's the kind of thing that experienced game developers end up having to do sometimes when circumstances demand it. It will take a physics programmer 1-2 years to build and debug a 3D physics engine with collisions from scratch with a "reasonable" subset of functionality. (You obviously can't have all the joint, shape, and solver options of a mature engine with a smaller amount of effort like that.)

Another option is not having physics on the server. This means trusting the clients, which means the clients COULD be hacked, but if your game is small and you don't have the time to solve the server physics problem, that might be a resaonable choice. There are also checks you can run against client data, without running the full simulation. ("Is this door open?" "Would this bullet hit report have gone through a wall?" and so forth.)

A third option is to declare one client machine as the "server" and simply use the golang server for authentication and score keeping, and forwarding packets from other players to the "server player."

Whichever option you choose, good luck!