I'm trying to implement a underwater lighting shader based on realistic scatter and absorption coefficients. Calculating these values depends on knowing the correct wave lengths as input to a table or function. Wondering if I should be using the SRGB waves length (611.3f, 549.1f, 464.3), wave lengths of primary colors (685.0f, 533.0f, 473.0f) or something else. Does anyone know which wave lengths are typically use for atmospheric scattering?

I dunno. You’d think that it would be like Rayleigh scattering, which depends on wavelength, no? The ocean looks blue because of the sky, which is blue because that’s what’s scattered the most.

I'd stick with SRGB colorspace, and thus fit whatever function (it looks relatively linear and I doubt anyone will notice otherwise) to the relative SRGB primary wavelengths; as unless you're doing some sort of ACES colorspace to HDR output you don't need to worry beyond that, assuming the rest of your lighting/etc. is in SRGB as well (a good bet).

The physically correct (and completely impractical) solution is to simulate the entire spectrum, then reduce to sRGB at a later stage. Sunlight isn't a mixture of red, green, and blue light, but a mixture of every frequency in the visible spectrum.

The practical solution is “whatever looks good”. Tweaking the values is a job for an artist, not for an engineer.

I looked through a frostbite paper on atmospheric rendering and they seem to use wavelengths 680 550 440 for calculating scattering coefficients. I guess I'll do the same.

https://media.contentapi.ea.com/content/dam/eacom/frostbite/files/s2016-pbs-frostbite-sky-clouds-new.pdf

Any for of mathematically calculation involving ‘light’ and implicitly wavelength should be done using linear mathematics. Color space conversion etc is really a display issue separate from the actually calculation itself as @frantic pone pointed out.