🎉 Celebrating 25 Years of GameDev.net! 🎉

Not many can claim 25 years on the Internet! Join us in celebrating this milestone. Learn more about our history, and thank you for being a part of our community!

Building scripts in multiple-threads.

Started by
4 comments, last by Asu 4 years, 9 months ago

Im curious to know if you have any current plays to implement being able to compile/Build threads from multiple-threads. In comment you say it is 'possible' curious if u had any plans on implementing it any time soon?

Advertisement

Right now I don't have any plans to implement this. I really don't see enough benefit for it to justify the effort for the implementation or the additional complexity in the compiler to guarantee threadsafety on all internal structures.

Chances are that making the compiler multithreaded is only going to slow it down. This is because the compiler is purely CPU limited, with little to no wait-time to be exploited by multiple threads running in parallel. 

Is there any particular reason why you are looking for this support?

AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

I was actually looking for info about this today. My only interest into this was to cut down loading times.
One way I was thinking I could implement this (in the game engine) is by spawning multiple engines in parallel, registering the interface for each of those, feed them scripts and copy bytecode to the main thread.
That being said compile times in AS were vastly improved "recently" (possibly several years ago), so I don't think I'm willing to bother implementing it to shove off an extra few percents.
Interestingly enough, enabling LTO results in a big performance increase for compilation times (25%+).

Compiling in one engine and then transferring the bytecode to another can be done, but increases the amount of work needed to be performed by the CPU, as you would have to save the compiled bytecode from the engine that compiled the script, and then load it into the target engine. I'm highly skeptical to this providing any performance benefit, over just compiling all of the scripts in a sequentially in a single thread.

I would be interested in seeing someone prove me wrong though :)

 

Have you tried caching the compiled scripts by saving the byte code to disk, and then in the next run instead of compiling again you load the previously saved byte code? That would definitely improve the load times (except for the first time)

 

Yes, I can imagine LTO when building your application will give you some performance benefit, since Link Time Optimization can do global optimizations that cannot otherwise be done locally. I'm surprised to see a 25% performance improvement though, that is much more than I could have imagined :)

AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

18 minutes ago, WitchLord said:

Compiling in one engine and then transferring the bytecode to another can be done, but increases the amount of work needed to be performed by the CPU, as you would have to save the compiled bytecode from the engine that compiled the script, and then load it into the target engine. I'm highly skeptical to this providing any performance benefit, over just compiling all of the scripts in a sequentially in a single thread.

I would be interested in seeing someone prove me wrong though :)

I would expect the overhead of saving and loading bytecode to be relatively low, but I could be wrong (this is not my code base, after all ?).

 

18 minutes ago, WitchLord said:

Have you tried caching the compiled scripts by saving the byte code to disk, and then in the next run instead of compiling again you load the previously saved byte code? That would definitely improve the load times (except for the first time)

Yes, but only using a much older version of AngelScript (2010-era). It unfortunately did not work well. I do not remember the details, but I remember dealing with problems related to shared classes and other crashes.

I have seen from the changelog that there were a lot of fixes and changes related to bytecode saving/loading, but I have not tried on newer versions. I could try again, but not so easily, because a lot of code has changed between my initial implementation for bytecode caching and the head of my branch for the game.
In the end I am not sure that I would want to deal with loading/saving bytecode (especially from the disk or from the network) for the already okay loading times - 6s for the entire game to start and load a local game on a T9600, without LTO. I believe it also compile scripts that are not in use by the gamemode.

18 minutes ago, WitchLord said:

Yes, I can imagine LTO when building your application will give you some performance benefit, since Link Time Optimization can do global optimizations that cannot otherwise be done locally. I'm surprised to see a 25% performance improvement though, that is much more than I could have imagined :)

I have not really formally verified the results, but loading times of the entire game (in the same scenario as above but on an i5-4670K@4.4GHz) are reduced from 3s to 2.3s on average. In my testing, basically all of that time is script compilation.
Interestingly enough, the rest of the game does not benefit nearly as much from it.

This topic is closed to new replies.

Advertisement