Faster Loading - for Developers and Whales

Discuss almost anything about RailWorks.

Faster Loading - for Developers and Whales

Unread postby AdamBeckett » Thu Jan 04, 2018 7:50 pm

Workflow Description.

This will help you with:

A) First-time loading into the game in 5-10 seconds! Independent of what kind of hardware you have. (Sadly, not when loading single scenarios, afterwards)

B) No need for any re-/de-installation ever again. All Routes and Assets incl. Locos & Repaints can stay permanently on your harddrives. Yes - m u l t i p l e harddrives.

C) Less crashes due to corrupt scenarios. Also, easier to find and isolate those (will be a separate tutorial, I will provide soon on how to methodically find a corrupt scenario. German language version already available on Rail-sim.de***)

D) Discussing Methods on how to load faster than with (PCIe-)SSD's (no joke).

The whole idea is based on a single premise: Detach the 'real' Routes and Assets folders from RailWorks.exe exectuable and LINK(!) to single Routes, Scenarios and Assets via Microsoft Junctions* **

Image

1.) Install e v e r y t h i n g on multiple harddrives you are running within your PC. For example: Install /Assets/Kuju on Harddrive 1, install all Routes on Harddrive 2, install /Assets/ChrisTrains etc ...

You are no longer bound to install everything on a SINGLE harddrive!

2.) Leave the /Railworks/Assets and /Railworks/Content/Routes folders EMPTY.

Either cut-paste them outside the Railworks folder - distribute them according to 1.) - or rename the original Assets and Routes folder: _Assets and _Routes and create an empty 'Assets' and 'Route' folder.

3.) Now, LINK single Routes, Assets, Scenario folders via MS Junctions.

With this method, you can reduce the necessary Assets and Routes to a minimum. The initial game loading time can truly be as fast as 5 to 10 seconds. It will depend on how many routes and scenarios you are allowing the game executable to load. As you now, RailWorks.exe is (re-)creating the Scenario Database with each new game start, which means, it is scanning the routes for the scenarios and rewriting the SDBCache.bin file.

Any developer, who has to deal with constant crashes during the creation of routes or assets, might want to try this method out, since even 10 seconds faster initial loading after a crash, is saving you life-time?!

What does not affect the first time loading is the size of the Assets folder and sub-folders. RailWorks.exe does not care about them, when you start the game (except for the Kuju folder which must be present).

(My Assets folder is now almost 500 GB big, btw. If you are a 'whale' - a gaming industry term for people, who buy everything available there is to buy, for their favorite game - you will be probably around that number too).

There is even a method to reduce the Reskin folders with "Symbolic Links", since all a folder would need is the LINK to GeoPcDx or bin file, instead creating a copy of everything. But this can become very complicated to keep track off. Space is cheaper than manual fondling, in that case.

As mentioned above, this method won't help you when you are already in the game menu and then loading a scenario. This is still taking (way too much) time - no matter how good your hardware is, or how fast your Intel/AMD CPU is.

There is a command line option to create a PreCache file (which will bloat up your harddrive, crating 1-3GB single files inside the route folder. As far as I can tell (and have measured it), this pre-caching of Assets and Route data is NOT faster then the regular scenario loading.

WHAT IS MS JUNCTIONS?

In laymen terms, MS Junctions is replacing the PLACE OF A FOLDER with a LINK TO THE FOLDER. The folder can be anywhere. A game like Train Simulator is fine if you just place such a SYMBOLIC LINK in the Routes folder, it will them accept the route and not complain that it cannot find it, where it should be.

MS Junctions was created by Microsoft's MVP Mark Russinovich. 'Older' folks, who have been around the early DOS/Windows days, may have heard his name. He was the brain behind "SysInternal Tools" - a toolset that basically ported every useful UNIX command tool to Windows NT.

HOW CAN WE MAKE RAILWORKS/TRAIN SIMULATOR FASTER? A DISCUSSION:

If you have a 'modern' PC, you might already use 16GB or even 32GB (64GB?) of System RAM. A way to boost assets loading beyond the use of SSD - or even the faster PCIe-SSD cards - is to use RAM. Using System RAM as a 'virtual harddrive' is accomplished by creating a so-called RAM-Disk.

DDR4 beats the fastest SSD.

Since not all of the 100-700GB of Train Simulator content can fit on a RAM-Disk, a method using MS Junctions ( = copying only the used Routes & Assets dynamically into the RAM-Disk. Like PreCache.) combined with a Freeware Tool, which automatically does it for you and keeps the dependencies, would be a really nice solution.

Think of a Tool like the "SceneryConfigEditor" for "Microsoft Flight Simulator X" and "Prepar3D" by Andrew Thornton.

His Freeware tool allows the user to create custome scenery.ini files and only load the ones the player actually is using during that session.

Imagine a Train Simulator Tool asking you after each start: "Which Routes do you want to drive today?" and your answer would be one of the pre-configured selections you made. For example: "Only the US Routes". Or "Only the BSNF Routes". Or "Only the UK Steam Routes".

The tool would then pre-load only those into the RAM-Disk and you would have the fastest possible way to load into a scenario... in theory.

So far, with my own adventures into RAM-Disk vs SSD loading have just started. I need to measure the loading times in a reproducible way, so others can compare my results with their own (different people - different hardware, etc).

Also, how feasible it is to write a tool that can take advantage of this 'partial' game data shuffling, is yet to be seen. I am running pregnant to write such a tool for a few years now, but am too lazy to dive deep into the Kuju/Railworks hierarchy.

-- Thanks for reading --

*) MS Junction Article and Download from the official Microsoft Developer Website: https://technet.microsoft.com/de-de/sysinternals/bb896768.aspx

**) Very helpful FREEWARE Tool, which is embedding the MS Junction (and other useful variations) to your right mouse context menu. I use this tool specifically for my RailWorks/TS folders.
http://schinagl.priv.at/nt/hardlinkshellext/linkshellextension.html

***) "Method to eliminate corrupt scenarios which are causing crashes (CrashDumps) on game startup"
http://rail-sim.de/forum/index.php/Thread/28244-Methode-zum-Auffinden-fehlerhafter-korrupter-Szenarios-welche-zum-Crash-TempDump/
(Article by me in German language. I will translate it into English soon. But you can guess the gist by looking at the pictures + googletranslate. I use MS Junctions to narrow the search.)
AdamBeckett
 

Re: Faster Loading - for Developers and Whales

Unread postby AmericanSteam » Thu Jan 04, 2018 10:35 pm

Interesting read.
Just an old Alaska guy trying to live in an insane world. Degree in life, Masters in common sense.
User avatar
AmericanSteam
 
Posts: 2862
Joined: Mon Nov 02, 2015 9:54 am
Location: Nikiski, Alaska

Re: Faster Loading - for Developers and Whales

Unread postby Chacal » Thu Jan 04, 2018 11:14 pm

Yes, although it really needs an introduction.
Over the hill and gathering speed
Chacal
Site Admin
 
Posts: 6513
Joined: Tue Jul 05, 2011 1:11 pm
Location: Quebec, Canada


Return to General Discussion

Who is online

Users browsing this forum: No registered users and 2 guests