That could be due to texture streaming, a feature of Unreal Engine. Try putting -notexturestreaming in your startup settings in Steam for Tower Unite. Be warned: this can cause increased VRAM usage.
-notexturestreaming