128 player instance

I wanted to recommend prioritizing doubling up the server size. 64 players isn’t quite enough for the amount of default content. Or alternatively, rotate some kind of bonus to attract players between the attractions. Though the latter could be a little lame. Recommend instead less servers with double capacity. It may seem like “well there aren’t enough players to need more”. This is a fallacy. If you build it they will come. In fact, most days that I have played this player count has filled to 64 then players spill off on to smaller servers and log out. This means you have self inflicted attrition. Also, you can cut the time zone specific servers - that has absolutely no application to your business model. Use that funding to bump the servers up to 128 and watch the magic happen.

as far as im aware the 64 player limit is because the game engine simply cannot handle much more players with how much is happening in this game, not to mention its already incredibly laggy with the current limit, mostly in part due to the unreal engine server applications being single threaded & not to mention all the stuff that can happen in plazas that needs to be kept track of by the server (arcade games, casino games, etc)

this is what caboose (primary server management dev) said about it in the discord in march 2022:

We did an engine upgrade a few years ago to incorporate the improvements Epic added for Fortnite, while also getting better server hardware. Unfortunately, though, we were only able to go from 40 Players to 64 even with those improvements.

it sorta seems like you’re assuming a lot in this post also, especially this part

there’s servers in multiple regions because people want to play on servers closer to them, either to socialize with people more local to them or simply to have better ping. though ill also say that the devs are planning something like “server instances” where servers are created on demand & aren’t gonna be listed in a menu for example

7 Likes

I agree with this, especially if Laser Tag remains server-specific. (IMO)

There’s also the possibility of setting it up to boot people for being AFK too long. Sometimes there’s a full server with 10 AFK players and then you hop over and see someone in a 10 player server trying to rally interest in ____ .

The devs have experimented with 100+ players for plaza, and hell- even game worlds (shoutout to when there was no player limit on Planet Panic alpha). And in doing so, a lot of fundementals for plaza begin to break.

Transactions began to choke. (This included Casino alongside stores).
Playermodels were taking disgustingly long to load.
Most player cosmetics weren’t loading at all.
Players were warping everywhere.
Crashes became common.

As it stands, it’d make plaza into an unplayable state due to the networking that Tower uses for fake physics for bowling, arcade and now for casino. As well as transactions with storefronts, casino and fishing. (Yes fishing is technically a transaction of items).

Maybe one day we can, but we just can’t right now.

3 Likes

I wonder how it would impact UX to chop the plaza into 4 quadrants that allow up to 64 players for a total of 256 on the map. then display in the plaza list all 4 numbers instead of one total population. Maybe that would be too geeky, could mock it up and see how it feels

ah you are right! especially the locality issue, I forgot about the data locality. Looks like there is non commodity server hosting with 5.5GHz cores. IDK if it would be cost effective. maybe feasible to rent one box with 5.5GHz cores for some kind of event with 200+ concurrent users (assuming the beta test goes ok) and see if the event can pay for more 5.5GHz boxes for other localities

This makes an insane amount of assumptions that are not realistically feasible, setting up a server limit to be double isn’t as easy as magically obtaining the obscene amount of money for the upgrade, it would require a whole rewrite of back end dev, requiring months to maybe even years of work. On top of that the servers run 24/7 as someone who tends to play late, I have seen no more than 16 people the most populated servers at the time. Averages are key you can’t do everything by peak, that’s the point of running multiple concurrent servers.

Also the game already has issues running as is due to poly counts, poorly optimized models, and other user created content that Pixel Tail has no control over! Upping the game to 128 players, killing a bunch of localized servers, and cutting back on the number of the server will create borderline un-runnable games for a bunch of people. They can try to play with 350 ping because the Australian servers with cut while getting only 15fps on their 1060 because it’s rendering 30 super poorly optimized models, but it wouldn’t be worth it.

You need to be aware that this is never as easy as you think, you can’t magically change a number, call it day, and hope it works out! Every change has the chance to ruin a bunch of things in a cascading cavalcade of hell!

Hell you even try to argue “if you build it they’ll come” without realizing the real potential to alienate and cripple the current players of the game who use it to hang out with friends who aren’t running high end spec PCs or folks who want to bowl without some guy sprinting around banshee screeching slurs while making high pitched sounds with the wowozela item. Sometimes it’s nice to log onto a server with 8 people and relax a bit.

2 Likes

I believe the eventual plan is to “merge” all the plaza servers into one server, where you have a set amount of players on the specific server you’re on, but every other player on every other plaza instance will be visible in some sort of abstracted form (simple model with nametag, updating every x seconds rather than in real-time), and you can hop on over to another player’s server instance if needed. This, plus the eventual return of the global chat system, should make plazas more populated while remaining within the limits of the engine.

I think it was also mentioned somewhere that major plaza attractions (casino/arcade) are eventually planned to be split into separate server instances, which would be created/shut down based on demand. That way, players in the main plaza would be able to group together properly independent of the demand for casino/arcade.

Unfortunately, it is not possible to have 128 player servers.

Each player takes a toll on the game, and the engine has trouble handling 64 players sometimes. To double that would double the amount of CPU, VRAM (texture memory), draw calls, animations, voice chat data, and workshop downloads on your own game. You’ll need way better hardware. On top of that, the Unreal 4 dedicated server is single threaded, which means it’ll not be able to handle that many players per instance, even if you throw it on a huge server. You’ll also be doubling bandwidth across the board as well.

Our plans are to separate the Plaza, Arcade, and Casino into their own servers. You’d have the Arcade and Casino on dedicated servers that would connect with the Plaza dedicated servers. This is something we are currently working on. By doing this, you’ll have more players in the Plaza (64) as the other players who are busy playing Casino or Arcade would be in their own instances. We plan to make this as seamless as possible, so you would just enter the doors to the Casino like you already do, and it’ll just connect you to it. By also having global chat (or even just Plaza chat) synced with the Casino and Arcade servers, it’ll be possible to maintain communication with other players. We’d also need to show the players in the Arcade/Casino servers on your scoreboard as well, which is something of a challenge.

Essentially though, in order to get more players, we’d need to cut back on a lot of player interactions - like MMOs do. Then we’d have to create MMO software. This is not what we envisioned with Tower. We don’t want to sacrifice the player interactions that Tower has to offer just to have more players per Plaza.

One thing we are also working towards is a Plaza quick join button and UI changes to the Plaza server list. Eventually, you’ll not use the Plaza server list and you’ll just select your region and hit quick join and it’ll put you in a populated Plaza. This will help manage player counts better and get people into Plazas quicker. There’s a bunch of smaller details to this feature that we are still working on, such as ensuring players can find their friends as well.

8 Likes

oh i see, so even if the servers are beefed up, there is still the issue of player’s with old computers. I think as far as rendering, that may be fixable with an auto tune graphics feature, e.g. instead of static graphics settings, an option (on by default) to auto tune down shaders, etc., even automatically un-subdivide player models as they load depending on the rendering score of the users computer. Doesn’t need to be super fancy logic, can just be a function of the user’s render time per frame.

e.g. every 3 seconds check LastRenderTime, LastRenderTime | Unreal Engine Documentation if render time great, keep shader settings, etc.

then you can still hang out with your friends with good render settings, if it gets crowded the quality drops if your client can’t handle it. would that be worth having more users to play with on individual features and more pug action?

This would not help entirely. Players take up CPU, more than GPU. The GPU and Unreal, already do a lot of stuff behind the scenes to ensure the rendering is as fast a possible. However, you still have to calculate on the CPU all the animations (among other things) which is extremely expensive. There is no easy fix. To go from 64 players to 128 players is a 2x increase of budget across the board.

7 Likes

Feels like my pc would catch on fire from 128 players at once

have mercy, my PC is frequently on verge of death with 64 and graphics settings don’t alter this sort of load at all

Arcade and casino isn’t very crowded as plaza. I think all closed buildings can have one decided server. (One server for arcade, casino, night club, theater, laser tag, bowling)

And when this implemented servers can be merged into one per region.