Building your cloud gaming stack 101

In order to provide a cloud gaming service, multiple elements have to be put together for the service to work. Many players are jumping in the band wagon. I’ll disect what I believe are the key technical layers of this offering, how they interact and where the culprits are.
As you will see, while many people are comparing this to “video streaming”, this is quite a complex scenario which may go sideways in many places. I may have missed a few things, feel free to reach out and let me know, I will be happy to correct any statement which by the way represents my personal view on how things are moving forward.
Let’s start easy. The first diagram shows the 3 main pillars of cloud gaming. You have the player on the left. She sits home or in a bus, she plays games with her friends through this new trendy service. In the middle we have the cloud gaming infrastructure. That’s where the actual game will be rendered, where 3d are calculated and where the number crunching happens. Last one on the right is where game specific communication services happen. As most games call for communication between players, this is where players will be matched, where you store stateful data, content, etc. While the last 2 could be seen as a single one, they are split from a technical perspective since cloud gaming services are not making game and have different view/needs. As I’ve shown in a previous post, there are technical reason why merging those would also not make much sense.
If we go one step further, we see that each of those is made of a mix of hardware and software. Each rectangle below represents a high level component needed for the service to work.
Starting from the left, the player will use a controller in the form of a joystick, a mouse/keyboard (or a kinect if microsoft wasn’t killing it) to tell the system what to do. In most cases, this controller is connected to a local device. Stadia came up with a joystick going directly over wifi to skip that layer (even though the saving must not be overly great considering wifi latency). The controller could also be the touch pad of a screen (i.e. tablet/smartphone). Next comes the client. This “thin” client can be access through an app, a browser and such. Its role is to display what happens in the cloud gaming infrastructure. It is tweaked to decode encrypted/encoded video streams quickly and most likely has very few other capabilities. The only one we could think would be around synchronizing what’s display vs. what will be rendered (i.e. v-sync). Next comes what will run this thin client. A phone, a smart tv, a PC and even a console. I’ve seen people use a raspberry pi for shadow service. Those devices have to be connected to the internet. That’s done through a personal network (home wifi router, wired, phone tethering). This private network will have access to a broader network through standard access like RF (i.e. LTE), fiber, dsl, cable, gpon, etc.
We are now on the internet where anything goes. MPLS, guaranteed route, qos, traffic shaping, throttling (!).
Next step, the cloud gaming infrastructure. From the top, the actual game will run in an os environment, the same way it would run at your home on your pc/console. Most offering today use windows to support existing game. Stadia is going against the grain as they use linux based os so are asking studios to re-compile their games to support this infrastructure. Linux community will be happy about this, but this may cause stadia offering to be limited initially. The next stop is the OS, which run in a virtualized environment. Most, if not all, use windows virtual machine today. I suspect stadia is pushing for linux to go the containerized route which imho makes a lot of sense (leaving alone security question here…). Those virtual environments/clients will access the network through a virtual network. Based on which infra is used, pick your poison. In every case, you added another layer and some may be better than others feature wise. Games have different need than web services, and not everyone seems to understand that. Next layer: hypervisor. Again, pick your poison but since the open source offering is quite mature and corporates one are expensive, if you build your own cloud gaming infra you will most likely go with openstack (unless you use a public cloud). Below is the hardware layer where cpu and gpu provided a service. This service calls for super low latency, so you are better off with multiple smaller sites closer from players vs large server farm. In which case, density is key, so high core cpu and GPU cards more expensive than my car are the way to go. Some gpu vendor (euh-hum hi nvidia) need a special software to limit how many vm can be leverage per gpu. I’m not super familiar with how it works but that’s definetely to consider. We then get in the network where your private network is going to be connected to the internet. As fast as the fiber connection may be, most boxes will add hops here and there.
Last but not least, gaming specific services. Those are match maker, multiplayer games, chat/communications, inter-connections, stats tools, etc.. Each studio has a pletora of tools they use, and most of the time those are needed for a game to work. Every modern multiplayer game use a server to interconnect players, and connecting them is done through a match making service. Without going into lengthy details about the specific of those services, let’s just remember that some of them are critical for in-game activities, well beyond offering you to buy a pink polka dotted shirt for your character. Those services, like any cloud application, will go through the same hoop of virtual layers over virtual layers over physical layers over and so on. You get the point.
Alright we’ve listed the high (maybe not so high anymore) level layers. There are variances for some of them, but I’m pretty sure we’re hitting 95+% of the examples out there. Who does what? I don’t want to list here every actor for every layer, but I regrouped them in 4 buckets. Manufacturers, Connectivity/cloud infrastructure, OS/software folks, and last but not least cloud gaming folks. I’m including gaming studios in the last group, and while we can debate as to what’s gonna happen in the next few years, last time I check studios don’t spend much time on infrastructures, their concern being the game itself.
Now to my favorite part. Where exactly will the train derails. Below is the same set of stack, and every places where lag can be introduce. What’s worst? my last layer at the bottom is wayyyyy more complex than i’m showing.
Some items are adding latency more than others. Milliseconds vs microseconds makes a huge difference, but when you look at it from this point of view, they all add up. One comment I’ve seen about those cloud gaming services is that they will offer this worldwide. Truth is, it will probably be bearable only if you’re located close from a data center or in a major metropolitain area. In which case, you most likely have a higher home income than those far from metro area, a faster internet speed, and enough money to get yourself a console or a decent PC. This service would be useful for the other type of players, those without enough money to purchase a high-end pc. Issue is they won’t have the connectivity to get this service.
I’m not saying cloud gaming will not happen, i’m just showing that there are many locations where innovation and improvement will be needed. The main one is around dynamically orchestrating this. Each scenario will be different, and getting this right will be done case by case. That’s most likely why Google Stadia went for linux based core and are asking studio to convert their game. This will allow them to use many existing tools (hello kubernetes). That’s also why they are investing in open source projects around that (See Agones).
My last diagram goes a bit into what type of stack could be used to build quickly a cloud gaming offering. This would not be really innovative but you could have something up within a day or 2. From the bottom up, you get multiple locations connected straight in a given ISP (hello edge computing!), deploy x86 servers with high density cpu and gpu. Make sure you get high Ghz as core is not everything in gaming! You install your prefered hypervisor. A good choice is probably Openstack to start with. While not easy (oh dear) to install, it has many large users and I’d hope the project is mature enough now. At this stage i’m hoping you have the whole stack running. You create a virtual image based on windows with some gaming client /store pre-installed (my number 1 is steam store). At this point you only need a website which will support openstack API and credit card to allow your user to create their PC. voila. Add salt and pepper, and send me a check if you make money. As silly as I am, that’s really all there is to it. Now, as i’ve spent the whole article above pointing out, devil is in the details. Tweaking every red lines I’ve shown above is what will make this succesful.
Edgegap can help you build and improve your solution to get the most of edge computing and reduce the amount of red lines as much as possible!
Mathieu Duperre
mathieu@edgegap.com