Bringing headless remote desktop to Wayland
Last September we published Wayland, TigerVNC, and ThinLinc: The future of remote desktops in Linux. In it we were honest about something uncomfortable: under Wayland, it will never be possible to write one remote desktop application that works with every compositor, because the remote desktop pieces live outside the Wayland core.
One of you replied with exactly the right wish:
I wish Wayland could include a remote protocol or at least that there could be one extension that works over any desktop environment that could become default on all flavours of distributions, so you don’t have to choose between a subset of popular distributions and leave all others behind.
That is the problem we have spent the past year working on, and we now have something concrete to share. We are betting on a draft specification called RemoteDesktop1, we are working on it together with GNOME and KDE, we’re building towards a wlroots implementation, and we want it to gain enough traction to become the de-facto way to do headless remote desktop on Linux.
1. How this started
One of our developers was digging through the state of remote desktop under Wayland and found a draft specification that Jonas Ådahl had been working on. Jonas maintains Mutter and works on GNOME at Red Hat, and his draft described something much closer to what we actually needed than anything else out there.
So he wrote to Jonas and asked for a meeting. Three of us drove to Gothenburg - our product owner Pierre Ossman, the developer who found the draft, and me - sat down with Jonas, and talked for six hours straight. It was one of the most constructive technical conversations we’ve had in years. Anyone who has worked with Pierre knows he is not easily impressed, and on the drive home he sounded hopeful about this problem for the first time. We decided there and then that RemoteDesktop1 was the approach we were going to invest in.
Everything since has followed from that. Our engineers have been building and specifying, and alongside that we have spent a lot of time finding the right people in each project, getting them in the same room and keeping them talking to each other. Both halves matter. A specification with one implementation and one interested party is not a standard, and getting from there to something the whole Linux desktop can rely on is as much about people as it is about code.
2. Remote is not the same thing as headless
This distinction sits at the heart of the whole effort.
A remote session is a graphical session you can connect to from somewhere other than the machine it runs on.
A headless session is a graphical session that has no physical display attached to it at all. No monitor, no seat, nothing on the end of a video cable. The session exists purely in memory and its pixels go over the network.
The first case is in reasonably good shape, and we’ve contributed to that ourselves. Over the past year we’ve built Wayland support into TigerVNC, which we maintain, and TigerVNC 1.16 shipped w0vncserver in January: a Wayland counterpart to x0vncserver that lets you connect to an existing local Wayland session, using the RemoteDesktop and ScreenCast portals on GNOME and KDE Plasma, or the native protocols on wlroots-based compositors such as sway, labwc and Wayfire. If you want to reach the desktop already running on your own machine, that works today.
The headless case is the one that’s still open, and it’s the case ThinLinc is built around. On a ThinLinc server there is no monitor. There is no user sitting in front of the machine to click “allow” on a portal dialog. Fifty people may be logged into their own independent sessions on the same host, each with a different resolution, each reconnecting from a different device tomorrow morning. Under X11 this has worked for decades because we ship a headless X server, Xvnc, and the desktop environment neither knows nor cares that no hardware is involved. Wayland has no equivalent, and the compositors that do support a headless mode each enable it their own way: wlroots uses an environment variable, others take a command line argument, and there’s no way to discover which is which.
So the gap isn’t “remote desktop on Wayland”. The gap is standardised, headless, multi-user, unattended graphical sessions on Wayland. This is not an exotic requirement. Shared servers have been a mainstream way to deliver Linux desktops for as long as Linux has had them, and today they run engineering departments, trading floors, research computing and public sector IT, often with hundreds of users on a single host. X11 made it straightforward, so it has quietly worked for decades.
The reason the Wayland story is behind isn’t that nobody cares. It’s that the effort quite reasonably went first to the case a desktop developer meets on their own machine: sharing or remote-controlling the session in front of you. The server case needs someone to bring it to the table and stay there until it’s designed in properly. That’s us.
3. What RemoteDesktop1 is
RemoteDesktop1 (RD1 for short) is a draft D-Bus interface authored by Jonas Ådahl and proposed as an addition to the xdg-specs. It’s a session bus API that a compositor exposes and a remote desktop server consumes.
Where the existing portal answers “may this app watch my screen?”, RD1 answers a different question: “how does a remote desktop server drive a session that has no screen?” It covers creating, listing and removing virtual monitors, connecting to the resulting video streams over PipeWire, injecting keyboard and pointer input via libei, and clipboard handling. For the moment, a session that offers RD1 on the session bus is implicitly a headless session.
If RD1 lands and gets implemented widely, a remote desktop server speaks one API to GNOME, KDE, sway, labwc and whatever comes next, instead of one integration per compositor. That’s the “one extension that works over any desktop environment” our community asked for. Nothing about it is Cendio-specific, and that’s the point.
It is still a draft, and it will change as more people implement it. That’s an argument for getting involved now, while the shape of it is still easy to influence.
4. Where we are on four fronts
A specification needs more than one implementation before anyone can call it a standard, so we’re pushing on four fronts at once.
The specification, and GNOME with it. We work on the RD1 spec directly with Jonas, in the open in merge request 115. Our job there is to keep the API general and genuinely headless-capable rather than shaped around any one compositor. Because Jonas works on GNOME at Red Hat, the spec and the GNOME implementation move together: progress on one is progress on the other.
Jonas put the ambition like this:
The idea is that it should be able to handle both what was possible before in the Xorg world, and the things that weren’t possible because of how X11 worked. So making sure that’s the case for your products too, and not just the ones I keep track of myself, would be good.
— Jonas Ådahl (translated from Swedish)
KDE, with TechPaladin Software. We’re funding an RD1 implementation in KWin, built by TechPaladin Software, who maintain large parts of KDE. A draft implementation is done and in real-world testing on both sides, it has been announced by David Edmundson on the KDE Plasma development mailing list, and the intention is to upstream it. What’s left is testing, the adjustments that come out of it, and landing the GNOME side so the two can be checked against each other. This was never meant to cover everything RD1 will eventually need. It was meant to prove the specification works by making it work, and so far it does.
David says:
In Techpaladin, our mission is to develop KDE software in directions that are both directly useful for our clients, and also good for the ecosystem as a whole. Our work with Cendio to standardize Wayland remote desktop interfaces reflects that mission — ultimately improving the Wayland remote desktop experience for users, administrators, software developers, and vendors.
— David Edmundson
wlroots. Madeleine Nilsson (@madni) is working out what an RD1 implementation could look like for wlroots-based compositors such as sway, labwc and Wayfire, and how much of it would belong in wlroots itself rather than in each compositor. Our plan is to build a prototype first and bring that to the wlroots community, rather than turning up with a proposal and no code. This is the front we’re furthest from finishing and the one we care most about. Two large desktops implementing a specification is a duopoly. A third, independent implementation is what turns it into a standard, and it is the honest test of whether the API really is as general as we think.
Session startup. Even with RD1 implemented everywhere, there is still no standard way to start a Wayland session headless in the first place, and no way to find out how a given desktop wants it done. Nobody owned this problem, so we took it on. Alexander Zeijlon (@Zeijlon) has a draft merge request out proposing how a session declares what it supports, designed so that desktops don’t have to change how they already work, and so that systemd isn’t a requirement. It needs review from the people who start sessions for a living.
5. Why we’re doing it in the open
Working in the open isn’t a concession for us, it’s how we build. Our product stands on open source, we maintain TigerVNC, noVNC and rfbproto, the community specification of the RFB protocol that documents it as it is actually implemented today, extensions and all, and we’ve been part of this community for a long time. When we need something from the Linux desktop stack, the natural move is to go and help build it where everyone can use it. Our competitors get the benefit of work we pay for, and that’s fine. We’ve had the benefit of work they and many others paid for.
We also think this is simply the better engineering. One well-specified API that every compositor implements is more robust than a pile of vendor-specific integrations, each one a private arrangement between one company and one desktop, each one breaking on its own schedule. A standard gets reviewed by people with different constraints, it gets tested against implementations nobody planned for, and it survives the companies that started it. That’s the thing worth building, and being in it early is how we make sure headless multi-user sessions are part of what it’s designed for.
6. Where the conversation happens, and how you can help
We’re aware that most compositor developers aren’t reading a ThinLinc forum, so the real work happens where they already are: merge request 115 for the specification and KDE’s GitLab for the KWin implementation. If you work on a compositor, a desktop environment, a display manager or a distribution, that’s where we’d like to meet you, and we’re happy to come to you instead. Ask us for a meeting the way we asked Jonas.
That said, this post has a request for everyone reading it:
-
If you maintain a compositor or desktop: read the draft and tell us whether it’s implementable in your architecture, and what’s missing or wrong. If you want to implement it, say so and we’ll help you do it.
-
If you work on display managers or distribution packaging: review Alexander’s session startup proposal and tell us whether it fits the way you actually start sessions. This is the piece that most needs your eyes.
-
If you work on another remote desktop project, including the ones we compete with: come and implement RD1 against your own requirements, and tell us where it doesn’t fit them. The specification gets better the more different needs it has to satisfy.
-
If you’re a ThinLinc user or administrator: you don’t need to know anything about Wayland to help. Tell us which desktop environments your users run, and what they’d notice immediately if it changed. That’s what decides which parts of this we push hardest on, and several of you already shaped our priorities that way in the thread on our last post.
7. What happens next
The next steps are concrete. Finish the KDE prototype and land the GNOME side so the two can be tested against each other. Get a wlroots prototype far enough along to show the people who maintain those compositors. Get the session startup proposal reviewed by the people it affects. Keep the specification moving toward something official.
We can’t put a date on Wayland session support in ThinLinc yet. Too much of it depends on work that isn’t ours alone. ThinLinc still needs an X11 desktop environment today, and with X11 support disappearing from newer GNOME and Plasma releases, closing that gap is the most important thing on our technical roadmap.
A year ago this was an open question with no obvious answer. Today there’s a specification, two implementations in progress, a third being designed, a proposal for the missing startup piece, and a group of people from different projects who talk to each other regularly about it. That’s how a standard starts. We’d like more company on the way there.