In this post, we look back on the virtual keyboard project that I worked on for xrdesktop as a participant in Google Summer of Code 2021. It was a great experience and a good introduction to open source development.
Current Status
The project was very successful in its goal of providing a text input method for xrdesktop independent of SteamVR’s virtual keyboard. The keyboard is usable for people in 56 languages and it works with KDE. You can try it out by compiling the ‘next’ branches of the various libraries. See below for detailed instructions.
How to Run
You will need to compile and run it for KDE. Follow this how-to guide to install xrdesktop from source. However, you need to use different branches in a few cases:
- gulkan: gulkan:next
- gxr: gxr:next
- xrdesktop: remcokranenburg:keyboard-internationalization
- libinputsynth: remcokranenburg:keyvals-input
- kwin-effect-xrdesktop: remcokranenburg:virtual-keyboard-support
Future Work
During our feedback sessions, my mentors and I decided to focus first on making the basic keyboard experience complete, by implementing it for a desktop environment and providing keyboard layouts for a lot of languages. In the revised planning, the prediction mode was postponed.
A start was made on a predictive mode that would work like the swipe keyboard mode on Android. Post-GSoC, I’d like to continue this work. When finished, I’d also like to investigate whether the same technology can be used to add a swipe mode to GNOME Shell’s on-screen keyboard.
The third and most experimental mode was dropped for the time being. It entailed a Dasher-like interface adapted for a virtual reality environment. Since VR has such a different way of interacting, a keyboard input method might not make the most use of your abilities. The hypothesis is that a Dasher-like method might be faster when used with six-degrees-of-freedom controllers.
While implementing keyboard layouts, I learned that languages with non-alphabetic writing systems, such as Chinese, need a different way of handling keyboard input. They need to be able to write tentative words, that can be committed by additional key presses. This feature could potentially be based on the prediction system, because that also predicts tentative words, which can be committed by choosing one of the proposals.
Merge Requests
Some warm-up bugfixes / improvements:
- MERGED xrdesktop!24 Improve menu position when attached to controller
- MERGED xrdesktop!27 Bump minimum meson version
- MERGED xrdesktop!28 Client to shell example
- MERGED xrdesktop!30 g3k-button: correct calculation of background circle radius
- MERGED gulkan!15 Add pango example
- MERGED libinputsynth!4 gitignore: ignore build directory
The main merge requests, implementing the G3kKeyboard and making it work in KDE:
- MERGED xrdesktop!31 Add virtual keyboard
- OPEN kwin-effect-xrdesktop!4 Support G3kKeyboard
Implementing multiple keyboard layouts:
- DECLINED libintputsynth!5 Keysequence input
- OPEN libinputsynth!6 Keyvals input
- OPEN xrdesktop!33 Keyboard internationalization
Blog Posts
This project also prompted me to finally start a blog:
- Getting Started with GLib
- Introduction to xrdesktop
- Keyboard Event Handling
- Emoji Mode ✨
- KDE in VR with Keyboard Support
- Keyboard Layouts
Final Words
I loved working on this project. I’ve always been interested in computer graphics and virtual reality, so this was the perfect fit for a start in open source development.
I would like to thank my mentors Christoph Haag and Lubosz Sarnecki for their fantastic support, and Google and Collabora for providing the opportunity to work on such a cool project. We will see each other again, I hope!