So I cooked up a 3D printed design with some 3-axis sticks and as many buttons as I could manage on an ESP32. It's partly inspired by the 3D printed controllers James Bruton and Keegan Neave have made but ESP32 based.
With a screen I happened to have fitted and the encrypted 2-device ESP-Now library I'm working on being bidirectional (another side quest) it should be able to give lots of info on the rover while it's in use like battery voltage, location and so on.This might be of use to others (although the design's quite specific to that screen) so I've made it available on GitHub and christened it Sticky.