Am I barking up the wrong tree?

After a successful test of painlessMesh at the weekend I posted about it on the ESP8266 Facebook group and somebody wondered if I'd considered ESP-NOW.

Which I may have seen mentioned a couple of years ago but have never fiddled with and completely forgotten.

While painlessMesh is doing what I want I have in the back of my mind that it's layering heavyweight stuff together (AP associations, WPA2 encryption, TCP servers, NTP-like time sync) to give you a simple messaging protocol.

Despite my assertion I didn't want to get sucked into writing my own networking code I had actually been Googling to see if you could send/receive packets directly between ESP8266 stations without actually associating with an AP at all. Most of the chatter about that is building antisocial Wi-Fi de-auther devices, and it didn't look a very fruitful route.

Of course this is exactly what ESP-NOW is. It's a proprietary messaging protocol that's part of the Espressif suite of dev tools for the ESP8266/32. It leverages standards compliant 'vendor action frames' which allow vendors to send frames directly between stations for their own proprietary reasons.

As this is very low level ESP-NOW supports star networks with 20 nodes, rather than the 5 nodes of the full WPA2 encrypted IP connection that painlessMesh is built on. So a very similar 'star of stars' mesh built out of ESP-NOW connections should scale more gracefully. It also leaves some resource for a conventional IP device to bridge in to the mesh and talk to it. It is unencypted by default but you can switch this on and it lowers the capacity to 10 nodes.

As painlessMesh seems to handle scaling quite well, I'm more interested in the greater power efficiency and range promised by ESP-NOW.

If you've ever noticed you can see a Wi-Fi SSID way outside the range you can make useful connections to it, that's the kind of range ESP-NOW promises. It's not doing all the encryption, background keepalive and management work needed for a full AP association, so it's much more tolerant and fewer packets means less power.

Digging deeper there is also ESP-MESH in the dev tools which the documentation promises is a self organising mesh built out of ESP-NOW connections.

Why have I not noticed these protocols before?

The answer seems to be nobody in the 'maker' community bothers with either ESP-NOW or ESP-MESH. I've found a handful of people talking about it but very much proof of concept builds of the example code, no mention of big mesh projects. I guess the IoT spin that's put on the ESP product lines has meant the non-IP protocol suite is mostly ignored.

As this is all changeable in the code I'm going to continue working with painlessMesh, build the rest of my static nodes and start on the wearable GPS-equipped nodes. This will allow me to conduct a bigger test including them at the end of October. If I subsequently find a way to drop in ESP-MESH or engineer my own thing with ESP-NOW it'll be a 'range upgrade'.

painlessMesh testing

Yesterday I had a go at testing six painlessMesh nodes in a similar situation to the one I need them for. As per usual this is for a LARP prop and we almost exclusively play outdoors in heavily wooded areas. As I discovered with the last networked thing I made, trees are very good at blocking Wi-Fi so you really need to do testing out in the woods.

For this test I invested a chunk of effort into building these nodes fairly robustly into the IP55 box you can see in my first post about them.

Testing based off almost finalised builds has a risk of being a dead end where you've wasted that time in making them almost complete. However if you're going to put them up in the air on poles spread across a wood
and you can't guarantee it won't rain, they need to be robust and vaguely weatherproof.

Putting them on poles was an attempt to make them inconspicuous to passers by but more importantly to stop the signal being blocked by low level obstacles. I had several sectional aluminium poles that allowed me to get the nodes out of eyeline without being so high they were in the canopy of the taller trees. You really don't notice the ones on poles wandering about and I lost one a couple of times before I made the sensible decision to take waypoints with my phone.

After a little experimentation with node placement I got a corridor along the main play area of our game covered. Once the full mesh was up I was able to send packets across painlessMesh from one end to another. So six nodes isn't enough for what I want to do but now I've seen how it scales I know it can be done without getting overly expensive.

I have ordered another six Wemos D1 mini Pro which should allow me to cover as much of this site as I need. My plan for these nodes is to cover a 'corridor' up the middle of the site linking the key locations, but as this is a mesh, once you add in the nodes carried by players and crew it should only get better.

Node placement was done with the useful behaviour of one of the painlessMesh example programs, where it flashes an LED counting out the size of the mesh, like in the video below. I chopped this example code around a little to include an activity LED and remove the 'hello world' stuff but as these nodes exist just to be part of the mesh the code is essentially done now.

The easily visible node count means you can extend coverage by walking away from the last node and when the count drops back to one, just retrace your steps until it rejoins. This gave me a mostly stable mesh really easily without tons of planning or having to consult a screen.

I'm going back to the site again in October and I plan to have at least ten nodes ready by then, mostly limited by having run out of the nice IP55 boxes I got cheaply from Maplin when they were closing down.

Building a painlessMesh ecosystem

I've spent the last week or so messing about with painlessMesh and it's been a mostly straightforward exercise.

As this is all based around microcontrollers, building any kind of monolithic device with multiple roles requires careful thought so I've stuck with an ecosystem of simple single purpose things.

The idea is also that if these are spread around in use then it extends the mesh, instead of having some central server everything speaks to, too far away to be reached.

The main one is pictured here, it's a combination Lasertag sensor & GPS tracker. OK that's two functions but the whole point of what I'm doing is for this thing to be usable. I need to know where people are and what state they're in.

I've also chucked together the following...

• Two relay nodes, as seen before, vaguely weatherproof with external Wi-Fi aerials and the option to connect big Yagis.
• Three compact painlessMesh to USB serial dongles, which allow you to interact with it by sending commands and viewing messages from a computer. These may eventually end up connected to a Raspberry Pi.
• A logger that writes everything it sees to CSV files on an SD card, one file per node, with timestamps.

All this has been chattering away quite happily but I've got a few minor issues...

• The Sensor & GPS node gets a bit overwhelmed and rebooted by the ESP8266 watchdog timer, especially if it's printing a lot of debug information over serial. It can manage 8 or more hours active so ditching the debugging info will I hope render it 100% reliable.
• The USB dongles seem to die when the computer goes to sleep for a long time, which I'm assuming is due to it putting the USB to sleep in some way that they won't recover from.
• Sometimes things just don't join the mesh until you power-cycle them.

None of this seems at all insurmountable, compared to writing my own mesh network library (again) so I'm sticking with painlessMesh for this project.

Going forward I really need to do some range testing of this and I've got a week to get ready for that.

Depending on how much range I get then I may need to do 'caching' of status messages to presently unreachable nodes. So as things move around they get delivered when they come into range. This is not a feature of painlessMesh so I'd be layering it on top, probably using the SPIFFS filesystem on the ESP8266 as storage if there's not enough free memory. Given these nodes will be attached to people moving around it'd basically be automated sneakernet.

I'm still using human readable, verbose messaging but that may have to go. Or perhaps send semi-readable codes with numeric values encoded as hex. Doing a load of parsing of things like latitude and longitude to/from readable strings, something that's kind of trivial on real computers, starts to chip away at available resource when playing with microcontrollers, especially if you want to buffer these up and store them for later delivery. When I wrote my mesh network for Ciseco xRF radios I agonised over whether to use 12 or 16 byte packets and settled on 12.