Zerotier Multipath not working

[xee-khan]

Hi,
first of all i must say its an awesome effort to make install and configure Zerotier on embedded devices by following your manual and instructions. Superb Job!
without your manual i was unable to figure out how and where i went wrong.

for now i have successfully installed Zerotier 1.10 on MikrotikRb951G2Hnd based on Openwrt 19.07.10 luci.
although its little bit old but i am unable to upgrade it , attempted many times but cant update, so had no chice to manually install zerotier using opkg and after successfully installing and configuring it, every thing works as expected, except Zerotier Multipath feature.

as per Zerotier documentation i created local.conf in Zerotier home directory, which as per your manual is /etc/zerotier
using same documentation of ZeroTier i successfully made 2 peers on ubuntu 22 working flawlessly with multipath as intended.
but This Openwrt based router is unable to Show any Bonds available or created by zerotier, it gives just an output which shows no bonds are available , doesnt even tell us any thing about our bonding policy which we created in local.conf in /etc/zerotier/ directory.

i dont have any clue what should i do now.

for creating bonds we need to give it at least two wan links, i have given one wan link at "port wan "which is port1 and almost standard for all openwrt routers, and another wan is provided at port 2 .

i configured another VLAN named "vlan2" on port 2, and statically assigned IP to it. but still no success.

is there any one who can let me know what i have done wrong or is it a feature stripped off Zerotier to be used on embedded device, although i think it can not be the case, because in this way core reason for haviig a sdwan based router will lost in such scenario.

any guidance or help me in troubleshooting this issue will be greatly appreciated.

kind regards,

Xee Khan

[mwarning]

Hi, I have not used ZeroTier in a long time. You might want to ask in the official forum and also check ps | grep zerotier on the router to see the location of the ZerotTier (native) configuration file that was generated.

[xee-khan]

Thanks for your kind response.
i have asked Zerotier guys , but as mentioned on their website, they cant help anything regarding OPENWRT based platform, and they recommended you to contact as you are the official maintainer of this repository as mentioned on their website.
any way can you just enlighten if any part of code from zerotier sources have been stripped when cross compiling for openwrt?

[mwarning]

ZerotTier for OpenWrt has been compiled with default options (whatever they are).
I can check tomorrow.

Can your scenario be tested with low effort? If so, how?

[xee-khan]

sorry for my misunderstanding/confusion, which scenario are you talking about, because i have discussed 2 scenarios on Zerotier official community. please let me know which scenario you would like to test.
i do have almost all the things available at hand.
whenever you would like to test, feel free to let me know.

[ogarcia]

I honestly don't think multipath works in OpenWrt as it tries to modify aspects of the system itself.

I think the best thing to do is to configure bonding in OpenWrt (I understand it should be easy once you install the luci-proto-bonding package) and then just use that new bond0 interface.

[mwarning]

I have no experience using the Zerotier mulitpath feature. :/

[xee-khan]

I honestly don't think multipath works in OpenWrt as it tries to modify aspects of the system itself.

I think the best thing to do is to configure bonding in OpenWrt (I understand it should be easy once you install the luci-proto-bonding package) and then just use that new bond0 interface.

Thanks for your feed back, but i am interested to know basis of your insights , a little bit more in detail, why Multipath dont works on Openwrt? , what i think is , theoretically all ZeroTier needs to know available WANs and initialized interfaces, in openwrt it would be extra work of creating vlans and creating new interfaces, just like proto bonding, after that its software level playing with packets. My preference with Zerotier is because of ease with nat udp hole punching, so that one dont need to scratch his head with nat issues. MultiPath is extra cherry on the cake, absolute fundamental benefit of software defined networking.

and its so simple to use on other linux based systems , just by creating new empty file called local.conf in zerotier home directory, and paste any policy available from Zerotier knowledge base, and reboot. its that simple in other Linux distros which can identify new interfaces dynamically when any new hardware is plugged in, and zerotier if we have provided it any policy in local.conf file , it works flawlessly.

as per my understanding, Openwrt due to its structure is just unable to know dynamically new interfaces available, so zerotier cant bond unless it knows available interfaces, like any other linux, so for Openwrt , i think we just need to create new interfaces and creating new Vlans so that zerotier can know as per local.conf policy how to bind those interfaces for its own environment.

i have tried many times but dont know where i am going wrong when creating VLANS and new interfaces , that MultiPath dont show any available links to bind.

Regards,

Xee Khan

[xee-khan]

Besides other thing, requested Zerotier team to look into the matter, but they mentioned that only person to contact regarding Zerotier on Openwrt is to contact this github repository maintainer, they cant help for openwrt.
while Mr.mwarning 's tutorials are so in depth for making life to Zerotier on openwrt, i also think, if he just spends 5 -10 minutes on kowing how multipath works on other linux distros, he will easily figure out how it will work on openwrt as well.

for reference and makin it easy for any one doing this on Ubuntu
Step1) install ubuntu64 bit ( ubuntu 22 in my case)
Step2) Configure two WAN interfaces ( two network interface cards in my case , by default configured to get IP from DHCP so need nothing to do in this step , if static ip is necessary then 1 minute on assigning relevant IPs to those two network interfaces)
Step3) download and install zerotier as per their instructions
"curl -s https://install.zerotier.com | sudo bash"

step4) zerotier-cli join

step 5) create a new empty file named "local.conf" in Zerotier home directory
for debian /var/lib/zerotier-one
and for Openwrt as per Mr mwarnings Uci manual permanently moved and linked to /etc/zerotier/

copy paste any policy from https://docs.zerotier.com/zerotier/multipath/
in my case
{ "settings": { "defaultBondingPolicy": "active-backup" } }

and reboot.

step 6) # zerotier-cli bond list
shows available links and bonded and boom its working as it should be.

but on openwrt what we are missing?

[ogarcia]

I think the problem with multipath not working is going to lie in how OpenWrt handles network interfaces.

OpenWrt works mainly with logical and vlan interfaces (an internal switch, see here), so the system that Zerotier uses to establish the multipath will probably not work because it expects to "attack" physical interfaces such as eth0, eth1, etc.

I have never used this functionality as I don't have two internet connections (two WAN), but from what you say the configuration is correct and it doesn't work, so it is possible that the fact that it doesn't work comes from what I have said above.