# Troubleshooting

## Overview

Here are some helpful troubleshooting steps for networking issues you may encounter while setting up your network.

## Netclient Connectivity Issues

On the netclient side, there could be many reasons why connectivity is not working as expected. Below are common troubleshooting steps for netclient connections.

### Verify Netclient Installation

Check that wireguard-tools is installed by running:

```bash
wg
```

<figure><img src="/files/iXfvNH6eV3mRP4dEsGn6" alt=""><figcaption></figcaption></figure>

The response should show the WireGuard interface named `netmaker`. If `wg` is not found, install wireguard-tools:

```bash
sudo apt install wireguard wireguard-tools -y
```

To verify netclient is installed and accessible, run:

```bash
netclient
```

<figure><img src="/files/w5byCl3nz4P27gRqUvfK" alt=""><figcaption></figcaption></figure>

If `netclient` is not found or fails, try reinstalling following the documentation: <https://docs.netmaker.io/docs/netclient#installation>

To verify the netclient daemon is running (Debian-based distros):

```bash
systemctl status netclient
```

The service should be `active (running)`. If it's not, try:

```bash
netclient install
```

If problems persist, uninstall and reinstall:

```bash
netclient uninstall
# then reinstall per docs
```

(Images showing `wg`, `netclient`, and `systemctl status netclient` appear in the original content.)

### Verify Network Connectivity

After joining a netmaker network, verify connectivity with:

```bash
netclient list
netclient server list
wg
```

* `netclient list` shows the network(s) the node joined.
* `netclient server list` shows the server(s) the node joined.
* `wg` lists peers and handshake/byte stats.

If `netclient list` or `netclient server list` show nothing, rejoin with the appropriate enrollment key.

If `wg` shows no peers despite multiple hosts in the network, try syncing:

```bash
netclient pull
```

Common causes for netclient failure to connect to others:

* Unable to establish connection to server over MQTT (failure to receive peer updates).
* Failure to create correct local routes.
* Unable to establish connection via TURN.

Basic troubleshooting steps:

* Run sync from server.
* Run `netclient pull` on the netclient machine.
* Restart the netclient daemon:

```bash
systemctl restart netclient
```

* Check the node status in the Netmaker UI.
* Consider relaying the client through a publicly accessible netclient as a relay server.

If issues persist, inspect logs:

```bash
sudo systemctl status netclient@<network-name>
sudo journalctl -u netclient@<network-name>
# For live logs:
sudo journalctl -u netclient@<network-name> -f
```

(Use the End key to jump to the most recent entries if using a pager.)

### Firewall Blocking Issues

Netclient (WireGuard) nodes must communicate with the server and peers. Firewalls commonly block required traffic by default. Typical rules to allow:

* UDP and TCP ports 51821-51830 (or your custom static ports)
* TCP port 443
* UDP ports 19302 & 3478 (STUN)
* UDP and TCP port 53 for DNS (optional)

Example iptables rule to allow a UDP port range:

```bash
iptables -A INPUT -p udp --dport 51821 -j ACCEPT
```

For UFW logging and inspection:

```bash
ufw logging low
cat /dev/null | sudo tee /var/log/ufw.log
ufw reload
cat /var/log/ufw.log | grep -e <netmaker server IP> -e <other nodes' IPs>
```

On Windows, allow Netclient and WireGuard applications through the firewall via the Windows Firewall settings.

For NAT/firewall appliances, check their blocked logs for traffic to/from your device IPs.

### MTU Issues

If WireGuard handshakes exist but peers are not reachable (cannot ping), MTU is often too high. Lower the MTU on the node via netconfig or by editing the host in the Netmaker UI.

Note: minimum recommended MTU is 1280 (IPv6 minimum and many routers expect standard MTU). Going lower may cause issues.

### Unresponsive Netclient / Segmentation Error

If netclient commands fail with segmentation faults or the daemon is unresponsive, the binary may be corrupted (failed update, interrupted install, OS issue). Reinstall netclient to overwrite the corrupt installation, then re-join the network if necessary.

### Incorrect Endpoint IP

You can set a static endpoint for a device in the Netmaker UI. This also disables endpoint detection if desired.

{% stepper %}
{% step %}

### Access All Devices

Open the All Devices interface in the Netmaker UI.
{% endstep %}

{% step %}

### Select Device

Select the device you want to modify.
{% endstep %}

{% step %}

### Edit Device

Click Edit.
{% endstep %}

{% step %}

### Enable Static Endpoint

Turn on "Static Endpoint".
{% endstep %}

{% step %}

### Set IP

Enter the correct endpoint IP and click "Update Device".
{% endstep %}

{% step %}

### Verify

Return to the Devices interface and confirm the IP has been updated.
{% endstep %}
{% endstepper %}

(Images illustrating the UI steps appear in the original content.)

## Relays

If relayed client traffic is not forwarded by the relay server:

* Ensure relay servers have required ports open and are publicly reachable.
* Run traceroute from relayed client to the destination client to identify where traffic is stopping. On Ubuntu:

```bash
sudo apt install traceroute
traceroute <destination-ip>
```

* If traceroute shows hops then a `*` at the relay server toward the destination, check Linux IP forwarding on the relay:

```bash
echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf
sysctl -p
```

* Check the relay server firewall and WAN/VPC rules to ensure forwarding is permitted.
* Ensure netmaker's default iptables rules exist on the relay server:

```bash
iptables -L
```

* Repeat verification (excluding the ip\_forward change) on the target netclient to ensure it can reach the relay and the relay can reach it. Troubleshoot firewall rules on that machine as needed.

## Static WireGuard

For static WireGuard peers that can't reach each other:

* Verify public/private key pairs for both peers.
* Verify endpoint addresses and ports.
* Verify AllowedIPs ranges include intended traffic.
* If handshake exists but ping fails, check MTU (lower it if needed).
* For NAT/restrictive firewalls, set PersistentKeepalive to keep NAT table entries alive.

You can set MTU in the peer config under the \[Interface] section:

```
MTU = <value>
```

Then restart the WireGuard interface.

## Remote Access Gateway and Client

Remote Access Gateways/Clients are static WireGuard configurations for devices that do not run netclient. The same relay and static WireGuard troubleshooting steps apply.

* Ensure "VPN Config Files" specify the correct egress address ranges.
* For Linux egressing external clients, enable IP forwarding and ensure iptables is installed on the gateway.
* Add a POSTROUTING MASQUERADE rule for the egress network interface, for example:

```bash
ip a
# assuming interface eth1
iptables -t nat -I POSTROUTING -o eth1 -j MASQUERADE
```

* The remote access gateway should show external clients as peers in `wg`. If peers are missing:

```bash
netclient pull
wg
```

* If per-peer domain names are used, ensure the gateway's DNS server can resolve those domain names.

## Egress

Netclient egress gateways forward traffic to/from specific IP ranges accessible to the gateway. This uses WireGuard AllowedIPs routing.

"External routes" determine which IP ranges are routed through the egress gateway. Correct configuration is essential.

Common problems:

* No traffic passing through although peer connects to the gateway.
* Only some internal ranges accessible.
* Traffic leaking to local network instead of through the egress.

Potential causes include misconfigured External routes, firewall rules, routing conflicts, or overlapping subnets.

Recommended troubleshooting:

1. Verify External routes / Allowed IPs match intended routing policy:

```bash
wg show netmaker allowed-ips
```

2. Check routing tables on the peer:

```bash
ip route
```

3. Test connectivity by pinging internal IPs and using traceroute to observe paths.
4. Review firewall rules and NAT configuration on the gateway.
5. If you need to preserve original source IPs, disable NAT on the egress gateway (note: this can introduce other issues).

Route conflicts example:

* If a local network 192.168.1.0/24 and a remote network 192.168.1.0/24 are both present, there will be duplicate routes to the same destination via different interfaces. Solutions:

{% stepper %}
{% step %}
Use different route metrics; the lower-metric route is preferred.
{% endstep %}

{% step %}
If only a few hosts must be egressed, add host IPs individually to the egress range rather than the whole network.
{% endstep %}

{% step %}
If many remote hosts but few local hosts, add the remote network to the egress range and add local hosts one-by-one in routes.
{% endstep %}
{% endstepper %}

If another host in the same local network is an egress gateway and the local network is added to the egress range, host route conflicts may occur. Adjust route metrics so the desired interface is preferred.

## Internet Gateway

Internet Gateway works like Egress but with AllowedIPs = 0.0.0.0/0 for selected hosts so all traffic routes through the internet gateway. Use the same troubleshooting steps as Egress (External routes = 0.0.0.0/0).

## Netmaker Server

If Let's Encrypt certificate retrieval fails when installing Netmaker with an auto-generated domain (nip.io), restart the Caddy container to retry:

```bash
docker restart caddy
```

After installation, verify the required containers are up:

```bash
docker ps
```

If containers failed, recreate them:

```bash
docker compose up -d
```

If signup/login to the Netmaker UI fails, you can reset the server (this deletes data and configs):

```bash
docker compose down --volumes
docker compose up -d
```

Then reopen the dashboard and create a new admin account.

If the UI shows an MQ error, restart the MQTT container:

```bash
docker restart mq
```

If restarting the server with docker-compose fails with an error like:

ERROR: Get "<https://registry-1.docker.io/v2/>": dial tcp: lookup registry-1.docker.io on \[::1]:53: read udp \[::1]:...: connection refused

This can occur because CoreDNS was taken down and systemd-resolved is disabled. To resolve:

{% stepper %}
{% step %}
Temporarily start systemd-resolved:

```bash
sudo systemctl start systemd-resolved.service
```

{% endstep %}

{% step %}
Start the netmaker server:

```bash
docker-compose up -d
```

{% endstep %}

{% step %}
Stop and disable systemd-resolved again:

```bash
sudo systemctl stop systemd-resolved.service
sudo systemctl disable systemd-resolved.service
```

{% endstep %}
{% endstepper %}

(Images showing container status and restarting steps appear in the original content.)

***

If you need any of the screenshot images preserved as separate assets for GitBook or want specific commands converted to runnable code blocks, tell me which sections and I will update accordingly.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://learn.netmaker.io/getting-started/operations-field-guide/troubleshooting.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.