# Policy-Based Routing: Dual-WAN with UDP Port 5001

## Scenario

| Interface  | Device       | Purpose                                          |
|------------|--------------|--------------------------------------------------|
| `wan`      | `ae_wan`     | Main WAN — default gateway for all traffic       |
| `wan_iptv` | `ae_wan.100` | Secondary WAN (VLAN 100) — carries UDP/5001 only |
| `lan`      | `br-lan`     | LAN bridge (eth0–eth3)                           |

**PBR policy:**
- UDP port 5001 traffic from LAN → marked with `fwmark 5` by QoS → routed via `ae_wan.100` (routing table 100)
- All other traffic → routed via `ae_wan` (main routing table, default)

The `ForwardingPolicy` value `5` is an arbitrary tag linking the QoS classification
outcome to the routing policy. It is implemented as a Linux firewall mark (`fwmark`).

---

## OpenWrt UCI — `/etc/config/network`

```
config interface 'loopback'
    option device   'lo'
    option proto    'static'
    option ipaddr   '127.0.0.1'
    option netmask  '255.0.0.0'

# LAN bridge with physical ports
config device 'br_lan'
    option type  'bridge'
    option name  'br-lan'
    list   ports 'eth0'
    list   ports 'eth1'
    list   ports 'eth2'
    list   ports 'eth3'

config interface 'lan'
    option device   'br-lan'
    option proto    'static'
    option ipaddr   '192.168.1.1'
    option netmask  '255.255.255.0'

# Main WAN — all-traffic default gateway (uses main routing table 254)
config interface 'wan'
    option device 'ae_wan'
    option proto  'dhcp'

# Secondary WAN — VLAN 100, dedicated routing table 100
# DHCP-assigned default route goes into table 100 automatically
config interface 'wan_iptv'
    option device   'ae_wan.100'
    option proto    'dhcp'
    option ip4table '100'
    option macaddr  '58:00:32:C0:0E:44'

# PBR rule: packets marked fwmark=5 (by QoS) -> routing table 100
# NOTE: the option name is 'mark', not 'fwmark'
config rule 'rule_iptv_udp5001'
    option disabled  '0'
    option priority  '100'
    option mark      '5'
    option lookup    '100'
```

> **Note:** The UCI `config rule` section uses `option mark`, not `option fwmark`.
> The kernel's `ip rule show` displays it as `fwmark 0x5` (hexadecimal), which can
> cause confusion, but the UCI field name is `mark`.

### Resulting kernel state

```
$ ip rule show
0:      from all lookup local
100:    from all fwmark 0x5 lookup 100      <-- UDP/5001 traffic (marked by QoS)
32766:  from all lookup main                <-- all other traffic (ae_wan default route)
32767:  from all lookup default

$ ip route show table 100
default via 10.10.10.1 dev ae_wan.100 proto static src 10.10.10.100
10.10.10.0/24 dev ae_wan.100 proto static scope link
```

---

## OpenWrt UCI — `/etc/config/firewall`

`wan_iptv` must be added to the `wan` firewall zone. Without this, forwarded
packets re-routed via `ae_wan.100` are silently dropped in the `FORWARD` chain.

```
config zone 'wan'
    option name     'wan'
    list   network  'wan'
    list   network  'wan6'
    list   network  'wan_iptv'    # <-- required, or forwarded traffic is dropped
    option input    'DROP'
    option output   'ACCEPT'
    option forward  'DROP'
    option masq     '1'
    option mtu_fix  '1'
```

> `option masq '1'` enables MASQUERADE for all listed networks including `wan_iptv`,
> ensuring forwarded LAN traffic has its source IP rewritten to the `ae_wan.100`
> address before leaving the router.

---

## OpenWrt UCI — `/etc/config/qos`

```
# Classify UDP port 5001 traffic and tag it with ForwardingPolicy=5
# qosmngr translates this into: iptables -t mangle MARK --set-mark 5
config classify 'udp5001_secondary_wan'
    option enable            '1'
    option order             '1'
    option all_interfaces    '1'
    option proto             'udp'
    option dest_port         '5001'
    option forwarding_policy '5'
```

### Resulting kernel state

```
$ iptables -t mangle -L qos_prerouting -n -v
Chain qos_prerouting (1 references)
 pkts bytes target  prot opt in  out  source    destination
    3    96 MARK    udp  --  *   *    0.0.0.0/0 0.0.0.0/0   udp dpt:5001  MARK set 0x5
```

---

## End-to-End Packet Flow

```
LAN device
     |  UDP packet, dest port 5001
     v
  br-lan
     |
  [iptables mangle PREROUTING — qos_prerouting]
     |  rule: proto udp dport 5001 -> MARK --set-mark 5
     |  packet now has fwmark=0x5
     v
  [routing decision — ip rule]
     |  rule: fwmark 0x5 -> lookup table 100
     v
  [routing table 100]
     |  default via 10.10.10.1 dev ae_wan.100
     v
  [iptables nat POSTROUTING — MASQUERADE]
     |  src rewritten to 10.10.10.100 (ae_wan.100 address)
     v
  ae_wan.100 (VLAN 100)
```

Non-UDP/5001 traffic falls through to the main routing table and exits via `ae_wan`.

---

## TR-181 Data Model Configuration

> Only `Device.Routing` and `Device.QoS` objects are shown.

### How Router discovery works

`Device.Routing.Router.{i}` instances are **auto-discovered** by netmngr at startup
by scanning `/etc/config/network` for:
- `option ip4table` on interface sections → one Router per unique table ID
- `option table` on route sections
- `option lookup` on rule sections

Router.{i} for table 254 (main) always exists. If `wan_iptv` already has
`option ip4table '100'`, then a Router with `ID=100` is created automatically —
no `add Device.Routing.Router.` needed.

When you **do** `add Device.Routing.Router.`, the code auto-assigns the
next free table ID from the range 1–252 (excluding 128). You cannot specify the
ID — note it from `Device.Routing.Router.{i}.ID` after the add.

---

### dmcli configuration steps

#### Step 0 — Find the Router instance for table 100

```sh
# List all Router instances and their table IDs:
get Device.Routing.Router.

# Example output might show:
#   Device.Routing.Router.1.ID = 254   (main table)
#   Device.Routing.Router.2.ID = 100   (wan_iptv table, auto-discovered)
# Assume Router.2 is the one with ID=100.
```

If `wan_iptv` does not yet have `ip4table` set (starting fresh):

```sh
# dmcli

add Device.Routing.Router.
# Returns e.g. Device.Routing.Router.2.

get Device.Routing.Router.2.ID   # note the auto-assigned table ID, e.g. 1
set Device.Routing.Router.2.Name udp5001_wan
```

#### Step 1 — Link wan_iptv to the routing table via Device.IP.Interface

Setting `Device.IP.Interface.{i}.Router` writes `ip4table` on the UCI interface
section, which causes the DHCP client to place all learned routes (including the
default gateway) into routing table 100 automatically.

```sh
# dmcli

# Find the IP.Interface instance for wan_iptv:
get Device.IP.Interface.
# Identify the instance whose Name = 'wan_iptv', e.g. Device.IP.Interface.3.

set Device.IP.Interface.3.Router Device.Routing.Router.2
# Writes: option ip4table '100' on the wan_iptv UCI interface section
```

#### Step 2 — Add static default route (skip if DHCP populates table 100)

With `ip4table '100'` set, the DHCP-assigned default route goes into table 100
automatically. Only add this for a static setup:

```sh
# dmcli

add Device.Routing.Router.2.IPv4Forwarding.
# Returns e.g. Device.Routing.Router.2.IPv4Forwarding.1.
# Creates: config route 'route_100_1' with option table '100' option disabled '1'

set Device.Routing.Router.2.IPv4Forwarding.1.DestIPAddress    0.0.0.0
set Device.Routing.Router.2.IPv4Forwarding.1.DestSubnetMask   0.0.0.0
set Device.Routing.Router.2.IPv4Forwarding.1.GatewayIPAddress 10.10.10.1
set Device.Routing.Router.2.IPv4Forwarding.1.Interface        Device.IP.Interface.3
set Device.Routing.Router.2.IPv4Forwarding.1.Enable           true
```

Resulting UCI:
```
config route 'route_100_1'
    option table     '100'
    option target    '0.0.0.0'
    option netmask   '0.0.0.0'
    option gateway   '10.10.10.1'
    option interface 'wan_iptv'
    option disabled  '0'
```

#### Step 3 — Add the PBR policy rule (fwmark 5 → table 100)

```sh
# dmcli

add Device.Routing.Policy.
# Returns e.g. Device.Routing.Policy.1.
# WARNING: add_policy always writes 'option in loopback' by default — see note below.

set Device.Routing.Policy.1.Priority          100
set Device.Routing.Policy.1.ForwardingPolicy  5
set Device.Routing.Policy.1.RouterRef         Device.Routing.Router.2
# Clear the default 'option in loopback' by setting SourceIPPrefix to match-all:
set Device.Routing.Policy.1.SourceIPPrefix    0.0.0.0/0
set Device.Routing.Policy.1.Enable            true
```

Resulting UCI:
```
config rule 'rule_1'
    option priority  '100'
    option mark      '5'
    option lookup    '100'
    option src       '0.0.0.0/0'
    option disabled  '0'
```

Resulting kernel rule:
```
ip rule add from 0.0.0.0/0 fwmark 5 table 100 priority 100
```

> `from 0.0.0.0/0` is equivalent to `from all` and has no practical effect.
> See the **known quirk** section below for why this is needed.

#### Step 4 — Add the QoS classification (mark UDP/5001 with fwmark 5)

```sh
# dmcli

add Device.QoS.Classification.
# Returns e.g. Device.QoS.Classification.13.

set Device.QoS.Classification.13.Order            1
set Device.QoS.Classification.13.AllInterfaces    true
set Device.QoS.Classification.13.Protocol         17      # UDP
set Device.QoS.Classification.13.DestPort         5001
set Device.QoS.Classification.13.ForwardingPolicy 5
set Device.QoS.Classification.13.Enable           true
```

Resulting UCI:
```
config classify 'udp5001_secondary_wan'
    option enable            '1'
    option order             '1'
    option all_interfaces    '1'
    option proto             '17'
    option dest_port         '5001'
    option forwarding_policy '5'
```

---

### Object relationship summary

```
Device.IP.Interface.3  (wan_iptv / ae_wan.100)
    |
    | .Router = Device.Routing.Router.2
    | → sets ip4table='100', DHCP routes go into table 100
    v
Device.Routing.Router.2  (ID=100)
    |
    | IPv4Forwarding.1.Interface = Device.IP.Interface.3
    v
Device.Routing.Router.2.IPv4Forwarding.1  (default route via 10.10.10.1)
    ^
    | .RouterRef = Device.Routing.Router.2
    |
Device.Routing.Policy.1  (ForwardingPolicy=5 → mark '5' → lookup 100)
    ^
    | ForwardingPolicy=5 is the shared tag
    |
Device.QoS.Classification.13  (UDP/5001 → MARK --set-mark 5)
```

---

## Troubleshooting

### All traffic goes through the new interface

This can happen if new interface was created after the default wan interface, causing udhcpc to make this one as the default. Rebooting the device should fix the issue. This can also be fixed by other methods, for example setting option defaultroute to 1 in the wan interface, but this is not supported via data model as of now.

### Traffic marked but not leaving the router

**Symptom:** `iptables -t mangle -L qos_prerouting -n -v` shows packets hitting the
MARK rule, but tcpdump shows nothing on `ae_wan.100` or `ae_wan`.

**Cause:** The OpenWrt firewall is dropping the packet in the `FORWARD` chain.
When traffic is re-routed via `ae_wan.100` by PBR, it exits through the `wan_iptv`
interface. If `wan_iptv` is not listed in the `wan` firewall zone, the default
`FORWARD DROP` policy silently discards it.

**Fix:** Add `wan_iptv` to the `wan` zone:
```sh
uci add_list firewall.@zone[1].network='wan_iptv'
uci commit firewall
/etc/init.d/firewall restart
```

---

### Known quirk: `option in 'loopback'` added by default

**Symptom:** After `add Device.Routing.Policy.`, the generated UCI rule contains
`option in 'loopback'`, even though no `SourceInterface` was configured.

**Cause:** `add_policy` (routing.c:1780) always writes `option in 'loopback'` as a
placeholder. The `SourceInterface` setter only clears it when `SourceIPPrefix` is
also non-empty; setting `SourceInterface` to an empty value when `SourceIPPrefix`
is empty simply reinstates `loopback`.

**Effect:** `option in 'loopback'` generates `ip rule add iif lo ...`, restricting
the rule to loopback-originated traffic. A fwmark-based rule must not have this
restriction.

**Fix — via data model:** Set `SourceIPPrefix` to the match-all prefix, which clears
the `in` field as a side effect:
```sh
set Device.Routing.Policy.1.SourceIPPrefix 0.0.0.0/0
```

**Fix — via UCI directly:**
```sh
uci del network.rule_1.in
uci commit network
/etc/init.d/network reload
```

---

### Verifying the full PBR chain

```sh
# 1. Verify routing lookup for marked vs unmarked packets (no traffic needed):
ip route get 8.8.8.8 mark 5     # expected: dev ae_wan.100 src 10.10.10.100
ip route get 8.8.8.8            # expected: dev ae_wan     src <wan_ip>

# 2. Verify mark counter increments with real forwarded traffic:
iptables -t mangle -Z qos_prerouting
# ...send UDP/5001 from a LAN device...
iptables -t mangle -L qos_prerouting -n -v
# UDP/5001 counter should have incremented
```

> **Important:** Run the test from a **LAN device**, not from the router itself.
> Locally generated traffic enters at the `OUTPUT` hook and bypasses `PREROUTING`,
> so the fwmark rule will never be applied to it.

---

### UCI mark vs kernel fwmark naming

The UCI `config rule` option is named `mark`:
```
option mark '5'
```
The kernel (`ip rule show`) displays this as `fwmark 0x5` (hexadecimal). These refer
to the same value — the UCI name and the kernel display name differ.

---

## Old Configuration (pre-TR-181 change)

Before this netmngr and qosmngr (https://dev.iopsys.eu/feed/iopsys/-/merge_requests/2276/diffs?commit_id=843d1fcd0b32447b9551d8085b4fee4c72aed566) change, `forwarding_policy` in a classify rule was used
**directly as a routing table ID**. qosmngr's `classify.sh` called `handle_ip_rule`
which read the classify criteria and generated `ip rule add ... table <forwarding_policy>`
at startup. No iptables marking was involved and `Device.Routing.Policy` played no role.

### What changed

| Aspect | Old behaviour | New behaviour |
|--------|--------------|---------------|
| `forwarding_policy` value | routing table ID (e.g. `100`) | fwmark tag (e.g. `5`) |
| ip rule creation | qosmngr scripts (classify.sh → ip_rule.sh) | netmngr via Device.Routing.Policy |
| iptables MARK | not used | MARK --set-mark in qos_prerouting |
| Device.Routing.Policy needed | no | yes |

---

### Old UCI — `/etc/config/qos`

The `forwarding_policy` value was set to the routing **table ID** (100), not an
fwmark tag. qosmngr extracted the match criteria from the same classify section
and built the ip rule itself.

```
config classify 'udp5001_secondary_wan'
    option enable            '1'
    option order             '1'
    option all_interfaces    '1'
    option proto             'udp'
    option dest_port         '5001'
    option forwarding_policy '100'   # routing table ID, not an fwmark
```

No `config rule` section in `/etc/config/network` was needed. qosmngr generated
and applied the rule at runtime from the classify criteria:

```sh
# Generated internally by ip_rule.sh and applied at qos restart:
ip rule add ipproto 17 dport 5001 table 100
```

The `/etc/config/network` only needed the interface and its routes — no rule section:

```
config interface 'wan_iptv'
    option device   'ae_wan.100'
    option proto    'dhcp'
    option ip4table '100'
    option macaddr  '58:00:32:C0:0E:44'

# No 'config rule' section — qosmngr created the ip rule dynamically
```

The firewall zone requirement was the same:
```
config zone 'wan'
    list network 'wan_iptv'
    option masq  '1'
    ...
```

### Old kernel state

```
$ ip rule show
0:      from all lookup local
100:    from all ipproto 17 dport 5001 lookup 100   <-- created by qosmngr
32766:  from all lookup main
32767:  from all lookup default

$ iptables -t mangle -L qos_prerouting -n
Chain qos_prerouting (0 references)
# empty — no MARK rule, no fwmark involved
```

---

### Old data model configuration

Only `Device.QoS.Classification` needed configuring. No `Device.Routing.Policy`
entry was required because qosmngr created the ip rule itself.

#### Device.QoS.Classification.1

| Parameter        | Value  | Notes                                         |
|------------------|--------|-----------------------------------------------|
| Enable           | `true` |                                               |
| Order            | `1`    |                                               |
| AllInterfaces    | `true` |                                               |
| Protocol         | `17`   | UDP                                           |
| DestPort         | `5001` |                                               |
| ForwardingPolicy | `100`  | **routing table ID**, not an fwmark tag       |
| TrafficClass     | `-1`   |                                               |

#### dmcli steps (old)

```sh
# Router and IP.Interface setup is the same as the new approach (steps 0–2)

# Classification — forwarding_policy is the table ID directly
add Device.QoS.Classification.
set Device.QoS.Classification.13.Order            1
set Device.QoS.Classification.13.AllInterfaces    true
set Device.QoS.Classification.13.Protocol         17
set Device.QoS.Classification.13.DestPort         5001
set Device.QoS.Classification.13.ForwardingPolicy 100   # table ID
set Device.QoS.Classification.13.Enable           true

# No Device.Routing.Policy step needed — qosmngr creates the ip rule from
# the classify criteria automatically on restart
```

#### What was NOT needed (old approach)

```sh
# These steps were NOT required with the old qosmngr behaviour:

# add Device.Routing.Policy.          # not needed
# set Device.Routing.Policy.1.ForwardingPolicy 5   # not needed
# set Device.Routing.Policy.1.RouterRef ...        # not needed
```

---

### Why the old approach was non-compliant with TR-181

TR-181 defines `Classification.ForwardingPolicy` as a **tag** carried with the
packet, not a routing table identifier. The intent is that the routing layer
(via `Device.Routing.Policy`) independently decides how to handle packets with
a given tag. By using the value as a table ID and having qosmngr create ip rules
directly, the two layers were coupled — changing the routing table required
updating the QoS classification, and `Device.Routing.Policy` was bypassed entirely.
