Network Sharing Case

Overview

When deploying array servers at scale, the site network environment is often not known during project planning. It is only during deployment that constraints such as a limited public IP quota, strict network control, and a ban on configuring multiple public IPs become apparent.

The traditional architecture uses a one-to-one "one core board, one public IP" model: every core board needs its own public IP, either requested separately or configured manually. This model only works for small deployments. For large array clusters it causes IP shortages, non-compliant multi-IP deployment, high O&M costs, and wasted addresses. Adding a router increases hardware and maintenance costs and requires multi-department coordination, budget approval, and procurement, which significantly extends the deployment cycle.

This case uses a high-speed BMC NIC (5 Gbps or higher recommended) with the internal L3 switch to create WAN and LAN logical interfaces through VLANs, and enables NAT and IP forwarding on the BMC so that the whole cluster uses only one public IP. The goal is to reduce public IP consumption, simplify O&M, improve address utilization, and isolate the internal and external networks without adding network devices.

Intended Audience

• Network administrators and delivery engineers: the site has a limited public IP quota or forbids bulk configuration of multiple public IPs.
• Operations staff: want to reduce the work of configuring IP addresses for each sub-board and the associated maintenance cost.
• Project owners: need to deploy the cluster without major changes to the site network.

Application Scenario

As the number of cluster nodes grows, the limitations of the one-to-one IP model become obvious:

This case applies to clusters with a limited public IP quota, a ban on multiple public IPs, or a requirement to avoid adding a router. If public IPs are sufficient, assign an independent IP to each sub-board and do not enable this case.

How It Works

WAN / LAN split

The high-speed BMC NIC works with the internal L3 switch to create two logical interfaces through VLANs:

The L3 switch puts the compute nodes and the site service network into two independent network zones, providing internal/external isolation.

One-way communication

This case requires one-way communication: compute cores can access site terminals such as cameras, but the site network cannot directly access the compute core intranet.

Figure: The external network sees only one address; compute cores use internal addresses and share the public IP through BMC NAT.

Figure: Compute cores use MASQUERADE to reach the external network; no DNAT is configured for the site network to access the compute core intranet.

Data flow

Compute core to site terminal:

1. The sub-board sends traffic to the BMC nic0.
2. The BMC forwards the traffic to the WAN interface (VLAN 50) with IP forwarding enabled.
3. MASQUERADE changes the source address to the BMC WAN address.
4. The traffic leaves for the site service network with this single public address.

Site network to compute core:

1. A site terminal accesses the BMC WAN address.
2. No DNAT rule or route to the compute core intranet exists on the BMC.
3. The traffic cannot enter the compute core intranet.

Procedure

Configure in the following order: switch VLANs → aBMC → sub-boards and routing → IP forwarding and NAT → verification.

# Enable IP forwarding
echo 1 > /proc/sys/net/ipv4/ip_forward

# Make it persistent
echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf
sysctl -p

# If nftables is also in use, flush its rules first to avoid conflicts
nft flush ruleset

# Outbound only: source NAT for compute cores accessing the external network
iptables-legacy -t nat -A POSTROUTING -o vlan50 -j MASQUERADE

# Show the rules
iptables-legacy -t nat -L -n -v