Skip to content
Emer NetworkingEmer Networking
Go back

Containers on HPE Aruba AOS-CX: iPerf, rsync, and DNS Without Another Box

Edit post link disabled per config.

HPE Aruba Networking AOS-CX switches can run containers. The switch’s Container Manager pulls an image over HTTP, runs it in the background as a service, and keeps it running through reboots. HPE documents it as a way to run things like web servers, small application servers, and monitoring tools directly on the switch (Container overview).

Until recently, a switch could run only two containers. AOS-CX 10.18.1002 removes that limit: the 10.18.1002 release notes list “support for multiple containerized applications on a single AOS-CX Switch.” Depending on the platform, a switch can now run up to 10 containers at a time: 10 on the 9300S, 8 on most of the larger platforms, and 3 on a 6300. That changes what the feature is useful for. You can run an iPerf target, a sync job, and a small web app on the same switch instead of choosing two.

This post covers four use cases I keep coming back to, why a switch can be a better host than a server for them, the trade-offs, and the deployment flow for more than one container. For a full step-by-step iPerf build (pulling the right architecture image, saving it as a .tar, hosting it), see my earlier post: AOS-CX Containers.

Official HPE documentation used here:

Assumptions

  • You have a container-capable AOS-CX switch (see Platforms and limits) running 10.18.1002 or later if you want more than two containers.
  • You can build or pull a Docker image for the switch’s CPU architecture (ARM or x86_64) and export it as a file.
  • You have a host that can serve that file over plain HTTP, reachable from the switch in a known VRF.
  • You are comfortable with AOS-CX config context and VRFs.
  • Out of scope: writing the applications themselves, the vSphere agent, and isolated-network mode (8325-family only).

What changed in 10.18.1002

  • More than two containers. Before 10.18.1002, the limit was two containers per switch. Now the limit depends on the platform and the resources available on it: 3 on a 6300, 4 on the 6400, 8100, 8320, 8325, and 8360, 7 on the 8400, 8 on the 8325H, 8325P, 9300, and 10000, and 10 on the 9300S (full table in Platforms and limits).
  • Containers persist through reboots. An enabled container starts again automatically after the switch reboots (enable).
  • New defaults. The default memory limit per container rose to 512 MB in 10.18.1002. The default CPU limit is now per platform: 7% on the 6300, 5% on the 6400, 8100, 8320, 8325, and 8360, and 4% on the 8325H, 8325P, 8400, 9300, 9300S, and 10000.
  • Resources are still capped. CPU and memory limits are set per container, and all containers combined can use at most 20% of the switch’s CPU and 20% of its memory (restrict cpu, restrict memory). The Max memory column in the platform table is that ceiling. More containers means splitting that budget, so set restrict cpu and restrict memory explicitly on each one when a service needs more (or less) than the default. CPU and memory limits only apply to containers that are enabled.
  • Image size limits did not change from 10.17: 500 MiB on the ARM platforms and the 8400, and 1000 MiB on the other x86_64 platforms (table below).

Typical multi-container placement: firewalls at the edge, a core pair, and an access/aggregation pair hosting Container 1 through Container N

Use cases

iPerf on the core: a throughput target for UXI sensors and APs

This is the one I see most. A few of my customers use switches throughout their environment as iPerf targets. Put an iPerf server in a container on a core switch, and both HPE UXI sensors and HPE Aruba APs can run speed tests against it from the edge. You can test across campus without standing up a dedicated box just for iPerf, and UXI will run the test on a schedule.

  • UXI sensors run iPerf3 or iPerf2 as a custom test (Settings → Services → Add Service → Custom), pointed at the switch’s IP and port (UXI: iPerf testing). iPerf3 serves one client at a time, so for many sensors, run several iPerf3 containers on different ports and give the UXI test a port range. Sensors pick a free one. Multiple containers make that easy on one switch: one container per port, each with its own port-map.
  • HPE Aruba APs run an iPerf speed test against the same target, either on demand from HPE Aruba Networking Central or from the AP itself. The switch makes a good target.

Note: How you start the AP test depends on the AP software. HPE Aruba Instant APs use Central’s Analyze → Tools → Network Check → Speed Test (iPerf) (Troubleshooting AP Connectivity Issues). HPE Aruba AOS 10 APs run the speed test and report results with show speed-test data (CLI reference).

AOS-CX switches already have iPerf built in, but using it means dropping to the switch’s Linux shell by hand, and it doesn’t persist across reboots. A container keeps running as long as it is enabled in config.

rsync: sync file systems across network boundaries

rsync in a container can sync files from one place to another without scheduled tasks on a server or a dedicated box to run them. Each container attaches to one VRF (network vrf), so you can place the sync job in the VRF that can reach both ends.

If the container needs local storage that survives restarts, AOS-CX supports USB persistent storage mounts (USB persistent storage):

switch(config)# container <NAME>
switch(config-container-<NAME>)# image-location <URL> vrf <VRF>
switch(config-container-<NAME>)# mount <ID>
switch(config-container-mount-<ID>)# source usb /app1-data
switch(config-container-mount-<ID>)# destination /app-data
switch(config-container-<NAME>)# enable

Read HPE’s caveats first. The container is not aware if the USB device is removed. On chassis with redundant management modules, USB data does not follow the container after a failover.

A small web app for telemetry (an idea)

A small web application could perhaps consume and display network telemetry: a health status page that lives outside a dedicated box or NMS. HPE lists web servers (Nginx, Apache) and application servers (Node.js, Java, Python APIs) among supported container workloads, so the hosting side fits. What the app reads, and how, is up to you. I haven’t built this one yet, so treat it as a direction, not a recipe.

The port-map in the configuration example below (host TCP 8000 to container 9000 in VRF mgmt) is the shape this would take.

DNS at a very small site

For a very small site like a branch, where no other infrastructure is present, a container can run a DNS server (BIND, for example) on the switch itself. The same pattern works for ad-blocking DNS like Pi-hole.

Two things to check before you rely on it:

  • Port maps can’t reuse a host port that the switch reserves for its own services (port-map). Confirm that TCP/UDP 53 is free on the host side in the VRF you choose before you build a branch design around it.
  • Clients need to resolve when the container isn’t running. Keep a secondary resolver in DHCP.

Why a switch and not a server?

Spinning up dedicated boxes, and having the know-how to run a Docker server, is a non-starter in a lot of cases. Deploying to a CX switch is much easier because AOS-CX abstracts most of the config. You declare the image, the VRF, the port maps, and the limits in config, and the switch handles the rest.

With multiple containers, one switch can support different services across the environment. The switches are already at every site and already monitored, and now they have room for more than two jobs.

Trade-offs

  • You still build the image with Docker. Broadly speaking, you have to build or pull the image with Docker anyway, for the right architecture.
  • HTTP hosting, temporarily. The switch pulls the image from a URL over HTTP (IPv4 or IPv6), so you need to host it on an HTTP server long enough for the switch to grab it (image-location). After the image is downloaded and verified, a container restart does not download it again unless you change the image location (Restart protection).
  • Unsigned images are opt-in. By default, only HPE-signed images run. allow-unsigned bypasses that check, shows a risk warning you must accept, and has no effect when the switch is in enhanced secure mode.
  • Background services only. Interactive containers (shells, interpreters, top, vim) aren’t supported and may exit immediately.
  • Crash loops get stopped. After three restarts in a short window, the switch stops the container and show container reports run failed.
  • Shared, capped resources. All containers share the 20% CPU and memory ceiling.
  • Feature pack notice. Creating a container prints a notice that the feature requires the AOS-CX Advanced Software Feature Pack. HPE notes that AOS-CX does not enforce this (container). Check your licensing.

Outside of that, the config is very easy.

Deployment flow

Step 1: Build and host the image

Pull or build the image for the switch’s architecture and save it to a file. Then serve that file over HTTP from a host the switch can reach. The earlier post walks through this end to end with iPerf3 on a 6300.

Step 2: Define the first container

Create the container, set limits, attach a VRF, map ports, point at the image, and enable it last. Each config change to an enabled container restarts it, so finish configuring before you enable.

6300(config)# container app1
6300(config-container-app1)# restrict cpu 7
6300(config-container-app1)# restrict memory 256
6300(config-container-app1)# network vrf mgmt
6300(config-container-network-vrf-mgmt)# port-map host-port 8000 container-port 9000 protocol tcp
exit
6300(config-container-app1)# image-location http://30.0.0.2:8000/app.img vrf mgmt
6300(config-container-app1)# enable
exit

For an unsigned image (most home-built images), add allow-unsigned to image-location and accept the warning:

switch(config-container-1)# image-location http://10.0.0.1/container.img vrf mgmt allow-unsigned

Step 3: Add more containers

Repeat for each service. When you hit the platform’s limit, the switch says so. On a 6300, the fourth container is refused:

6300(config)# container app2
6300(config-container-app2) exit
6300(config)# container app3
6300(config-container-app3) exit
6300(config)# container app4
Maximum number of containerized applications configured

Check the limit for your switch before you plan services around it:

switch# show capacities containers

Step 4: Verify

Example output:

Switch# show container app1
Container : app1
Container status : operational
Manifest status : success
Image status : verified
Image location VRF : mgmt
Image location URL : http://192.168.1.100/app.tar
CPU limit : 7%
Memory limit : 256MB
VRFs : default

operational means deployed and running. Other states, such as acquiring, acquire failed, deploy failed, and run failed, tell you which stage broke (show container).

Optional: REST API

The same objects are exposed under /system/containers in the AOS-CX REST API.

Create a container with POST /system/containers:

{
"name": "app1",
"enable": true,
"image_location_url": "http://192.168.1.100/app.tar",
"image_download_vrf": "mgmt",
"allow_unsigned_image": true,
"runtime_constraints": {
"cpu": 7,
"memory": 256
}
}

Change limits on an existing container with PATCH /system/containers/<container_name>:

{
"runtime_constraints": {
"cpu": 15,
"memory": 512
}
}

GET /system/containers/app1 returns the same fields. Two errors to expect: creating a container past the platform limit (for example, a fourth container on a 6300) returns HTTP/1.1 500 Internal Server Error, and patching a container that doesn’t exist returns HTTP/1.1 404 Not Found.

Platforms and limits

Multiple containers (more than two) are supported in 10.18.1002 on the 6300M/F/L, 6400 (including V2 models), 8100, 8320, 8325, 8325H, 8325P, 8360 (including V2 models), 8400, 9300, 9300S, 10000, and 10040. They are not supported on the 4100i, 6000, 6100, 6200, 5420, or the OVA.

PlatformContainer limitCPU default (%)Max memory (MB)ArchitectureMax image size (MiB)
6300371638arm500
6400453277arm500
8100453277arm500
8320453277x86_641000
8325453277x86_641000
8325H846554x86_641000
8325P846554x86_641000
8360453277arm500
8400746554x86_64500
9300846554x86_641000
9300S10413107x86_641000
10000846554x86_641000

Container limit, CPU default, and max memory are from the 10.18.1002 multiple-containers feature data. Architecture and image size are from HPE’s Supported switches page. Image size limits are unchanged from 10.17. show capacities containers confirms the limit on your switch.

Wrap-up

10.18.1002 turns containers on AOS-CX from “pick two” into a small service host. Run an iPerf target on the core for UXI and AP throughput tests, sync files across boundaries with rsync, try a small telemetry page, and run DNS at a branch with no other infrastructure. All of it runs on a switch you already have.


Back to top
Edit post link disabled per config.

Previous Post
Bench Tools: answers that stay in the browser
Next Post
Monitoring my home lab with Grok Bot: read-only, outbound-only, least privilege
EOF