Applying DevOps to networks

Managing multiple Go versions

rdodin — Fri, 31 Jan 2025 00:00:00 +0000

Managing multiple Go versions

When dealing with several Go projects you inevitably end up needing more than one Go version installed and selected correctly for a given project.

Somehow I always forget how I do this when I need to fix a new version for another project, so here is what I do for the future me:

Install the target release Go downloader. For example, for go1.22.11
bashgo install golang.org/dl/go1.22.11@latest
Now download the actual sdk
bashgo1.22.11 download
Now for whatever project that you want to set this version for, create the direnv's .envrc file and set the path accordingly:

bashPATH=~/sdk/go1.22.11/bin:$PATH

Managing multiple kubeconfigs by merging them

rdodin — Mon, 24 Jun 2024 00:00:00 +0000

Managing multiple kubeconfigs by merging them

Recently I've been deploying k8s clusters on different cloud providers, with different automation stacks and in different configurations. As a result, I ended up with a bunch of of kubeconfigs polluting my ~/.kube directory.

As much as kubectl is an swiss knife for all things k8s, it still can't quite make dealing with different kubeconfigs a breeze. The annoying part is that you have to have a single kubeconfig environment with all your clusters, users, contexts listed, and this is not the case when you have multiple kubeconfigs. So, naturally one would want to merge them into a single kubeconfig.

To do that I had to write a small script that does the following high level steps:

Backup existing ~/.kube/config file: you don't want to lose your existing kubeconfig because you messed up something in your script or overwrote it with a wrong kubeconfig.
Find paths to all kubeconfig files in a particular directory and set them as a KUBECONFIG environment variable.
merge these kubeconfigs into a single ~/.kube/config file.

Here is the script:

bashcp ~/.kube/config ~/.kube/bak/config_$(date +"%Y-%m-%d_%H:%M:%S")_bak && \KUBECONFIG=$(find ~/.kube/cluster_configs -type f | tr '\n' ':') \kubectl config view --flatten > ~/.kube/config

Nothing fancy, but it does the job. Essentially the rule of thumb that I follow now is to keep all my kubeconfigs in ~/.kube/cluster_configs directory and run this script whenever I add or delete a kubeconfig in this directory. The resulting kubeconfig will contain all the contexts from all of the files and I could do kubectl config use-context <context> to switch between them.

Using skopeo container image

rdodin — Fri, 08 Mar 2024 00:00:00 +0000

Using skopeo container image

By now you know I hate to "install" things on the systems I work on, and that is because I have too many machines I carry work on. Hence, I prefer to containerize all the things and use handy aliases.

Here is one for skopeo to copy images between registries:

bashalias skopeo='sudo docker run --rm \-v ~/.config/gcloud:/root/.config/gcloud:ro \-v ~/.docker/config.json:/tmp/auth.json:ro \-v /var/run/docker.sock:/var/run/docker.sock \-v /usr/bin/docker-credential-gcr:/usr/bin/docker-credential-gcr \-v $(pwd):/workdir \-w /workdir \quay.io/skopeo/stable:v1.14'

bash$ skopeo --versionskopeo version 1.14.2

Note this quirky docker-credential-gcr binary mount, this is an authentication helper for skopeo to authenticate with GCP.Other clouds might require other helpers or file mounts.

A nice blog post on how to use skopeo different transports - https://www.redhat.com/sysadmin/7-transports-features

Adding border to the logos

rdodin — Thu, 15 Feb 2024 00:00:00 +0000

Adding border to the logos

It is super easy nowadays to generate a decent logo for your OSS project using any of genAI tool (dall-e, bing, etc). But one of the side-effects of the image generation might be the noise around the edges.

If that's your case you can apply a border (offset) to your logo to mask the potential rough edges. I can recommend borderize service that makes it a matter of a few clicks.

Formatting bash

rdodin — Mon, 05 Feb 2024 00:00:00 +0000

Formatting bash

Whenever I need to format bash scripts I use the mvdan's shfmt - https://github.com/mvdan/sh/blob/master/cmd/shfmt/shfmt.1.scd as a docker container:

bashsudo docker run --rm -u "$(id -u):$(id -g)" -v $(pwd):/mnt -w /mnt mvdan/shfmt:v3 -w utils/if-wait.sh

Upscaling images

rdodin — Wed, 31 Jan 2024 00:00:00 +0000

Upscaling images

You know how in TV shows with IMDB rating ranging from 5 to 7 cops use that magic image recognition software that transforms the cctv image of a bad guy from the "mashed potato" quality to the Vogue cover? Yeah...

But recently I needed to upscale the AI generated logo I made for Clabernetes project, and the quality was "meh", since it was a free service from Bing and whatnot. So whilst it was printable, the DPI was not good enough. So I decided to try my luck and google for these AI-powered image upscale services; frankly, my hopes were quite low.

The first hit with a fishy DNS name - https://www.upscale.media/ - did not reinforce my beliefs that this is all a gimmick. But I tried, and it was legit good. It upscaled my logo while removed the noise and blurriness from the original image.

Here is the comparison:

So yeah, something I wanted to save here because I will likely use it next time as well.

OpenStack Client Container Image

rdodin — Thu, 11 Jan 2024 00:00:00 +0000

OpenStack Client Container Image

I like the portability, managability and package manager agnostic nature of container images. Especially for the tools I use couple of times a month. And even more so for Python tools that don't have native wheels for all their dependencies. Like OpenStack Client.

So I built a small multi-stage Dockerfile to build a container image with OpenStack Client and all its dependencies. It's based on the official Python image and has a slim footprint:

Dockerfile--8<-- "https://raw.githubusercontent.com/hellt/dockerfiles/main/openstack-client/openstack-client.dockerfile"

You can pull the image from ghcr:

docker pull ghcr.io/hellt/openstack-client:6.4.0

To use this image you first need to source the env vars from your openrc file:

source myopenrc.sh

Then I prefer to install the alias openstack to my shell so that it feels like I have the client installed locally:

bashalias openstack="docker run --rm -it \ -e OS_AUTH_URL=${OS_AUTH_URL} -e OS_PROJECT_ID=${OS_PROJECT_ID} \ -e OS_USER_DOMAIN_NAME=${OS_USER_DOMAIN_NAME} \ -e OS_PROJECT_NAME=${OS_PROJECT_NAME} \ -e OS_USERNAME=${OS_USERNAME} -e OS_PASSWORD=${OS_PASSWORD} \ ghcr.io/hellt/openstack-client:6.4.0 openstack $@"

Then you can use the client as usual:

❯ openstack server list+-----------------------------+----------------+--------+-----------------------------+------------------------------+---------------------+| ID | Name | Status | Networks | Image | Flavor |+-----------------------------+----------------+--------+-----------------------------+------------------------------+---------------------+| 0fa75185-0f76-482f-8cc3- | k8s-w3-411e6d7 | ACTIVE | k8s-net-304e6df=10.10.0.11 | nesc-baseimages- | ea.008-0024 || 38e4d60212c8 | | | | debian-11-latest | |-- snip --

Test coverage for Go integration tests

rdodin — Mon, 14 Aug 2023 00:00:00 +0000

Test coverage for Go integration tests

I have been working on containerlab for a while now; a project that once started as a simple idea of a tool that would create and wire up SR Linux containers grew into a full-blown network emulation tool loved by the community and used in labs by many.

As it became evident that many more users started to rely on containerlab for their daily work, the looming feeling of responsibility for the quality of the tool started to creep in. At the same time, the growing user base exposed us to many more feature requests and integrations, making it harder to find time to address technical debt and improve testing.

Given the nature of the project, it was clear that integration tests offer a quick way to validate the functionality, as we could replicate the user's workflow and verify the outcome. However, the integration tests are not without their own challenges, and one of them is the test coverage which is not as easy to get as with unit tests.

In this post, I will share how coverage enhancements introduced in Go 1.20 helped us to get the coverage for our integration tests and jump from a miserable 20% to a (less sad) 50%.

gNMIc talks at DKNOG and NANOG

rdodin — Tue, 21 Mar 2023 00:00:00 +0000

gNMIc talks at DKNOG and NANOG

If you have never heard of gNMI and/or gNMIc project, you can start with my last week's talk DKNOG

Thirty minutes introduction to gNMI and gNMIc to get you started.

After this taster, you will likely want to know more, and Karim Radhouani has you covered. A 1-hour gNMIc tutorial has recently been published directly from NANOG 87 stage.

After this one you'll never wanna see OIDs again.

Refreshing Go package index for your package

rdodin — Wed, 15 Feb 2023 00:00:00 +0000

Refreshing Go package index for your package

It is quite frustrating to wait for pkg.go.dev to refresh your index, and I always forget how give it a slight push:

bashGOPROXY=proxy.golang.org go list -m example.com/mymodule@v0.1.0

The new version won't appear immediately, but at least it seems it will be quicker to show up.

SR Linux logging with ELK

rdodin — Sun, 12 Feb 2023 00:00:00 +0000

SR Linux logging with ELK

Implementing centralized logging using modern log collectors is an interesting task even before you start solving scaling problems.

My colleague and I opened up a series of posts dedicated to logging in the context of datacenter networks. We started with the basics of SR Linux logging and used the famous ELK stack as our log storage/processing solution.

Integrating SR Linux logging with ELK via Syslog was fun, and we tried to capture every step of the way. Plus, we create a containerlab-based lab that anyone can use to test the solution themselves.

Dig into "SR Linux logging with ELK" and open up the world of modern logging.

Creating a syntax highlighter for SR Linux CLI snippets

rdodin — Sun, 08 Jan 2023 00:00:00 +0000

Creating a syntax highlighter for SR Linux CLI snippets

How to write a custom syntax highligher for your favorite Network OS CLI and integrate it the doc engine?

=== "Raw text CLI snippet" --{ * candidate shared default }--[ network-instance black ]-- A:leaf1# info static-routes static-routes { route 192.168.18.0/24 { admin-state enable metric 1 preference 5 next-hop-group static-ipv4-grp } route 2001:1::192:168:18:0/64 { admin-state enable metric 1 preference 6 next-hop-group static-ipv6-grp } }=== "With srl syntax applied" srl --{ * candidate shared default }--[ network-instance black ]-- A:leaf1# info static-routes static-routes { route 192.168.18.0/24 { admin-state enable metric 1 preference 5 next-hop-group static-ipv4-grp } route 2001:1::192:168:18:0/64 { admin-state enable metric 1 preference 6 next-hop-group static-ipv6-grp } }

Read in my post "SR Linux Syntax Highlighting with Pygments" at learn.srlinux.dev portal.

SR Linux, JSON-RPC and Ansible

rdodin — Fri, 09 Dec 2022 00:00:00 +0000

SR Linux, JSON-RPC and Ansible

This week I have authored two tutorials for our https://learn.srlinux.dev portal.

The prime reason for this deep dive was to help our customers marry Ansible automation platform with network operations on SR Linux NOS. While I am personally not in the camp "Ansible for network automation" users, the reality is that it is still used by many small/mid teams who by now have large collections of playbooks and trained engineers.

gNMIc joins Openconfig 🚀

rdodin — Tue, 04 Oct 2022 00:00:00 +0000

gNMIc joins Openconfig 🚀

Two years ago, a dozen contributors less, 400 Pull Requests, and 2000 commits behind, another pet project appeared on a vast GitHub landscape. It was a learning exercise by [Karim Radhouani][karim-github] to sharpen his skills in [gNMI][gnmi-ref] - a niche network management protocol promoted by the [Openconfig][oc] group.

Initially named gnmi_client, it had a noble but narrow scope of providing a feature-rich, complete, yet intuitive CLI for gNMI-enabled routers. Fast forward two years, and we have the [gNMIc][gnmic-main-site] software suite that is much more than just a CLI for gNMI.

Today, Nokia donates the gNMIc project to Openconfig, and with that move, we expect to see gNMIc adopted by even more companies and organizations :partying_face:

- :material-home:{ .lg .middle } **gNMIc new address** --- :material-github: [openconfig/gnmic][gnmic-repo] :material-book: [https://gnmic.openconfig.net][gnmic-main-site]

In this post I'd like to give you a brief overview of gNMIc's core features and share my thoughts on what we expect to happen with gNMIc moving under the Openconfig's wing.

Using guestfish container image

rdodin — Tue, 22 Feb 2022 00:00:00 +0000

Using guestfish container image

Once in a while, one still needs to get down to a VM-land and dust off some guestfish skills.

Like today I got the IPInfusion OcNOS qcow2 image whose devs decided it is best to use VNC console by default. VNC console for a text-based terminal...

So along come guestfish commands.

DIY YANG Browser

rdodin — Wed, 26 Jan 2022 00:00:00 +0000

DIY YANG Browser

Here comes the second episode of the NetRel show: NetRel episode 002 - DIY YANG Browser. Be ready to dive into the paths we took to create a YANG Browser for Nokia SR Linux platform.

YANG data models are the map one should use when looking for their way to configure or retrieve any data on SR Linux system. A central role that is given to YANG in SR Linux demands a convenient interface to browse, search through, and process these data models.

To answer these demands, we created a web portal - https://yang.srlinux.dev - it offers:

Fast Path Browser to effectively search through thousands of available YANG paths
Beautiful Tree Browser to navigate the tree representation of the entire YANG data model of SR Linux
Source .yang files neatly stored in nokia/srlinux-yang-models repository for programmatic access and code generation

Decoding gNMI with Wireshark

rdodin — Tue, 18 Jan 2022 00:00:00 +0000

Decoding gNMI with Wireshark

Okay, here goes my first attempt fitting the shoes of a content creator.

Please welcome the NetRel episode 001 - Decoding gNMI with Wireshark, it is a 35min journey of using Wireshark to parse the gNMI traffic (both non-secured and secured).

I won't spend your time explaining the first episode; instead, let me tell you what I want the NetRel series to be about. I am interested in covering the aspects of network automation that are not widely covered.

Using scrapligo with kubectl exec

rdodin — Tue, 17 Aug 2021 00:00:00 +0000

Using scrapligo with kubectl exec

As the networking industry is (slowly) moving towards forklifting networking functions to the cloud-native space we often become the witnesses of mixing decade old tools with cloud-native approaches and architectures.

This post is about one such crazy mixture of using screen scraping library scrapligo with kubectl exec and docker exec commands.

Nokia SR Linux goes public

rdodin — Wed, 21 Jul 2021 00:00:00 +0000

Nokia SR Linux goes public

It's been almost two years since Nokia announced its Data Center Fabric solution. The three-layered solution ranged from hardware platforms all the way up in the stack to the DC fabric lifecycle management suite - Fabric Services System (FSS).

At the very heart of the DC Fabric solution lies a purpose-built, modern Network OS - SR Linux.

SR Linux comes with quite some interesting and innovative ideas. By being able to design the NOS from the ground up, the product team was freed from the legacy burdens which will be there have they decided to built the NOS on top of the existing one. Features like:

YANG-first APIs
Protobuf based SDK
Disaggregated application stack
Programmable CLI

are the result of taking a fresh look at the modern data center networks and building the NOS for the Netdevops era.

No wonders engineers around the world wanted to play with SR Linux and take those features for a spin first hand. And today it is finally possible!

Network automation options in Go with scrapligo

rdodin — Wed, 09 Jun 2021 00:00:00 +0000

Network automation options in Go with scrapligo

Just recently the network automation folks witnessed a great library to be ported from Python to Go - scrapligo.

Been working on learning go a bit and have published scrapligo https://t.co/NDXQ6khxCr -- still a work in progress, but has been a fun learning experience! Check it out and let me know what ya think! 🤠
— Carl Montanari (@carlrmontanari) May 19, 2021

For me personally this was a pivotal point because with scrapligo the Go-minded netengs can now automate their networks with a solid and performant library.

One of the things that scrapligo packs is, of course, the ability to reliably talk to the network devices using the same command line interface as a human would normally do. That means that scrapligo would send and receive the pieces of data that an operator would send/receive if they were connected with a terminal over SSH.