WGet Weave to kubectl apply

4 min readJan 26, 2023

A cryptic title, isn’t it?

In my Certified Kubernetes for Admin course, I needed to install Weave in a lab. Weave can be installed as pods and to do so, you can find YAML files from GitHub here.

Pipe wget to kubectl apply

Now, you can download the latest YAML file and feed that to kubectl apply -f. But, I thought: why not do it on one line? I had to extend my Bash knowledge for this a bit.

First, kubectl apply -f expects a filename as an input. I found this StackOverflow article: Need some explaination of kubectl stdin and pipe, which explains that a simple dash is a POSIX way to make kubectl interpret this as Standard In. So, instead of using a filename, I need to use kubectl apply -f -.

Second, wget by default outputs the download to a file. The Bealdung article Output Document and Headers to Stdout Using Wget tells that the argument — output-document also can output it to the dash which in this case means Standard Out. To not have the logging/output of wget itself in the output, we need to run it in quite-mode, using the -q option.

So tying this together, I get:

wget -q https://github.com/weaveworks/weave/releases/download/v2.8.1/weave-daemonset-k8s-1.11.yaml --output-document - | kubectl apply -f -

Get Weave Version

Reaching this point, it would be nice to get the correct version of Weave. In fact, to automatically find the latest version.

The URL https://github.com/weaveworks/weave/releases/latest will get you through a redirect to the page with the latest version of Weave.

So, I need to curl it using a redirect, and since I do not want to actually download the whole page to file, but just output it to Standard In, I use:

$ curl -Ls https://github.com/weaveworks/weave/releases/latest

Here is:

-L: follow redirects
-s: Silent

See Get final URL after curl is redirected.

Then I just want to grep only the first occurrence of the string with URI to the tag release, identified by weaveworks/weave/releases/tag/:

export WEAVE_TAG="weaveworks/weave/releases/tag/"
WEAVE_VER=$(curl -Ls https://github.com/weaveworks/weave/releases/latest |grep $WEAVE_TAG -m 1)
echo $WEAVE_VER

See Grep only the first match and stop.

This outputs a line like:

<meta name="twitter:image:src" content="https://opengraph.githubassets.com/f5159cd2a19f17e168b575a6ebc117627ff93b16609e21ac28cb5e427b45f804/weaveworks/weave/releases/tag/v2.8.1" /><meta name="twitter:site" content="@github" /><meta name="twitter:card" content="summary_large_image" /><meta name="twitter:title" content="Release Weave Net 2.8.1 · weaveworks/weave" /><meta name="twitter:description" content="Release 2.8.1

And thus I want to strip everything up to the content of the $WEAVE_TAG variable. This can be done using parameter expansion in Bash. The #-character will strip everything (using wildcard *) up to the given text:

$ export WEAVE_VER=${WEAVE_VER#*$WEAVE_TAG}
$ echo $WEAVE_VER

Which outputs:

v2.8.1" /><meta name="twitter:site" content="@github" /><meta name="twitter:card" content="summary_large_image" /><meta name="twitter:title" content="Release Weave Net 2.8.1 · weaveworks/weave" /><meta name="twitter:description" content="Release 2.8.1

Now, removing everything after a certain string can be done using the % character. In this case, I need to use double %%-characters to move the largest part after the first occurrence of the double-quote:

$ export WEAVE_VER=${WEAVE_VER%%\"*}
$ echo $WEAVE_VER

Which outputs:

v2.8.1

This what I actually need assemble the proper URLs. I combined that in a script:

#!/bin/bash
SCRIPTPATH=$(dirname $0)
export WEAVE_TAG="weaveworks/weave/releases/tag/"
# Get first occurence of the WEAVE_TAG
WEAVE_VER=$(curl -Ls https://github.com/weaveworks/weave/releases/latest |grep $WEAVE_TAG -m 1)
# Remove everything upto the WEAVE_TAG
WEAVE_VER=${WEAVE_VER#*$WEAVE_TAG}
# Remove everyting after the first occurence of the double-quote
WEAVE_VER=${WEAVE_VER%%\"*}
# Echo Result
echo Weave Version: $WEAVE_VER

See Delete everyting preceding and including a certain substring from variable, Extract substring in Bash, and especially How can I remove all text after a character in bash? on stripping strings in Bash.

Get Weave YAML Version

Now, for a specific Weave version, there are apparently a few follow-up versions of the particular YAML to install the Weave DeamonSet. Reading the HTML source file of the latest Weave version page, I found that these YAMLs are on the versioned URL: https://github.com/weaveworks/weave/releases/expanded_assets/v2.8.1

Bringing the earlier string handling knowledge into play for the YAML, I got the following script snippet:

GIT_URL=https://github.com
WEAVE_GIT_URL="$GIT_URL/weaveworks/weave"
WEAVE_REL_URL="$WEAVE_GIT_URL/releases"
export WEAVE_YAML_TAG=weave-daemonset-k8s-
# Get the first occurence of the WEAVE_YAML_TAG on the expanded_assests of the WEAVE_VER version
WEAVE_YAML_URL=$(curl -Ls "$WEAVE_REL_URL/expanded_assets/$WEAVE_VER" |grep $WEAVE_YAML_TAG -m 1)
# Strip upto the first double-quote
WEAVE_YAML_URL=${WEAVE_YAML_URL#*\"}
# Strip after the following double quote:
WEAVE_YAML_URL=${WEAVE_YAML_URL%%\"*}
# Add the GIT URL in front:
WEAVE_YAML_URL="$GIT_URL/${WEAVE_YAML_URL}"
# Echo the result:
echo Latest Weave Deamon Set YAML: $WEAVE_YAML_URL

Get and Install the latest Weave DeamonSet

As a result, I put this all down into a script on my Kubernetes GitHub Repository: scripts/weave/getLatestWeaveVersion.sh. This will determine the latest version of Weave, and the URL to the DaemonSet YAML. With that URL it feeds the YAML into the kubectl apply command.

Conclusion

I tried this in a lab, where it did install Weave. It seems to work.

But, the main reason for this article is to show how to pipe a download with wget into kubectl apply and how to determine the latest version of a tool by investigating the GitHub page, using curl, grep and bash parameter expansion in shell variables.
This can be very helpful in creating install scripts for tools you can download from a GitHub website.