3 min read

Automating internal certificates with Cloudflare DNS-01

Internal names can have a normal, publicly trusted certificate. Let's Encrypt will issue one when you prove you control the DNS label directly under the zone, such as lab in lab.example.com. That is the top-level name this script groups on. You do not prove anything about the host itself. For a machine that is not on the internet, the proof is a DNS record instead of a web request.

DNS-01 asks for a TXT record at _acme-challenge under that label. Nothing connects to the server. It can be on a private address, behind a firewall, or powered off. Certbot's Cloudflare plugin creates the record, waits, and deletes it. You keep a list of the names you want. A weekly timer renews them.

What you need

  • A Cloudflare zone. In the script below, that zone is exactly two DNS labels, such as example.com.
  • Certbot and the certbot-dns-cloudflare plugin.
  • An API token for that one zone, with only Zone:Read and DNS:Edit. Not an account-wide token.

Directory

Make one directory and do every step below inside it. This example uses certs:

mkdir certs
cd certs

You will add three files: cloudflare.ini, issue.conf, and issue.sh. The first run of ./issue.sh creates everything else.

Files

Create cloudflare.ini:

# Cloudflare API token used by Certbot
# Token needs Zone:Read and DNS:Edit on example.com only.
dns_cloudflare_api_token = CLOUDFLARE_DNS_EDIT_TOKEN

Create issue.conf beside it. Settings go above : DOMAINS. One hostname per line goes below it. A setting whose name is a label directly under the zone chooses that group's output folder. If you skip it, the folder name is the label.

ZONE=example.com
EMAIL=you@example.com
CREDENTIALS=cloudflare.ini
CERTBOT_DIR=.letsencrypt
PROPAGATION_SECONDS=30
P12_PASSWORD=change-me
lab=services/lab

: DOMAINS
app.lab.example.com
db.lab.example.com

lab is the label under example.com, so both names share one certificate and land in services/lab. A name such as printer.example.com gets its own certificate in a folder called printer.

Issue

Save this as issue.sh, run chmod 700 issue.sh, then run ./issue.sh.

#!/bin/bash
set -eu

cd "$(dirname "$0")"

CONF="${1:-issue.conf}"
# Settings only. The hostname list below the marker must not be executed.
eval "$(awk '/^: DOMAINS$/ { exit } { print }' "$CONF")"

mkdir -p logs
LOG="logs/issue-$(date +%Y%m%d-%H%M%S).log"
exec > >(tee -a "$LOG") 2>&1
echo "logging to $LOG"

: "${ZONE:?ZONE is not set in $CONF}"
: "${EMAIL:?EMAIL is not set in $CONF}"
: "${CREDENTIALS:?CREDENTIALS is not set in $CONF}"
: "${CERTBOT_DIR:?CERTBOT_DIR is not set in $CONF}"
: "${PROPAGATION_SECONDS:?PROPAGATION_SECONDS is not set in $CONF}"

if [ ! -s "$CREDENTIALS" ]; then
  echo "$CREDENTIALS is missing" >&2
  exit 1
fi

NAMES=$(awk 'f && $0 !~ /^[[:space:]]*#/ && NF { gsub(/[[:space:]]/, ""); print } /^: DOMAINS$/ { f = 1 }' "$CONF")

if [ -z "$NAMES" ]; then
  echo "$CONF has no hostnames under DOMAINS" >&2
  exit 1
fi

# Group by the label directly under ZONE. ZONE is two labels, so that
# label is field (n - 2). A folder for a label comes from $CONF; otherwise
# the folder is the label itself.
echo "$NAMES" | awk -F. -v zone="$ZONE" '
  BEGIN { zn = split(zone, zp, ".") }
  {
    n = split($0, p, ".")
    if (n <= zn) next
    g = p[n - zn]
    group[g] = group[g] (group[g] ? " " : "") $0
  }
  END { for (g in group) print g "\t" group[g] }
' | while IFS=$(printf '\t') read -r GROUP DOMAINS; do
  DEST=$(awk -F= -v g="$GROUP" '
    /^: DOMAINS$/ { exit }
    $0 ~ /^[[:space:]]*#/ { next }
    $1 == g { sub(/^[^=]*=/, ""); print; found = 1; exit }
    END { if (!found) print g }
  ' "$CONF")

  mkdir -p "$DEST"

  # Saved into the renewal config, so `certbot renew` re-files on its own.
  # $RENEWED_LINEAGE is set only when the hook fires after a renewal.
  HOOK="$PWD/$CERTBOT_DIR/deploy-$GROUP.sh"
  cat > "$HOOK" <<EOF
#!/bin/sh
set -eu
LIVE="\${RENEWED_LINEAGE:-$PWD/$CERTBOT_DIR/live/$GROUP}"
cp "\$LIVE/fullchain.pem" "$PWD/$DEST/fullchain.pem"
cp "\$LIVE/privkey.pem" "$PWD/$DEST/privkey.pem"
chmod 600 "$PWD/$DEST/privkey.pem"
openssl pkcs12 -export \\
  -in "\$LIVE/fullchain.pem" \\
  -inkey "\$LIVE/privkey.pem" \\
  -out "$PWD/$DEST/$GROUP.p12" \\
  -name "$GROUP" \\
  -passout "pass:${P12_PASSWORD}"
chmod 600 "$PWD/$DEST/$GROUP.p12"
EOF
  chmod 700 "$HOOK"

  set -- $DOMAINS
  ARGS=""
  for d in "$@"; do
    ARGS="$ARGS -d $d"
  done

  certbot certonly \
    --non-interactive --agree-tos -m "$EMAIL" \
    --dns-cloudflare \
    --dns-cloudflare-credentials "$PWD/$CREDENTIALS" \
    --dns-cloudflare-propagation-seconds "$PROPAGATION_SECONDS" \
    --cert-name "$GROUP" \
    --deploy-hook "$HOOK" \
    --config-dir "$PWD/$CERTBOT_DIR" \
    --work-dir "$PWD/$CERTBOT_DIR/work" \
    --logs-dir "$PWD/$CERTBOT_DIR/logs" \
    $ARGS

  "$HOOK"
  echo "wrote $DEST ($DOMAINS)"
done

# `certbot renew` trusts the name list in the renewal config, not this file.
# Rewrite it so names added or removed since issuance take effect, and drop a
# lineage outright once none of its names remain. Files already written stay.
find "$CERTBOT_DIR/renewal" -name '*.conf' -type f 2>/dev/null | while read -r RC; do
  LINEAGE=$(basename "$RC" .conf)
  WANTED=$(echo "$NAMES" | awk -F. -v zone="$ZONE" -v lineage="$LINEAGE" '
    BEGIN { zn = split(zone, zp, ".") }
    {
      n = split($0, p, ".")
      if (n > zn && p[n - zn] == lineage) print
    }')

  if [ -z "$WANTED" ]; then
    certbot delete --non-interactive --cert-name "$LINEAGE" \
      --config-dir "$PWD/$CERTBOT_DIR" \
      --work-dir "$PWD/$CERTBOT_DIR/work" \
      --logs-dir "$PWD/$CERTBOT_DIR/logs"
    echo "removed $LINEAGE from renewal; its files were left in place"
    continue
  fi

  WANTED="$WANTED" awk '
    /^\[\[/ { exit }
    { print }
    END {
      print ""
      n = split(ENVIRON["WANTED"], w, "\n")
      for (i = 1; i <= n; i++) if (w[i] != "") print "[[webroot_map]]\n" w[i] " = None\n"
    }
  ' "$RC" > "$RC.tmp" && mv "$RC.tmp" "$RC"
done

Layout

After ./issue.sh finishes, certs looks like this:

certs/
  issue.sh
  issue.conf
  cloudflare.ini
  logs/
    issue-YYYYMMDD-HHMMSS.log
  .letsencrypt/
    deploy-lab.sh
    live/lab/
    renewal/lab.conf
    work/
    logs/
  services/lab/
    fullchain.pem
    privkey.pem
    lab.p12

The three files at the top are the ones you wrote. logs/ is the script transcript. .letsencrypt/ is Certbot's certificate, renewal config, and deploy hook. services/lab/ is the copy a service should read. A name such as printer.example.com adds a printer/ directory next to services/ instead of inside it.

Renew

Issuing once is not the point. The certificate lasts ninety days. Renewal reuses the config Certbot already saved:

certbot renew \
  --config-dir .letsencrypt \
  --work-dir .letsencrypt/work \
  --logs-dir .letsencrypt/logs

Run that from the same directory, weekly. Certbot only renews certificates inside the thirty-day window and skips the rest. When a certificate actually changes, the deploy hook copies the new files into place. It does not reload your service. Add that to the hook if a service needs it.

On Linux, use a systemd timer or cron. On macOS, use a LaunchAgent. Weekly is enough. Setup notes are at <https://certbot.org/renewal-setup>.

Changing the list

Edit the names under : DOMAINS and run ./issue.sh again. Names you add join a certificate and get renewed. Names you remove are dropped from renewal. The script does not delete fullchain.pem, privkey.pem, or the PKCS#12 file. Leave deletion to a person. A typo in the list should not wipe a live key.

Do not point the script at every record in the zone. Old tests and leftovers would get publicly trusted certificates. Add a name when you mean to, and remove it when you mean to.

A wildcard is supported even though this example does not use one. Add *.toplevel.example.com under : DOMAINS, where toplevel is the label directly under the zone, such as *.lab.example.com. DNS-01 can validate that name. One certificate then covers every host under that label.