@@ -6,6 +6,132 @@ Interact with [NearlyFreeSpeech.NET][]'s [API][] from the command line.

 [NearlyFreeSpeech.NET]: https://www.nearlyfreespeech.net

 [API]: https://en.wikipedia.org/wiki/API

+## Goals

+

+`nfsn-utils` has three main goals:

+

+1.  **Be portable** (even to things like [routers][]).

+

+    POSIX shell is used as glue for standard utilities. See

+    [dependencies](#dependencies).

+

+2.  **Be modular** where it makes sense.

+

+    [`nfsn-send`](#nfsn-send) is a general purpose [NearlyFreeSpeech.NET][]

+    [API][] wrapper. [`nfsn-dns-update`](#nfsn-dns-update) is a general purpose

+    [NearlyFreeSpeech.NET][] DNS update utility.

+

+3.  **Be opinionated with sane defaults** where it makes sense.

+

+    The smaller utilities assume things like that you want to use [standard

+    email addresses][].

+

+4.  **Be easily auditable**.

+

+    The scripts are well abstracted and no more than about 100 lines of code.

+

+[routers]: https://openwrt.org

+[standard email addresses]: https://www.ietf.org/rfc/rfc2142.txt

+

+## Prerequisites

+

+### API key

+

+As described in the [NearlyFreeSpeech.NET][] [documentation][], one needs to

+submit a [free assistance request][] to obtain an API key.

+

+Place the credentials in the environment variables `NFSN_LOGIN` and

+`NFSN_API_KEY` or in the file `./.nfsn-api` or `$HOME/.nfsn-api` (location

+overridable by the `NFSN_CREDENTIALS_PATH` environment variable). This file

+should be a JSON file consisting of an object with the keys `login` and

+`api-key`. (The file format and default location is compatible with

+[WebService::NFSN][] and [python-nfsn][].)

+

+[documentation]: https://members.nearlyfreespeech.net/wiki/API/Introduction

+[free assistance request]: https://members.nearlyfreespeech.net/support/assist?tag=apikey

+[WebService::NFSN]: https://metacpan.org/pod/WebService::NFSN#INTERFACE

+[python-nfsn]: https://github.com/ktdreyer/python-nfsn#authentication

+

+### Dependencies

+

+-  Unix-like environment (in particular, `/dev/urandom`).

+-  [POSIX utilities][] with `date` supporting `+%s` (such as GNU `date`).

+-  `sha1sum` (for instance, the one in `coreutils`).

+-  [curl][].

+-  [jq][].

+-  `certbot` (only needed for `nfsn-dns-certbot*`).

+

+[POSIX utilities]: http://pubs.opengroup.org/onlinepubs/9699919799/idx/utilities.html

+[curl]: https://curl.haxx.se

+[jq]: https://github.com/stedolan/jq

+

+## Included programs

+

+Dependency graph:

+

+![included programs](doc/included-programs.dot.png)

+

+### `nfsn-send`

+

+Wraps the Requests, Responses and Authentication described in the

+[NearlyFreeSpeech.NET][] [documentation][].

+

+### `nfsn-dns-update`

+

+Updates several DNS records and outputs what data was actually changed.

+

+### `nfsn-dns-a`

+

+Updates DNS [A][] records, used to map hostnames to an IPv4 address.

+

+[A]: https://en.wikipedia.org/wiki/List_of_DNS_record_types#A

+

+### `nfsn-dns-spf`

+

+Updates DNS [SPF][] records, used for email authorization (specifying who is

+allowed to send mail from a domain).

+

+[SPF]: https://en.wikipedia.org/wiki/Sender_Policy_Framework

+

+### `nfsn-dns-dkim`

+

+Updates DNS [DKIM][] records, used for email authentication (using digital

+signatures).

+

+[DKIM]: https://en.wikipedia.org/wiki/DomainKeys_Identified_Mail

+

+### `nfsn-dns-dmarc`

+

+Updates DNS [DMARC][] records, extending [SPF][] and [DKIM][] by specifying

+failure policy and reporting.

+

+See also the [dmarc.org FAQ][].

+

+[DMARC]: https://en.wikipedia.org/wiki/DMARC

+[dmarc.org FAQ]: https://dmarc.org/wiki/FAQ#Sender_Questions

+

+### `nfsn-dns-certbot*`

+

+`nfsn-dns-certbot` calls [certbot][] (the [Electronic Frontier Foundation][]'s

+(EFF) [Let's Encrypt][] client, for getting HTTPS certificates) in [manual

+mode][] to make it use `nfsn-utils` to update DNS records in order to fullfill

+the [`dns-01` challenge][]. It does this by registering

+`nfsn-dns-certbot-{auth,cleanup}` as [hooks][].

+

+By default, the `auth` hook sleeps for 30 seconds to let the DNS records

+propagate. This can be overridden with the environment variable

+`NFSN_DNS_CERTBOT_AUTH_SLEEP`.

+

+Given that `nfsn-dns-certbot` has successfully run once, running `certbot

+renew` will suffice to renew the certificates.

+

+[certbot]: https://certbot.eff.org

+[Electronic Frontier Foundation]: https://www.eff.org

+[Let's Encrypt]: https://letsencrypt.org

+[manual mode]: https://certbot.eff.org/docs/using.html#manual

+[`dns-01` challenge]: https://tools.ietf.org/html/draft-ietf-acme-acme-03#section-7.4

+[hooks]: https://certbot.eff.org/docs/using.html#hooks

+

 ## License

 Licensed under the [ISC License][] unless otherwise noted, see the

new file mode 100644

@@ -0,0 +1,20 @@

+digraph "included programs" {

+    node [shape=box]

+

+    {

+        { "nfsn-dns-a" [URL="nfsn-dns-a"] }

+        { "nfsn-dns-mx" [URL="nfsn-dns-mx"] }

+        { "nfsn-dns-spf" [URL="nfsn-dns-spf"] }

+        { "nfsn-dns-dkim" [URL="nfsn-dns-dkim"] }

+        { "nfsn-dns-dmarc" [URL="nfsn-dns-dmarc"] }

+    }

+    -> { "nfsn-dns-update" [URL="nfsn-dns-update"] }

+    -> { "nfsn-send" [URL="nfsn-send"] }

+

+    { "nfsn-dns-certbot" [URL="nfsn-dns-certbot"] }

+    -> {

+        { "nfsn-dns-certbot-auth" [URL="nfsn-dns-certbot-auth"] }

+        { "nfsn-dns-certbot-cleanup" [URL="nfsn-dns-certbot-cleanup"] }

+    }

+    -> { "nfsn-send" [URL="nfsn-send"] }

+}

new file mode 100644

Binary files /dev/null and b/docs/included-programs.dot.png differ

new file mode 100755

@@ -0,0 +1,21 @@

+#!/bin/sh

+set -euC

+

+# nfsn-dns-a DOMAIN NAME [IP]...

+

+# nfsn-dns-a "example.com" "git" \

+#     "$(curl -s "https://api.ipify.org")"

+# nfsn-dns-a "example.com" "" \

+#     "185.199.108.153" \

+#     "185.199.109.153" \

+#     "185.199.110.153" \

+#     "185.199.111.153"

+

+# Arguments.

+

+domain="$1" ; shift

+name="$1" ; shift

+

+# Update.

+

+nfsn-dns-update "$domain" "$name" "A" '' "$@"

new file mode 100755

@@ -0,0 +1,47 @@

+#!/bin/sh

+set -euC

+

+# nfsn-dns-certbot DOMAIN NAME EMAIL_NAME INSTALLER [CERTBOT_ARG]...

+

+# nfsn-dns-certbot "example.com" "" "" ""

+# nfsn-dns-certbot "example.com" "git" "" "apache" --quiet

+

+# Arguments.

+

+domain="$1" ; shift

+name="$1" ; shift

+email_name="${1:-"hostmaster"}" ; shift

+installer="${1:-}" ; shift

+

+# Certbot.

+

+host="${name:+"$name."}$domain"

+dir="$(cd "$(dirname "$0")" ; pwd)"

+

+certbot certonly \

+    --non-interactive \

+    --email "$email_name@$host" \

+    --agree-tos \

+    --manual \

+    --manual-public-ip-logging-ok \

+    --manual-auth-hook "$dir/nfsn-dns-certbot-auth" \

+    --manual-cleanup-hook "$dir/nfsn-dns-certbot-cleanup" \

+    --preferred-challenges="dns" \

+    --domains "$host" \

+    "$@"

+

+if [ -n "$installer" ]

+then

+    certbot install \

+        --cert-name "$host" \

+        --installer "$installer" \

+

+    certbot enhance \

+        --non-interactive \

+        --cert-name "$host" \

+        --domain "$host" \

+        --installer "$installer" \

+        --redirect \

+        --hsts \

+        --uir

+fi

new file mode 100755

@@ -0,0 +1,20 @@

+#!/bin/sh

+set -euC

+

+# This is a naive guess. A better implementation would use e.g. the Public

+# Suffix List.

+re='\(\(.*\)\.\)\?\([^.]\+\.[^.]\+\)'

+domain="$(echo "$CERTBOT_DOMAIN" | sed -n "s/$re/\3/p")"

+name="$(echo "$CERTBOT_DOMAIN" | sed -n "s/$re/\2/p")"

+

+name="_acme-challenge${name:+".$name"}"

+data="$CERTBOT_VALIDATION"

+

+dir="$(cd "$(dirname "$0")" ; pwd)"

+

+"$dir/nfsn-send" "POST" "/dns/$domain/addRR" \

+    "name" "$name" \

+    "type" "TXT" \

+    "data" "$data"

+

+sleep "${NFSN_DNS_CERTBOT_AUTH_SLEEP:-"30"}"

new file mode 100755

@@ -0,0 +1,18 @@

+#!/bin/sh

+set -euC

+

+# This is a naive guess. A better implementation would use e.g. the Public

+# Suffix List.

+re='\(\(.*\)\.\)\?\([^.]\+\.[^.]\+\)'

+domain="$(echo "$CERTBOT_DOMAIN" | sed -n "s/$re/\3/p")"

+name="$(echo "$CERTBOT_DOMAIN" | sed -n "s/$re/\2/p")"

+

+name="_acme-challenge${name:+".$name"}"

+data="$CERTBOT_VALIDATION"

+

+dir="$(cd "$(dirname "$0")" ; pwd)"

+

+"$dir/nfsn-send" "POST" "/dns/$domain/removeRR" \

+    "name" "$name" \

+    "type" "TXT" \

+    "data" "$data"

new file mode 100755

@@ -0,0 +1,23 @@

+#!/bin/sh

+set -euC

+

+# nfsn-dns-dkim DOMAIN NAME SELECTOR KEY_TYPE KEY_PUB

+

+# nfsn-dns-dkim "example.com" "" "k1" "rsa" "$key_pub"

+

+# Arguments.

+

+domain="$1" ; shift

+name="$1" ; shift

+selector="$1" ; shift

+key_type="$1" ; shift

+key_pub="$1" ; shift

+

+# Process.

+

+name="$selector._domainkey${name:+".$name"}"

+data="v=DKIM1; k=$key_type; p=$key_pub"

+

+# Update.

+

+nfsn-dns-update "$domain" "$name" "TXT" '^v=DKIM1' "$data"

new file mode 100755

@@ -0,0 +1,31 @@

+#!/bin/sh

+set -euC

+

+# nfsn-dns-dmarc DOMAIN NAME EMAIL_NAME POLICY

+

+# nfsn-dns-dmarc "example.com" "" "" "none"

+# nfsn-dns-dmarc "example.com" "" "" "quarantine"

+# nfsn-dns-dmarc "example.com" "" "" "reject"

+

+# Arguments.

+

+domain="$1" ; shift

+name="$1" ; shift

+email_name="${1:-"postmaster"}" ; shift

+policy="$1" ; shift

+

+# Process.

+

+name="_dmarc${name:+".$name"}"

+data="$(

+    printf "%s" \

+        "v=DMARC1; " \

+        "p=$policy; " \

+        "sp=$policy; " \

+        "pct=100; " \

+        "rua=mailto:$email_name@$domain"

+)"

+

+# Update.

+

+nfsn-dns-update "$domain" "$name" "TXT" '' "$data"

new file mode 100755

@@ -0,0 +1,17 @@

+#!/bin/sh

+set -euC

+

+# nfsn-dns-mx DOMAIN NAME [MX_DATA]...

+

+# nfsn-dns-mx "example.com" "" \

+#     "10 mxa.mailgun.org." \

+#     "10 mxb.mailgun.org."

+

+# Arguments.

+

+domain="$1" ; shift

+name="$1" ; shift

+

+# Update.

+

+nfsn-dns-update "$domain" "$name" "MX" '' "$@"

new file mode 100755

@@ -0,0 +1,27 @@

+#!/bin/sh

+set -euC

+

+# nfsn-dns-spf DOMAIN NAME [INCLUDE]...

+

+# nfsn-dns-spf "example.com" "mailgun.org"

+

+# Arguments.

+

+domain="$1" ; shift

+name="$1" ; shift

+

+# Process.

+

+data="$(

+    printf -- "v=spf1"

+    for include in "$@"

+    do

+        printf -- " include:%s" "$include"

+    done

+    printf -- " -all"

+    printf -- "\n"

+)"

+

+# Update.

+

+nfsn-dns-update "$domain" "$name" "TXT" '^v=spf1' "$data"

new file mode 100755

@@ -0,0 +1,87 @@

+#!/bin/sh

+set -euC

+

+# nfsn-dns-update DOMAIN NAME TYPE FILTER_REGEX [DATA]...

+

+# nfsn-dns-update "example.com" "git" "A" '' \

+#     "$(curl -s "https://api.ipify.org")"

+

+# Arguments.

+

+domain="$1" ; shift

+name="$1" ; shift

+type="$1" ; shift

+filter_regex="$1" ; shift

+

+host="${name:+"$name."}$domain"

+

+data_new_list=""

+while [ "$#" -gt "0" ]

+do

+    data="$1" ; shift

+    data_new_list="$(

+        printf "%s${data_new_list:+"\n"}%s\n" \

+            "${data_new_list:-}" \

+            "$data"

+    )"

+done

+

+# Old data.

+

+data_old_response="$(

+    nfsn-send "POST" "/dns/$domain/listRRs" \

+        "name" "$name" \

+        "type" "$type"

+)"

+data_old_list="$(

+    printf "%s\n" "$data_old_response" \

+    | jq -r '

+        .[] |

+        if ."aux"

+        then

+            "\(."aux") "

+        else

+            ""

+        end

+        +

+        "\(."data"?)"

+    ' \

+    | grep "$filter_regex" \

+    || true

+)"

+

+# Update.

+

+if [ -n "$data_old_list" ]

+then

+    printf "%s\n" "$data_old_list" \

+    | while IFS= read -r data

+    do

+        if ! printf "%s\n" "$data_new_list" | grep -qFx "$data"

+        then

+            printf "Removing data: %s\n" "$data"

+            nfsn-send "POST" "/dns/$domain/removeRR" \

+                "name" "$name" \

+                "type" "$type" \

+                "data" "$data"

+        fi

+    done

+fi

+

+if [ -n "$data_new_list" ]

+then

+    printf "%s\n" "$data_new_list" \

+    | while IFS= read -r data

+    do

+        if ! printf "%s\n" "$data_old_list" | grep -qFx "$data"

+        then

+            printf "Adding data: %s\n" "$data"

+            nfsn-send "POST" "/dns/$domain/addRR" \

+                "name" "$name" \

+                "type" "$type" \

+                "data" "$data"

+        else

+            printf "Data already present: %s\n" "$data"

+        fi

+    done

+fi

new file mode 100755

@@ -0,0 +1,114 @@

+#!/bin/sh

+set -euC

+

+# nfsn-send METHOD REQUEST_URI [KEY VALUE]...

+

+# nfsn-send "GET" "member/$member/sites"

+# nfsn-send "POST" "dns/$domain/listRRs" \

+#     "name" "" \

+#     "type" "MX"

+

+# Arguments.

+

+method="$1" ; shift

+request_uri="$1" ; shift

+

+body=""

+while [ "$#" -gt "0" ]

+do

+    key="$1" ; shift

+    value="$1" ; shift

+    body="${body:-}${body:+"&"}$key=$(printf "%s" "$value" | jq -sRr '@uri')"

+done

+

+# Authentication.

+

+login=""

+api_key=""

+for auth in "${NFSN_CREDENTIALS_PATH:-}" ".nfsn-api" "$HOME/.nfsn-api"

+do

+    if [ -f "$auth" ]

+    then

+        login="$(jq -r '."login"' "$auth")"

+        api_key="$(jq -r '."api-key"' "$auth")"

+        break

+    fi

+done

+login="${NFSN_LOGIN:-"$login"}"

+api_key="${NFSN_API_KEY:-"$api_key"}"

+

+

+if [ -z "${login:-}" ] || [ -z "${api_key:-}" ]

+then

+    >&2 printf "%s: %s\n" \

+        "$0" \

+        "Could not find authentication credentials."

+    exit 1

+fi

+

+# URL.

+

+protocol="https"

+host="api.nearlyfreespeech.net"

+

+# protocol="http"

+# host="localhost:8080"

+# nc -l 8080 &

+

+url="$protocol://$host/$request_uri"

+

+# Data.

+

+body_hash="$(

+    printf "%s" "$body" \

+    | sha1sum -b \

+    | cut -d ' ' -f 1

+)"

+

+timestamp="$(

+    date "+%s"

+)"

+salt="$(

+    < "/dev/urandom" \

+    tr -dc "a-zA-Z0-9" \

+    | head -c 16

+)"

+

+hash="$(

+    printf "%s" \

+        "$login;$timestamp;$salt;$api_key;$request_uri;$body_hash" \

+    | sha1sum -b \

+    | cut -d ' ' -f 1

+)"

+

+header="X-NFSN-Authentication: $login;$timestamp;$salt;$hash"

+

+# Request.

+

+response="$(

+    curl \

+        --silent \

+        --request "$method" \

+        --header "$header" \

+        --data-raw "$body" \

+        "$url"

+)"

+

+error="$(printf "%s\n" "$response" | jq -r '."error"?')"

+debug="$(printf "%s\n" "$response" | jq -r '."debug"?')"

+

+if [ -n "$error" ]

+then

+    >&2 printf "%s" "$error"

+    if [ -n "$debug" ]

+    then

+        >&2 printf " %s" "$debug"

+    fi

+    >&2 printf "\n"

+    return 1

+fi

+

+if [ -n "$response" ]

+then

+    printf "%s\n" "$response"

+fi