I haven't written one yet, but the provided docs are pretty easy to follow:

1. Tailscale has their custom OIDC docs that tell you everything you need, plus the Webfinger setup: https://tailscale.com/kb/1240/sso-custom-oidc/

2. I set up Webfinger first, so assuming you're setting it up from scratch you can either run a Webfinger server yourself, or just configure the paths in whatever web server you have for your base domain. I didn't feel like running Yet Another Server and since the Tailnet's only for me I just plugged the following section into Caddy:

  @webfinger {
    path /.well-known/webfinger
    method GET HEAD
    query resource=acct:MY@EMAIL
  }
  rewrite @webfinger /webfinger.json
  header @webfinger {
    Content-Type "application/jrd+json"
    Access-Control-Allow-Origin "\*"
    X-Robots-Tag "noindex"
  }

Where webfinger.json is file containing the response tailscale is looking for from their doc. You can verify it works right at https://webfinger.net/lookup/ .

3. For Dex you can just set it up like any OIDC connection; Authelia was about the same but they have their own page: https://www.authelia.com/integration/openid-connect/tailscal...

Took me about an hour or two, most of that being wishy-washy on how I wanted to serve Webfinger.