Discussion
Loading...

Post

Log in
  • About
  • Code of conduct
  • Privacy
  • Users
  • Instances
  • About Bonfire
Darius Kazemi
Darius Kazemi
@darius@friend.camp  ·  activity timestamp 14 hours ago

Seeking advice. I have many services that require cert renewal for HTTPS. My usual workflow for nginx is:

1) set up a cron job to call certbot monthly (strictly more often than needed for a 90 day cert but I'm lazy)
2) 91 days later, re-cert fails, realize that there was a bug in my script, fix it
3) 91 days later, re-cert fails, realize there was a bug in my script, fix it
4) at this point it probably now works forever

Is there a good way to test things properly so I skip steps 2 and 3??

  • Copy link
  • Flag this post
  • Block
Renaud Chaput
Renaud Chaput
@renchap@oisaur.com replied  ·  activity timestamp 5 hours ago

@darius if you want to stick with nginx, try https://nginx.org/en/docs/http/ngx_http_acme_module.html which will handle LE certificates directly within nginx. Caddy or Apache do the same if you can/want to change your web server.
As do monitoring, updown.io does check certs and emails you if they are about to expire

Module ngx_http_acme_module

  • Copy link
  • Flag this comment
  • Block
Pomax
Pomax
@TheRealPomax@mastodon.social replied  ·  activity timestamp 10 hours ago

@darius is "have you looked at Caddy?" an option?

  • Copy link
  • Flag this comment
  • Block
Dan Cassidy 🦌
Dan Cassidy 🦌
@whimsy@chitter.xyz replied  ·  activity timestamp 11 hours ago

@darius I might be missing something here but, at step 1, run the script that cron will run in order to do your initial certificate setup. If that works then it should work a month later when cron runs it. The only variable is if you've actually set cron up to run the correct script at the correct time, which is admittedly tricky.

To solve that problem, if you're on debian or a derivative, put your script in /etc/cron.monthly/ and chmod +x it. Debian has a built-in cron rule to run everything that's executable by root in that directory once a month. Then test it by running sudo /etc/cron.monthly/your_script. If it runs ok and sets up the certificate then it should work when cron runs it.

Also, answering the question you didn't ask, iirc letsencrypt actually recommend you run certbot every day. At least on my servers it runs every day and I will have done that for good reason. There is an /etc/cron.daily for that purpose on debian.

  • Copy link
  • Flag this comment
  • Block
Darius Kazemi
Darius Kazemi
@darius@friend.camp replied  ·  activity timestamp 10 hours ago

@whimsy it is often a permissions issue where cron is running as the wrong user (for example I run the script manually with sudo and it works but then I install it in crontab running as some other user)

  • Copy link
  • Flag this comment
  • Block
Dan Cassidy 🦌
Dan Cassidy 🦌
@whimsy@chitter.xyz replied  ·  activity timestamp 10 hours ago

@darius I don't think I have any helpful advice then. Hope you find something that works for you 😊

  • Copy link
  • Flag this comment
  • Block
Darius Kazemi
Darius Kazemi
@darius@friend.camp replied  ·  activity timestamp 10 hours ago

@whimsy it is good to know letsencrypt encourages daily recert! I was worried i was being a bad citizen doing monthly

  • Copy link
  • Flag this comment
  • Block
Dan Cassidy 🦌
Dan Cassidy 🦌
@whimsy@chitter.xyz replied  ·  activity timestamp 10 hours ago

@darius I can't find the recommendation right now but monthly is definitely ok! You should get the same certificate back every time until the certificate is a certain age so there's no actual renewal most of the time, but it gives the service an opportunity to renew the certificate early if that's necessary for some reason.

  • Copy link
  • Flag this comment
  • Block
nintegge
nintegge
@nintegge@post.lurk.org replied  ·  activity timestamp 13 hours ago

@darius do your first certificate deployment with your cronjob script. Add to cron only after it works. (Basically intentional „test in prod“). If you run certificate renewal from a major script which runs bespoke ones: monitor the first one for failure (eg by setting a working email alias for „root“ in your /etc/aliases and setting „set -e“ or other hard fail options in your major script).

  • Copy link
  • Flag this comment
  • Block
Justin 🇻🇦
Justin 🇻🇦
@jforseth210@rcsocial.net replied  ·  activity timestamp 14 hours ago

@darius
1. Put everything behind Caddy
There is no step 2. I know it doesn't work for every use case, but it's super hand for me

  • Copy link
  • Flag this comment
  • Block
A
A
@a@852260996.91268476.xyz replied  ·  activity timestamp 14 hours ago

@darius@friend.camp I used the exact same method until I discovered linuxserver swag and I barely have to worry about it

  • Copy link
  • Flag this comment
  • Block
technomancy
technomancy
@technomancy@hey.hagelb.org replied  ·  activity timestamp 14 hours ago

@darius apologies if this is unhelpful but honestly the main reason I switched from nginx to caddy is that caddy jumps straight to step 4 for you

  • Copy link
  • Flag this comment
  • Block
Darius Kazemi
Darius Kazemi
@darius@friend.camp replied  ·  activity timestamp 14 hours ago

I did not know about out-of-band monitoring services for this! Makes sense that they exist... might have to try one

  • Copy link
  • Flag this comment
  • Block
Eli the Bearded
Eli the Bearded
@elithebearded@fed.qaz.red replied  ·  activity timestamp 13 hours ago

@darius

I monitor from an old phone that I keep around for the games with this:
https://f-droid.org/packages/net.ibbaa.keepitup

  • Copy link
  • Flag this comment
  • Block
erik
erik
@iamdoon@mspsocial.net replied  ·  activity timestamp 14 hours ago

@darius I typically pair auto-renewal with an out-of-band monitor via something like Uptime Kuma or Zabbix - either of these can check cert expiry and alert if the expiry date is within some range.

This doesn’t fix script errors, but will help to catch all manner of failure modes.

  • Copy link
  • Flag this comment
  • Block
infinite love ⴳ
infinite love ⴳ
@trwnh@mastodon.social replied  ·  activity timestamp 14 hours ago

@darius never make mistakes

in all seriousness though, does letsencrypt have a sandbox environment? or are you actually verifying failures at 30/60/90 days before the observable failure on day 91?

  • Copy link
  • Flag this comment
  • Block
David Fleetwood - RG Admin
David Fleetwood - RG Admin
@reflex@retrogaming.social replied  ·  activity timestamp 14 hours ago

@trwnh @darius It does, there is a flag you can use when calling it but it escapes me at the moment. Best way to test a script with LE.

  • Copy link
  • Flag this comment
  • Block
Darius Kazemi
Darius Kazemi
@darius@friend.camp replied  ·  activity timestamp 14 hours ago

@trwnh I forget this stuff exists until the observable failure happens!

  • Copy link
  • Flag this comment
  • Block
Darius Kazemi
Darius Kazemi
@darius@friend.camp replied  ·  activity timestamp 14 hours ago

@trwnh I forget this stuff exists until the observable failure happens!

  • Copy link
  • Flag this comment
  • Block
infinite love ⴳ
infinite love ⴳ
@trwnh@mastodon.social replied  ·  activity timestamp 14 hours ago

@darius maybe certbot can email you on failure? or your cronjob/timer/whatever can detect failure from certbot's output and email you?

  • Copy link
  • Flag this comment
  • Block
Darius Kazemi
Darius Kazemi
@darius@friend.camp replied  ·  activity timestamp 14 hours ago

@trwnh maybe! seems like there are out-of-band monitor services too that I might look into for my more critical certs

  • Copy link
  • Flag this comment
  • Block

bonfire.cafe

A space for Bonfire maintainers and contributors to communicate

bonfire.cafe: About · Code of conduct · Privacy · Users · Instances
Bonfire social · 1.0.1-alpha.37 no JS en
Automatic federation enabled
Log in
  • Explore
  • About
  • Members
  • Code of Conduct