When you spin up a fresh public Linux instance, automated scanners start knocking on your doors within milliseconds.

It doesn’t take much to get locked down properly, but missing a few basics can leave you exposed.

I’ve put together a straightforward guide covering the essential hardening steps from UFW and SSH key-only auth to secure Docker bindings and automated maintenance.


Original content by The Unknown Universe, written by a human and not AI generated!

  • mlg@lemmy.world
    link
    fedilink
    English
    arrow-up
    17
    ·
    20 hours ago

    I’ll have to find the post again (it was somewhere here on Lemmy) but UFW does not apply to docker containers.

    Docker bypasses UFW because it uses iptables and UFW doesn’t support firewall zones.

    Any docker/podman ports not bound to localhost will be publicly accessible even if you have a block rule on UFW.

    There are some wacky script workarounds but the real solution is to install & use a proper firewall like firewalld (firewall-cmd command) and not the joke utilities of Canonical/Ubuntu (ufw, netplan, etc).

    At which point you might as well chuck Ubuntu into the trash since you’d be replacing a bunch if its defaults anyway, and just stick with Debian or another quality stable distro like OpenSUSE.

    • notthebees@reddthat.com
      link
      fedilink
      arrow-up
      2
      ·
      2 hours ago

      Wouldn’t docker do the same with firewalld?

      It acknowledges that it exists and just still uses it’s own rules. You’d still have to configure it, configuration is just less stupid than ufw with docker.

      My question is, who thought this was acceptable behavior at docker?

    • TootGuitar@sh.itjust.works
      link
      fedilink
      arrow-up
      4
      arrow-down
      1
      ·
      17 hours ago

      I’ve been bitten by this before.

      OP, this guide reeks of inexperience. It’s admirable that you’re trying to help, but there are footguns in just about every section of the post.

  • nonentity@sh.itjust.works
    link
    fedilink
    arrow-up
    1
    ·
    13 hours ago

    Setup a dummy SSH daemon, instant black hole for a random span of 2~6 weeks for any IP that attempts to login as root.

  • talkingpumpkin@lemmy.world
    link
    fedilink
    arrow-up
    56
    arrow-down
    1
    ·
    2 days ago

    I really appreciate the effort but… how to say it?.. it might be useful to make sure you understand a topic before trying to teach it to others :)

    Some notes:

    Create a Non-Root Administrative User

    Not as important as most people think (local privilege escalation is a thing, and it’s also quite easy for your user to alias sudo to something that will capture your password the next time you type it).

    Basically: if an hostile actor gets a shell on your server, you are fucked anyway.

    sudo usermod -aG docker your_username

    This is the same as being root. Yep, the very same thing you went that whole length to avoid :)

    Automate Maintenance with a System Update Script

    Look into unattended upgrades. Also… it’s not automation if you have to run a script manually :)

    • MangoCat@lemmy.sdf.org
      link
      fedilink
      arrow-up
      3
      ·
      14 hours ago

      if an hostile actor gets a shell on your server, you are fucked anyway.

      I keep telling EVERYONE this, and we continue to get “findings” around privilege escalation.

    • TrollAccount69@lemmy.ml
      link
      fedilink
      arrow-up
      14
      ·
      2 days ago

      Not today, satan.

      Passwordless root allows you to require another factor of authentication on the off chance you didn’t set up key only ssh login like the top of the guide says. That second factor is the administrator account name. Now you can make this as generic as you want but assuming your non root admin account isn’t named “administrator” or “admin” then you just increased the security of your system.

      Suggesting that because privilege escalation exists there is no difference between a user and root is absurd on the face of it. Privesc requires vulnerable versions of vulnerable packages with the specific circumstances present that allow pre and post exploitation actions. In case you were operating under the assumption that most privesc goes from local administrator to root, usually you’re going from some relatively unprivileged user account that handles a service up to root and that’s harder than just invoking sudo.

      You would still want to roll over your credentials and restore from backups but “safe cracking tools exist so just leave the cash register deposits in plain view on a shelf behind the desk” isn’t a strong argument for password root.

      • talkingpumpkin@lemmy.world
        link
        fedilink
        arrow-up
        1
        ·
        10 hours ago

        A “secret” administrator account name is not a second authentication factor: it’s just another “something you know”, basically the same as using a slightly longer password.

        On top of that, account names are generally not treated with the same confidentiality passwords or private keys are treated with: while you can trust that software and collaborators will not leak a password or private key, you cannot make the same assumption about usernames.

        • TrollAccount69@lemmy.ml
          link
          fedilink
          arrow-up
          1
          ·
          13 minutes ago

          Yeah that was a bad choice of words on my part.

          Consider though that everything I handle with a public facing ip gets 69 million attempts to login as “root”, “admin”, “docker” and everything else under the sun but not even once has someone tried to login as my admin user. It’s acting as more than a slightly longer password.

      • MangoCat@lemmy.sdf.org
        link
        fedilink
        arrow-up
        1
        ·
        14 hours ago

        It’s nice that your system has layers, but when the cutworms get on your onion they can still spoil the whole thing.

    • TeaWithDani@lemmy.world
      link
      fedilink
      arrow-up
      8
      ·
      2 days ago

      Exactly, you shouldnt add your user to Docker and you should run your Docker containers as your user or another limited user rather than the default root.

      Be very selective with whether you expose your docker socket to any container. I just give Beszel Read Only. Gives me container perfs and flags updates.

      For the update script, you can just make it a cron job. Id add that to the guide. A daily update and weekly backup cron job is my setup.

      • nixos_fan@lemmy.world
        link
        fedilink
        arrow-up
        1
        ·
        36 minutes ago

        Use podman instead, it doesn’t need root. And putting the docker socket in ro mode doesn’t help at all, because ro is for files, while this here is a connection.

    • iByteABit@lemmy.ml
      link
      fedilink
      arrow-up
      14
      arrow-down
      1
      ·
      2 days ago

      Of course privilege escalation exists, but why roll out the red carpet on an attacker that managed to get shell access? Maybe in the time that they take to figure out that privilege escalation you have the time to detect the attack and defend against them

      • MangoCat@lemmy.sdf.org
        link
        fedilink
        arrow-up
        1
        ·
        14 hours ago

        Most shell users have enough privilege to read (exfiltrate) most data-value from a server, and they almost always have the capacity to damage it beyond functional operation.

      • underisk@lemmy.ml
        link
        fedilink
        arrow-up
        9
        ·
        2 days ago

        If someone put a script on your server that tried to do privilege escalation you should just assume it was successful and act accordingly. It doesn’t really change what should happen.

        • eldavi@lemmy.ml
          link
          fedilink
          English
          arrow-up
          3
          ·
          2 days ago

          replacing or reimaging it automatically when you see something like this feels like overkill.

          • underisk@lemmy.ml
            link
            fedilink
            arrow-up
            4
            ·
            2 days ago

            Only if you haven’t built the infrastructure to make that trivial, which you should have. If you have any data worth securing it’s not even close to overkill.

      • MangoCat@lemmy.sdf.org
        link
        fedilink
        arrow-up
        2
        ·
        14 hours ago

        sudo is a great tool, for people who play nicely.

        It’s like a lock on a glass door, with a rock garden outside.

  • TeaWithDani@lemmy.world
    link
    fedilink
    arrow-up
    14
    ·
    2 days ago

    I recommend self hosting something like a NTFY server to send you texts if certain events happen, like someone logging in.

  • UnknownUniverse@lemmy.mlOP
    link
    fedilink
    arrow-up
    6
    ·
    2 days ago

    Editor’s Note & Update

    Following some comments on this guide, a couple of key clarifications are worth adding:

    Docker Permissions & Rootless Containers: A few eagle-eyed readers rightly pointed out that adding your user to the  docker  group is functionally equivalent to giving yourself root-level access to the host machine (since container runtimes run with root privileges).

    I have updated to remove that shortcut. While container security can go much deeper (such as setting up rootless Docker or Podman), this guide focuses on keeping things simple as a solid quick-start baseline for security.

    System Updates, while fully automated background upgrades have their place, many admins including myself prefer a hands-on approach. Running a script manually and using  needrestart  gives you deliberate control over when services are recycled and ensures you aren’t caught off guard by unexpected reboots.

    Thanks you all for reading and I always welcome any feedback on my work.

    • MangoCat@lemmy.sdf.org
      link
      fedilink
      arrow-up
      3
      ·
      14 hours ago

      I’m going to take exception right here:

      “Install Essential Administrative Tools”

      The first thing you do to lock down a server is: Uninstall everything that is non-essential.

      Do you need any wireless access to the server? If not, remove all WiFi, Bluetooth, and other wireless support from the kernel (block the modules).

      Do you need anyone to be able to plug in a USB dongle that can bridge to ethernet? Or anything else?

      Remove support for everything you don’t need first, un-necessary attack surface is the first foot-gun.

      Are these administrative tools all essential? Keep / install what you need, sudo apt purge the rest.

      Do you know WTF you’re doing with Docker containers and firewalls / iptables? The thing to do is learn how that really works, before getting your application servers / containers up on the open internet.

      Do you know what un-necessary attack surfaces exist within your container images? That’s another step toward a hardened system - know what you’re running, know what that means, and

      above all else: run as little as possible.

  • CallMeAl (like Alan)@piefed.world
    link
    fedilink
    English
    arrow-up
    35
    ·
    2 days ago

    sudo apt update && sudo apt upgrade -y
    should be
    sudo apt update && sudo apt full-upgrade -y

    also for

    echo “PASTE_YOUR_PUBLIC_KEY_HERE” | ssh your_username@your_server_ip “mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys”

    why not just use ssh-copy-id ?

    edit: added more thoughts

    • UnknownUniverse@lemmy.mlOP
      link
      fedilink
      arrow-up
      14
      ·
      2 days ago

      You’re completely right on both counts, full-upgrade handles dependency shifts properly without leaving packages held back, and ssh-copy-id is much cleaner. Updated the guide to reflect both.

    • blueworld@piefed.world
      link
      fedilink
      English
      arrow-up
      3
      ·
      2 days ago

      I love ssh-copy-id, but end up doing it manually about half the time anyway. I’ve tried integrating it into build scripts and find what the op did more reliable.

      Could just be me though… it is Linux, and there a dozen ways to skin Linus. We all have our assholes and opinions.

      • CallMeAl (like Alan)@piefed.world
        link
        fedilink
        English
        arrow-up
        3
        ·
        2 days ago

        I’ve tried integrating it into build scripts In a script I would also write out all the commands I want to run. After all, ssh-copy-id is a shell script itself. For instructions for someone to run interactively, I think ssh-copy-id makes the most sense.

  • LewdLemon@sh.itjust.works
    link
    fedilink
    arrow-up
    16
    ·
    2 days ago

    More Docker hardening*

    Consider rootless docker or Podman. If that is not for you:

    * Note that these tips are less “fire-and-forget, sane defaults” and can be much more finicky. Whether they work well depends on the service in question. They may break container deployments and are often a matter of trial and error until you get them just right. (Feel free to hit me up on details; if I happen to be running the same services as you are, I may be able to tell you what works and what doesn’t.)

  • bacon_pdp@lemmy.world
    link
    fedilink
    arrow-up
    4
    arrow-down
    1
    ·
    2 days ago

    You missed the first step: port knocking.

    If there is no way to connect without having a cert or port sequence; then only the real threats show up in your logs.

  • blueworld@piefed.world
    link
    fedilink
    English
    arrow-up
    5
    ·
    2 days ago

    Good guide as a base. Few thought…

    I’d use unattended-upgrades instead of the script. A long know mehod and does similar, and the script doesn’t reboot for a kernel CVE. I’d pair it alongside out-of-band console and an uptime check.

    Fail2ban needs rsyslog enabled (or proper logging) or it silently reads nothing.

    And lastly Journald needs retention set, or you can loose logs in reboots.

    Also I’m not sure if ‘core’ is enough anymore for a public operating and accessed server, but it’s absolutely vital as a start.