glizzyguzzler

The official docs have everything, though not as well ordered and concise as they could be.

I’ve taken my server from 11 to 12 to 13. Only issue was 11->12 where backports caused some issue that I no longer remember the specifics of (never broke SSH link), so I just don’t fuck with backports anymore. You need to have your shit together for backports, from what I’ve gathered.

12->13 was abs smooth as butter. Extra 3rd party repos weren’t even an issue. One I had to leave “bookworm” because they didn’t have trixie entries; I found it had no trixie entry by looking at the repo itself - it just kept working no problem after the upgrade anyway.
And I have a “stretch” repo I took from 12 to 13 on another server (it’s mongoDB 4.4 for unifi, an abs shitshow of software those bozos cook up)

Anyway tldr docs are good narry a problem will occur.

Feel free to ask any Qs if any come up, there is ?likely? better value than the Wyze 5070 extended now a days, I looked and bozos still want like $100 for it. Getting AVX2 would be a solid W for software in the future and worth a minor increase in cost. An ARM board (as long as it has UEFI and can boot regular Debian - not a custom distro for the board) would be a solid move too, but you’ll not get Opnsense then - it’s openwrt only for ARM.

This of course is me thinking of pricing when it wasn’t AI bro “lemme order all your RAM on an IOU” stupid, good luck there!! Just opnsense does not need much RAM - like 3 GB is easily safe (mine is given 2GB right now; uses 600 MB right now but I’ve seen it go past 1.5 GB sometimes). Straight Debian will add a gig or so. So even 4 GB would be fine. Proxmox is needy so it’s not fine there, but cutting it out of the equation would make 4 work if you want to save. And you’d prob get away with a $5 16 GB optane M.2 drive from China (aliexpress) with just opnsense - it’s using 3.2 GB of the 16 I’ve given it. And the Optane drive will never wear out as a bonus.

Edit: I’d estimate stock Debian at ~6 GB used. I think both can fit in 16 GB, even with Debian loaded with Incus to run the Opnsense in a VM.

Got a “Dell Wyze 5070 extended”, the extended is important because it lets it have a half height PCIe card. Put a dual NIC Intel card in it. It came with 8 GB of RAM and a 64 GB SATA M.2 drive, plenty. Price out a power brick if it doesn’t come with one, which is a Dell laptop one; it refuses to turn on with a PCIe card in and if the power brick is less than 130W (DA130PE1-00 is a product # of it)!! Very annoying. Note the Intel atom processor in it lacks AVX2 instructions which makes mongoDB past v4 not work on it, so you can’t run the Unifi controller, which has vexed me. But it’s fine for internet routing! Never tested throughput but others have online.

(Note: I would not do this now!) I put proxmox on it then did opnsense (passing through the PCIe NICs) and Debian. I chose opnsense over openwrt because opnsense had an automatic update functionality, while openwrt required you to manually wipe the OS and reinstall - when they fix nasty upgrade path that I’ll come knockin. (Openwrt is like that because that’s how routers do upgrades)

I would not choose proxmox now because its upgrade path is fraught since the free level is rolling and they’ve had critical breaking update errors in the past. No good for your router setup. I want upgrades on and automatic so I don’t get kiddie script hacked.

I’m prepping to do either opnsense only and not in a VM or Debian with Incus (VM manager like promox, but not its own OS) installed and then opnsense in a VM. I’m leaning opnsense direct to just make sure my router works, but I’ve never had trouble with Debian yet so still pondering.

I run a pihole and wireguard on the current Debian VM, and opnsense can do wireguard so most of the way there. In the future I’ll switch that pihole instance (wherever it is) to adguardhome, the local rewrite rules in pihole don’t support wildcards so it’s annoying af to have to load in every local domain I set up.

This has worked well for years and I’ve never measured power but I presume it is low.

Hope this infodump helps!

I have an admin account on the server and an SSO admin login for the services. Different credentials obviously - ones an SSH key and ones a password/passkey for SSO - but it seems to fit well. My regular account is bozo level and I don’t need to worry much, I just logout or go to a private window to do admin for an specific program, which is rare but easy enough when it’s needed.

So far for the SSO services with an admin account design, I just set it up so the admin account for the service is named the SSO admin account so it maps directly when I connect the SSO to it.

And I SSH into the server for admin there as needed.

And no mixing with my regular user account!

The best way to run Podman is root with UserNS to dole out UID/GID protection. Running Podman as root allows you to share networks between containers while having the containers run under different users. If you go rootless, you’d need to run under one user to share the user’s network space with all the containers you want.

As for your issue, I can’t really divine what the problem is from the errors. I avoid nginx because it’s coded to not play well with user abstraction and changing the user with the files it wants to write to etc. Gotta write into a ton of random folders! So not sure exactly what is up. But with Podman root it is easy to run as root 0 internally and make nginx think it has all the control it could ever want.

Try this setup (it is in Podman Quadlet format, apologies I don’t know the compose versions). It runs the container as root 0 internally, externally it runs as some random UID/GID - secure! It uses Volume idmap to map the internal root 0 user to 1001 for write access to the Volume.

Note that in Debian 13 symlinks are broken and won’t work with idmap, just point to the original source. If you need symlinks, I have an alternate UserNS that maps internal user root 0 to external user 1001 directly. You’d drop the idmap in Volume then and use that. You lose some extra security - now the container is running as external user 1001 instead of some random UID/GID - but that’s a pretty minor hit as long as your external user doesn’t have access to tons of things.

# Volumes to mount -> the @ is essential for saying "1001 is absolute and external" basically. 0 is internal. size of 1. You can map 1001 to 0 and 1002 to 1 with @1001-0-2, etc., etc., etc.  
Volume=/mnt/something:/etc/nginx/wants/to/write/here:rw,noexec,nosuid,nodev,Z,idmap=uids=@1001-0-1;gids=@1001-0-1  

# Run as user running the container  
UserNS=auto  
# [use this if req symlink b/c idmap does NOT work with symlinks] -> I tested and it is fixed in at least Podman v5.8.3, so Debian 14 will work with idmap and symlinks directly ! drop the idmap if using !  
# UserNS=auto:uidmapping=0:@1001:1,gidmapping=0:@1001:1  

# Security time  
NoNewPrivileges=true  
# https://man7.org/linux/man-pages/man7/capabilities.7.html  
DropCapability=all  
ReadOnly=true  
ReadOnlyTmpfs=True  
# These capabilities are needed for linuxserver's s6 "launcher" thing  
#AddCapability=CAP_CHOWN  
#AddCapability=CAP_DAC_OVERRIDE  
#AddCapability=CAP_FOWNER  
#AddCapability=CAP_SETGID  
#AddCapability=CAP_SETUID  
# Needs this if the container tries to bind below port 1024. I'm not sure if it is needed if it only tries to bind internally.  
#AddCapability=CAP_NET_BIND_SERVICE  

# TempFS for ReadOnly fixes I've used for nginx - may not be relevant for you. These are from getting Frigate running.  
PodmanArgs=--tmpfs /usr/local/nginx/conf:size=1M,rw,noexec,nosuid,nodev  
PodmanArgs=--tmpfs /usr/local/nginx/logs:size=40M,rw,noexec,nosuid,nodev  
PodmanArgs=--tmpfs /usr/local/nginx/client_body_temp:size=1M,rw,noexec,nosuid,nodev  
PodmanArgs=--tmpfs /usr/local/nginx/proxy_temp:size=1M,rw,noexec,nosuid,nodev  
PodmanArgs=--tmpfs /usr/local/nginx/fastcgi_temp:size=1M,rw,noexec,nosuid,nodev  
PodmanArgs=--tmpfs /usr/local/nginx/uwsgi_temp:size=1M,rw,noexec,nosuid,nodev  
PodmanArgs=--tmpfs /usr/local/nginx/scgi_temp:size=1M,rw,noexec,nosuid,nodev  
PodmanArgs=--tmpfs /etc/letsencrypt:size=1M,rw,noexec,nosuid,nodev  

Root Podman and UserNS=auto needs a containers user to pull uid/gid from.

# Root Podman needs a `containers` "user" (not really a user, just a reserved uid/gid space)   [fixt!]  
#~~sudo echo "containers:2147483647:2147483648" >> /etc/subuid~~  
#~~sudo echo "containers:2147483647:2147483648" >> /etc/subgid~~  
echo "containers:2147483647:2147483648" | sudo tee -a /etc/subuid > /dev/null  
echo "containers:2147483647:2147483648" | sudo tee -a /etc/subgid > /dev/null  

The documentation for Podman is critically lacking in the “hobbyist” space. Hope this helps.

Edit: This approach works well because most Docker containers are built assuming they’ll run as root 0. That’s why Linuxserver uses the S6 overlay thing to jump from root 0 to something else. The container can be built to run as any user though, if you look at the Dockerfile for the container you’re using, you’ll see what user they’re declaring it will run as (and likely what user owns all the files). Root 0 usually gets around that problem - unless they “cleverly” code it to try to prevent you from running the container as root 0 (I’ve run into this before! It was Heimdall from the Linuxserver people).

Edit2: I’ve noticed you said no Volumes, so drop that. But you can still use the UserNS mapping to run it as root internally which should fix the internal permissions issues. The tmpfs stuff is if you declare ReadOnly for extra security - it’s a great idea - but nginx is extra difficult in that regard. Disregard it while you get going.

# Run as user running the container  
UserNS=auto  

# Security time  
NoNewPrivileges=true  
# https://man7.org/linux/man-pages/man7/capabilities.7.html  
DropCapability=all  
# These capabilities are needed for linuxserver's s6 "launcher" thing  
#AddCapability=CAP_CHOWN  
#AddCapability=CAP_DAC_OVERRIDE  
#AddCapability=CAP_FOWNER  
#AddCapability=CAP_SETGID  
#AddCapability=CAP_SETUID  
# Needs this if the container tries to bind below port 1024. I'm not sure if it is needed if it only tries to bind internally.  
#AddCapability=CAP_NET_BIND_SERVICE  

Edit3:
Use sudo podman top nginx user huser group hgroup groups hgroups to see the internal user/host user (huser) mappings easily for debug.

USER	HUSER		GROUP	HGROUP		GROUPS	HGROUPS  
root		2147485695	root		2147485695	105		105  

Here’s an output from my frigate container. Internally (USER) it is root, externally (HUSER) it’s some random UID. I’ve also mapped the internal group (GROUPS) 105 to the external group (HGROUPS) 105 so that it has render access.