Another option that’s available is hosting your own Kiwix instance and downloading the Arch Wiki .zim file.
I have a few other .zim’s from the Kiwix library including Alpine Wiki, Stack Overflow, Man pages and a full copy of Wikipedia. There’s a lot available at that Kiwix library which can make for a good offline digital library.
I have two systems that sort of work together.
The first system involves a bunch of text files for each task. OS installation, basic post OS installation tasks and a file for each program I add (like UFW, apparmor, ddclient, docker and so on). They basically look like scripts with comments. If I want to I can just copy/paste everything into a terminal and reach a a specific state that I want to be at.
The second system is a sort of “skeleton” file tree that only contains all the files that I have added or modified.
.
├── etc
│ ├── crontabs
│ │ └── root
│ ├── ddclient
│ │ └── ddclient.conf
│ ├── doas.d
│ │ └── doas.conf
│ ├── fail2ban
│ │ ├── filter.d
│ │ │ └── alpine-sshd-key.conf
│ │ └── jail.d
│ │ └── alpine-ssh.conf
│ ├── modprobe.d
│ │ ├── backlist-extra.conf
│ │ └── disable-filesystems.conf
│ ├── network
│ │ └── interfaces
│ ├── periodic
│ │ └── 1min
│ │ └── dynamic-motd
│ ├── profile.d
│ │ └── profile.sh
│ ├── ssh
│ │ └── sshd_config
│ ├── wpa_supplicant
│ │ └── wpa_supplicant.conf
│ ├── fstab
│ ├── nanorc
│ ├── profile
│ └── sysctl.conf
├── home
│ └── pi-user
│ ├── .config
│ │ └── ash
│ │ ├── ashrc
│ │ └── profile
│ ├── .ssh
│ │ └── authorized_keys
│ ├── .sync
│ │ ├── file-system-backup
│ │ │ ├── .sync-server-fs_01_root
│ │ │ └── .sync-server-fs_02_boot
│ │ └── .sync-caddy_certs_backup
│ ├── .nanorc
│ └── .tmux.conf
├── root
│ ├── .config
│ │ └── mc
│ │ └── ini
│ ├── .local
│ │ └── share
│ │ └── mc
│ │ └── history -> /dev/null
│ ├── .ssh
│ │ └── authorized_keys
│ ├── scripts
│ │ ├── automated-backup
│ │ └── maintenance
│ ├── .ash_history -> /dev/null
│ └── .nanorc
├── srv
│ ├── caddy
│ │ ├── Caddyfile
│ │ ├── Dockerfile
│ │ └── docker-compose.yml
│ └── kiwix
│ └── docker-compose.yml
└── usr
└── sbin
├── containers-down
├── containers-up
├── emountman
├── fs-backup-quick
└── rtransfer
This is useful to me because I can keep track of every change I make. I even have it set up so I can use rsync to quickly chuck all the files into place after a fresh install or after adding/modifying files.
I also created and maintain a “quick install” guide so I can install a fresh OS, rsync all the modified files from my skeleton file tree into place, then run through all the commands in my quick install guide to get myself back to the same state in a minimal amount of time.
I do want to write up a guide about how to setup Caddy + DeSec.io but I don’t have the time at the moment. If you have any questions, feel free to ask. I can try to help where I can.
I’ll leave you this previous post I made, you might find some additional information in there if you get stuck. https://lemmy.dbzer0.com/post/51117983
Also, someone suggested using a wildcard cert for the use of any sub-domain names. I chose to learn and use that because it helps obscure my services. If you have any interest in security, it might interest you. It terms of security, it’s not the absolute way to protect yourself, but I think it helps when combined with other security measures. If you read the comments in the post, you should get some more insight about it.
True.
My self-hosting strategy is wildly alternative and not one I speak much about publicly. I’m the only person connecting to my own domain so as long as I continue to practice shutting the fuck up, I can get away with using multiple layers of obscurity rather than fiddling with third party solutions.
I check my logs daily and the only activity I ever see is my own. Since I am not hosting anything critical or sensitive, I have the opportunity to experiment this way without much risk to myself.
The way I’m set up, I am not concerned with DDOS attacks because it would fail to get past the Dynamic DNS. If I were hosting a social media platform or something more public, then I would need to take stronger measures to protect myself and that data.
Even though I don’t host anything important, I’m still glad I found alternative ways to hosting my own stuff without the use of any of Cloudflare services.
I’ve noticed over time that the self-hosted communities have been suggesting Cloudflare Tunnels less and less since Trump and his gang took over America. Maybe this latest outage will push more people to not recommend Cloudflare again in the future.
I still remember when I first got into self-hosting and being mocked pretty hard for questioning the use of such a large centralized service like Cloudflare. I’m glad I persisted and kept learning in my own direction but that still was very demotivating at the time.
I actually started with RPi’s. The first one, a used Pi 4b, is dedicated only to HomeAssistant. I don’t tinker with it anymore because it does what I want and I don’t want unexpected downtime when I have to use the bathroom or use the lights in my room.
I bought a used Pi5 with the intention of upgrading later. In life I am quite minimal and find a joy in using what little tools and material I have to create something new. That seems to hold true to technology and scripting too. The RPi5 with an old USB3 HDD is actually way more power than I can currently use and can imagine using for a long time. The extra room to work is convenient though.
I’ll have a look into some of the places you suggested, those seem like the places to draw good inspiration from, thank you.
I started out rewriting my network backup scripts only to realize I was adding functionality to a previous script I wrote to automatically mount and dismount luks encrypted volumes. I still want to type in my luks passphrase because I don’t want everything automated and prefer to include inconvenience as an additonal security measure in securing some of my data.
I also came to the realization recently that the reason I don’t relate strongly to other self hosters is because I’ve unknowingly been trying to create a minimal self hosted system that is more beneficial to small, low powered devices.
I’ve been using Alpine Linux, I install only the bare, older but well established tools and have been creating scripts soley based off those tools instead of seeking out bigger, more complicated modern tools. For example creating workflows by only using rsync or using https://github.com/RayCC51/BashWrite to create a blog that only uses bash and GNU sed to create a static blog site.
At least now that I’m aware of this, I can keep an eye out for such projects or communities and would hopefully be able to contribute something in that direction.
I’ve experienced gatekeeping issues long before I got into self-hosting specifically. Years ago I wanted to learn C++ for Arduino and I was constantly talked down for asking questions.
“Why don’t you just do …” in response to a question feels very rude as a newcomer because it feels like I am being talked down to for not knowing what others already know. Even when I made an effort to show I was making an effort to learn on my own, I was still belittled.
I’m all for hearing different ways of approaching my issue but from the replies, it often feels like other people insist there is only one true specific way to handle an issue.
When I first got into self-hosting, people kept pushing Cloudflare on me. When I expressed concern over a large centralized corporation having that much control and how they might have service issues, I was mocked really hard. Half a year later and there was a significant outage and suddenly there’s all this talk about how centralized the internet is and how that is bad.
After that I took it upon myself to find alternative ways to protect myself without Cloudflare’s services but every step of the way has been an isolating experience. Every step of the way has been full of people saying that my efforts are pointless and that the bots will win anyways so I shouldn’t bother.
I decided to try to secure myself through multiple layers of obscurity and every question in that direction has been full of people saying that obscurity is not security, the bots will find you anyways!
I’ve stopped myself from asking too many questions now. I still keep learning in my direction. I feel like I’ve managed to find multiple solutions that both obscure and protect myself. I’ve constantly check my logs for months now and the bot is less than I expected in places I expect them to be and completely zero in other places I thought there would be some activity.
I want to share what I have learned and my experiences but I know I will receive backlash for deviating from the norm.
I’ve spent a lot of my self-hosting efforts trying to find ways to protect myself with minimal use of third party services, documenting as much as I could only feel afraid to share what I have learned.
This comment may not be about learning self-hosting as a beginner specifically but the vibe has been pretty damn consistent throughout me learning C++, self-hosting, linux and shell scripting. All things I enjoy but all so full of people ready to talk down to someone who wants to learn.
I have three backups. One is my laptop where all the backups initially start. Then that gets copied to a plugin USB SSD. Then another copy goes to my server which has another USB SSD. That means I don’t have an off site backup.
I don’t have a place to host an off site backup and I’m not comfortable or interested in using cloud services. Instead I just decided that if it all goes up in flames. So be it.
It’s just data and backups are just nice a convenience. I’ll be upset but there’s more important things in life to worry about.
I’ve always lived a life of minimalism and to me stuff is stuff. None of it mattered before I was born and none of it will matter after I die. That happiest and most free feeling I ever experienced was when I spent years travelling with only a 34 litre backpack and that’s kind of been my baseline for happiness ever since.
apk seems to have some tricks in there that aren’t as well known.
I managed to catch in the IRC channel that apk add doc will automatically download any related man pages for packages with any future downloads through apk. That made life a bit more convenient instead of downloading all those packages separately.
My web facing server has just enough packages installed to (kinda securely) host a Caddy and Kiwix docker container to work with my domain name and make a comfortable work environment through SSH. My Pi for my HomeAssistant docker container has less because it’s locked down to just my local network.
I also wrote my own install scripts so reinstalling everything and getting it back to a running state would take about 15 minutes for each device.
And I also wrote my own backup/restore scripts that evolved over 3/4 of a year. I use them often so I have confidence in those scripts.
I personally don’t really care too much. I have multiple ways of dealing with issues for something that’s a hobby to me. Which is why I stick to simplicity.
I’m sure this is a thing for people to worry about when dealing with more complex setups. I just wanna vibe out in my tiny corner of the internet.
On Alpine Linux I update my two Pi servers at 2 in the morning daily. It’s simpler compared to Debian which needs unattended-updates. Just add apk update && apk upgrade to a cron job and you’re good to go.
I only have three docker services which is simple enough to update manually.
I like to keep things as simple as possible for my already chaotic brain.
I’ve read about that and I already have that in my notes as well.
It doesn’t really affect my needs because my ISP blocks incoming on those ports anyways. Also I’m choosing not to use a tunnel at the moment so I’ll be using a higher port anyways.
The last time I asked about it, a few people seemed to agree it was something to do with the firewall settings. That seems most likely since I was able to connect when I disabled my firewall. I’m not a fan of working with iptables. The language for that type of networking is gibberish to me.
I had also tried going from docker compose to rootful podman compose and ran into the same issue. Although I’m trying to work away from podman compose in the future, just taking it in steps.
Yeah, I mainly just want to move away to more open projects. When I first started, everyone kept suggesting using Cloudflare. After half a year using their service, I just felt icky the entire time.
In the past couple months I was able to move away and chose to protect myself by learning how to harden my server as well as hiding my server behind multiple layers of obscurity.
With my current setup, the only site traffic I get has only been myself and my custom ssh port only gets hit by bots about 3-10 times a week according to my logs. Only time will tell how effective my layers of obscurity will hold up but so far it seems to satisfy my needs better than I was expecting.
Once I get podman in a state I like, I’ll pretty much be all open sourced and all I’ll have to do for myself is be in maintenance mode unless I care to add a new service. I like to keep things simple so I don’t normally go crazy adding new services anyways.
Thank you for the offer. I still need a bit more more time to experiment and zero in on the issue again. Fortunately my setup is quite simple and the only bottleneck will be Caddy.
I basically run Caddy which redirects to a static generated blog, simple file server page and a Kiwix instance. I’m mostly making a self hosted reference site of materials for Linux and Scripting resources.
One day I may add a Forgeo instance but currently my entire workflow exists around rsync. I’m happy just having my single file scripts hosted as text files and don’t really need the power of git. At least not at the moment.
I’ve been making another attempt to replace Docker with Podman. The issue is I can’t connect to my server through a web browser. I think it’s a firewall issue.
Networking and networking troubleshooting is a bit confusing for me and that’s the least favourite part about self hosting for me. Turns out I actually enjoy writing scripts more and the challenge of writing POSIX scripts especially.
If I can figure it out, I’ll probably write a guide for setting up Podman and Caddy on Alpine Linux since there isn’t a lot of recent information out there from what I found in my searches so far.
I use rsync for many of the reasons covered in the video. It’s widely available and has a long history. To me that feels important because it’s had time to become stable and reliable. Using Linux is a hobby for me so my needs are quite low. It’s nice to have a tool that just works.
I use it for all my backups and moving my backups to off network locations as well as file/folder transfers on my own network.
I even made my own tool (https://codeberg.org/taters/rTransfer) to simplify all my rsync commands into readable files because rsync commands can get quite long and overwhelming. It’s especially useful chaining multiple rsync commands together to run under a single command.
I’ve tried other backup and syncing programs and I’ve had bad experiences with all of them. Other backup programs have failed to restore my system. Syncing programs constantly stop working and I got tired of always troubleshooting. Rsync when set up properly has given me a lot less headaches.
I don’t have root access on my phone but I still copy backups of my media and apps that export data to accessible files.
I keep my process very simple using Termux with rsync openssh and termux-services packages.
I created a folder dedicated on my for syncing between phone to computer called sync but you can change this for your needs.
From a fresh Termux install, the setup should look something like the following:
# Update package list and packages
pkg update && pkg upgrade
# Install required packages
pkg install rsync openssh termux-services
# Setup Termux's access to your phone's files
termux-setup-storage
# Make the required folder
mkdir ~/storage/shared/sync/
cd ~/storage/shared/sync/
# Automatically start your SSH server when you open Termux
sv-enable sshd
~ $ whoami
u0_a205
passwd (I can’t remember if this step is important)A quick note:
Termux on android has a file system quite different than a computer so file and directory names can get quite long. The pwd command would show /data/data/com.termux/files/home/storage/shared/sync for my sync folder.
This can be made simpler by using the realpath command. realpath /data/data/com.termux/files/home/storage/shared/sync then shows /storage/emulated/0/sync as a result. If you’re using CLI, this may make your commands easier to read.
Now you can start to build your rsync command to transfer your files.
When setting up an rsync command, ALWAYS use the --dry-run- option. This performs a “transfer” without any files being moved.
rsync --dry-run --archive --verbose --human-readable --partial --progress --compress -e 'ssh -p 8022' u0_a205@192.168.40.210:/storage/emulated/0/sync/ /home/computer_username/backup/
rsync --dry-run --archive --verbose --human-readable --partial --progress --compress /storage/emulated/0/sync/ computer_username@192.168.40.205:/home/computer_username/backup/
Explanation:
--archive preserves several file attributes--verbose --human-readable --partial --progress creates a readable output to see what is happening--compress compresses the data during the actual transfer (good for over a network)-e 'ssh -p 8022' SSH on termux runs on port 8022u0_a205@192.168.40.210:/storage/emulated/0/sync/ and computer_username@192.168.40.205:/home/computer_username/backup/ are how rsync identifies remote folders. Basic format is <username>@<remote IP address>:/path/to/folder//home/computer_username/backup/ and /storage/emulated/0/sync/ are the local folders, relative to what machine the rsync command is being run from.In order to reverse the direction of a command relative to the machine you are running on, simple swap the remote folder and local folder in the command. Example: From only my computer:
# Direction: Phone -> Computer
rsync --dry-run --archive --verbose --human-readable --partial --progress --compress -e 'ssh -p 8022' u0_a205@192.168.40.210:/storage/emulated/0/sync/ /home/computer_username/backup/
# Direction: Computer -> Phone
rsync --dry-run --archive --verbose --human-readable --partial --progress --compress -e 'ssh -p 8022' /home/computer_username/backup/ u0_a205@192.168.40.210:/storage/emulated/0/sync/
In order to actually transfer files, remove the --dry-run option from the previous rsync commands. The output in your terminal will show additional information regarding transfer status.
Additionally, you can also add the --delete option to the rsync command, this will “deduplicate” files, meaning the source folder will force the destination folder match, file by file. That means deleting any file in the destination folder that does not match the source folder list of files.
A command WITHOUT --dry-run and WITH --delete would look like the following (CAUTION: THIS CAN DELETE FILES IF UNTEST):
rsync --delete --archive --verbose --human-readable --partial --progress --compress -e 'ssh -p 8022' u0_a205@192.168.40.210:/storage/emulated/0/sync/ /home/computer_username/backup/
I personally manually transfer my backups into an encrypted external drive which I manually decrypt. /u/emhl@feddit.org has a suggestion for automated encrypted backups if that’s more to your needs.
I do something similar using only
rsync. I’ve had nothing but headaches whenever I used automated file syncing programs. The bare bones aspect of just using rsync has made it a much more consistent experience.I found using automated file syncing programs have too much complexity under the hood that just seems to lead to more time troubleshooting issues.