I’ve been using gocryptfs now for a few years and it works fine as you describe.
You initiate the encrypted folder, set up automatic backups for it. Then whenever you want to access it you mount it into another folder.
There is a distinction here between the permanently encrypted folder that you can upload backup whatever, and your temporary mount, unencrypted folder.
If you’re alright with the rare conflicts to fix yourself something like syncthing works well for this setup even across computers.
You can probably encase a Raspberry PI with a battery and a touch screen, micro SD cards can go much higher than 16, and install Linux. Keep in mind that the Linux touch UIs aren’t really great imo, the best experience I’ve had so far is the steam deck.