#PrivateBin 1.5.2 released - S3 storage improvements & library updates

We are still looking for additional long term maintainers. What can we offer you in return for your help?

• We can offer you our mentorship, if this is your first time participating as a maintainer of an open source software project. We can guide you through submitting your first pull requests and work with you to ensure your change fulfills the communities quality standards, gets merged and makes it into a release.
• Your work gets publicly credited. This can help you build up a resume, showing off your growing skill set, in programming as well as your soft skills.
• PrivateBin is a smaller project. If you'd like to learn how to participate and contribute in an open source git project, this should be less overwhelming then larger projects.
• We do have a decent unit test code coverage, so it is an environment forgiving of mistakes. You may still introduce logical flaws or issues in new features, not yet covered in the tests, but you can rely on the tests preventing any regressions in other areas.
• You don't have to be proficient in multiple programming languages, there are a lot of things to improve within either the JavaScript or PHP areas that don't need you to understand the other side, beyond their shared API.
• It can be an opportunity to learn about continuos integration tools to automate tasks like tests, security scans, etc.

If you are interested in helping with any of these points, we have prepared a development guide including design goals, code structure and tools to get you started. For any questions, you can chat with the maintainers in the discussion area or reach us via email or here in the Fediverse.

At work we use more and more YAML documents (with all it's benefits and issues) so I've long started using a handy python alias to convert them into JSON to query them using jq:

alias yaml2json='python3 -c '\''import sys, yaml, json; y=yaml.safe_load(sys.stdin.read()); print(json.dumps(y))'\'''

Because we also started using it in the test harness to validate YAML documents, we made it available to that environment as well. Test creates some YAML report, which is piped into yaml2json, which is piped into jq --exit-status '[...tests here...]'.

Fast forward a few months, more tests got added, and eventually I notice that one of these, on a 30k line document, spends about 15 seconds just converting it. Hm.

Now, it's not like there are not already tons of C, C++, Go, Rust, etc. versions of that tool around, I'm aware of many of those, unfortunately none are prepackaged for the distro used around here. Still, it nags me. I know that in Rust we have all these convenient Serde libraries. Would be trivial to write it using these. People have done so. Hm. And we could make a little wrapper around jq, like yq (python), which makes it easier to use. And we could also support other formats, like XML or CSV, like yq (Go) - the latter also replicates jq itself, that seems a step too far and risks running into incompatibilities, jq itself does the job alright.

So, eventually I did put these puzzle pieces together and we have yet-another-format-to-JSON-converter: github.com/simonrupf/convert2j…

Unique selling points? Well I've experimented a bit with github actions and it is indeed not to difficult to automate the release and publication of statically linked RPMs (for use on different RPM-based distros) and dynamically linked binaries for Ubuntu, MacOS (universal binaries for x86_64 & aarch64) and even Windows (untested - I have no access to that OS, nor do I want to). I'll try to keep it light weight and have some automation set up to keep it updated when dependencies change.

And yes, that 30k YAML file now parses in 0.2s instead of 13s. OCD satisfied.

For over 15 years, most PHP setups I maintained were based on a stack of nginx webserver and PHP-fpm. So, unsurprisingly, the container images I maintain for PrivateBin also use this setup. A while ago, I stumbled on the nginx unit project, which describes itself as:

Unit is a lightweight and versatile application runtime [and] was created by nginx team members from scratch [...].

There is a package for it and it's PHP 8.1 module in the stable Alpine Linux repositories (and soon also for PHP 8.2, when Alpine 3.18 releases). It's installed size is about half of the regular nginx package and because it is a single service one no longer needs a service manager in the container, to keep the two services alive.

Two weeks ago I started toying with it and last weekend I finalized the work. It lets you configure all deviations from the default php.ini via it's own service configuration file, so I now got all the configurations in a single file. It took a bit of work to replicate most former settings over to it. Nginx unit doesn't (yet) allow to set custom HTTP headers, which we use to improve security on the static resources and to prevent cloudflare users getting errors (cloudflare will by default try to optimize all Javascript files for performance, but that breaks the SRI-hashes of our application, so the app doesn't load). It also doesn't allow configuring inline gzip compression of text-based resources.

For now I've documented these limitations in the forked repository and have started testing it first on a personal PrivateBin instance and since that worked well, now switched the main demo instance over to it. The HTTP port and all volumes still attach at the same places, the same UID & GID are used and both logs and included maintenance scripts work the same as before, so it is as close to a drop-in replacement as possible.

So far I've observed slightly higher RAM usage (223 MiB vs 190 MiB used, for a VM that also runs the nginx webserver, for TLS termination, serves the static project webpage and runs the PrivateBin directory application server, all in docker containers) and no observable difference in load (still idles at less than 1%). The images are all about 1 MiB less in compressed size (as displayed on the docker hub in the tags).

I'll probably start migrating my other PHP-fpm containers over to unit, but I'll probably wait for that Alpine 3.18 release, so I can switch to PHP 8.2 at the same time. It's a much simpler stack to maintain.