Through these interviews, we aim to make security easier and more efficient for OSS projects. Anyone over the age of 18 who regularly contributes to at least one OSS project, and has contributed to an OSS project for at least one year, is eligible --- no security background or experience required.
Interview participants will receive a $40.00 Amazon or Tango gift card for completing one 60-75 minute interview via Zoom. We will pr
Through these interviews, we aim to make security easier and more efficient for OSS projects. Anyone over the age of 18 who regularly contributes to at least one OSS project, and has contributed to an OSS project for at least one year, is eligible --- no security background or experience required.
Interview participants will receive a $40.00 Amazon or Tango gift card for completing one 60-75 minute interview via Zoom. We will protect the privacy and confidentiality of any information shared during the interviews to the utmost of our abilities.
If you might be interested, please fill out this survey to answer some initial questions. Contact Carson Powers or Harjot Kaur with any questions about the study, or click here to learn more!
I had the opportunity to participate in an interview last week with Harjut and Carson and wish them success in furthering our collective insights into how we can improve security of our FOSS software projects. It is, in my own expirience, an often thank-less task, but important to do. So any way of making it more popular will help us all in the long run.
Also, as an aside, I look forward meeting some of you at the #37c3 very soon. 😷
I got to follow rocket's journey to async and stable #rust with the PrivateBin directory service. Coming from Python flask apps, it is really easy to pick up and get going with your webservice, offering static & templated content, easy to create web forms and JSON APIs.
Thanks to rust's strict type system I could focus on the logic and didn't have to waste time double checking and casting data received by clients. If my API accepts an integer in a certain parameter, Rocket will ensure I only receive valid requests in my logic.
#PrivateBinv1.6.0 released - adding translations for Japanese & Arabic, configuration option to disable email button and increases the minimal required PHP version to 7.3.
We are still looking for additional long term maintainers. What can we offer you in return for your help?
We can offer you our mentorship, if this is your first time participating as a maintainer of an open source software project. We can guide you through submitting your first pull requests and work with you to ensure your change fulfills the communities quality standards, gets merged and makes it into a release.
Your work gets publicly credited. This can help you build up a resume, showing off your growing skill set, in programming as well as your soft skills.
PrivateBin is a smaller project. If you'd like to learn how to participate and contribute in an open source git project, this should be less overwhelming then larger projects.
We are still looking for additional long term maintainers. What can we offer you in return for your help?
We can offer you our mentorship, if this is your first time participating as a maintainer of an open source software project. We can guide you through submitting your first pull requests and work with you to ensure your change fulfills the communities quality standards, gets merged and makes it into a release.
Your work gets publicly credited. This can help you build up a resume, showing off your growing skill set, in programming as well as your soft skills.
PrivateBin is a smaller project. If you'd like to learn how to participate and contribute in an open source git project, this should be less overwhelming then larger projects.
We do have a decent unit test code coverage, so it is an environment forgiving of mistakes. You may still introduce logical flaws or issues in new features, not yet covered in the tests, but you can rely on the tests preventing any regressions in other areas.
You don't have to be proficient in multiple programming languages, there are a lot of things to improve within either the JavaScript or PHP areas that don't need you to understand the other side, beyond their shared API.
It can be an opportunity to learn about continuos integration tools to automate tasks like tests, security scans, etc.
If you are interested in helping with any of these points, we have prepared a development guide including design goals, code structure and tools to get you started. For any questions, you can chat with the maintainers in the discussion area or reach us via email or here in the Fediverse.
For over 15 years, most PHP setups I maintained were based on a stack of nginx webserver and PHP-fpm. So, unsurprisingly, the container images I maintain for PrivateBin also use this setup. A while ago, I stumbled on the nginx unit project, which describes itself as:
Unit is a lightweight and versatile application runtime [and] was created by nginx team members from scratch [...].
There is a package for it and it's PHP 8.1 module in the stable Alpine Linux repositories (and soon also for PHP 8.2, when Alpine 3.18 releases). It's installed size is about half of the regular nginx package and because it is a single service one no longer needs a service manager in the container, to keep the two services alive.... mehr anzeigen
For over 15 years, most PHP setups I maintained were based on a stack of nginx webserver and PHP-fpm. So, unsurprisingly, the container images I maintain for PrivateBin also use this setup. A while ago, I stumbled on the nginx unit project, which describes itself as:
Unit is a lightweight and versatile application runtime [and] was created by nginx team members from scratch [...].
There is a package for it and it's PHP 8.1 module in the stable Alpine Linux repositories (and soon also for PHP 8.2, when Alpine 3.18 releases). It's installed size is about half of the regular nginx package and because it is a single service one no longer needs a service manager in the container, to keep the two services alive.
Two weeks ago I started toying with it and last weekend I finalized the work. It lets you configure all deviations from the default php.ini via it's own service configuration file, so I now got all the configurations in a single file. It took a bit of work to replicate most former settings over to it. Nginx unit doesn't (yet) allow to set custom HTTP headers, which we use to improve security on the static resources and to prevent cloudflare users getting errors (cloudflare will by default try to optimize all Javascript files for performance, but that breaks the SRI-hashes of our application, so the app doesn't load). It also doesn't allow configuring inline gzip compression of text-based resources.
For now I've documented these limitations in the forked repository and have started testing it first on a personal PrivateBin instance and since that worked well, now switched the main demo instance over to it. The HTTP port and all volumes still attach at the same places, the same UID & GID are used and both logs and included maintenance scripts work the same as before, so it is as close to a drop-in replacement as possible.
So far I've observed slightly higher RAM usage (223 MiB vs 190 MiB used, for a VM that also runs the nginx webserver, for TLS termination, serves the static project webpage and runs the PrivateBin directory application server, all in docker containers) and no observable difference in load (still idles at less than 1%). The images are all about 1 MiB less in compressed size (as displayed on the docker hub in the tags).
I'll probably start migrating myotherPHP-fpm containers over to unit, but I'll probably wait for that Alpine 3.18 release, so I can switch to PHP 8.2 at the same time. It's a much simpler stack to maintain.
This minor release adds support for Simple Storage Service (S3), a storage migration script, adds four new languages to the translations and includes updated libraries.
The application in question is my first foray into the rust language and a very simple, CRUD-style web application. It consists of: - a page with list of entries in a database, sorted by fixed criteria, for now - a single JSON API endpoint to retrieve that same list, with optional filters - a form to add an entry to the list - a nearly static about page - two tasks started via cron jobs, one running every few minutes and the other once a day
The rocket framework appealed to me because it offered a similar semantic to flask for python. It comes with several options for templating and state (aka database) integrations. I had picked tera templates, as they are near identical to jinja2 templates that are used all over python application (i.e. in pelican, ansible, flask, ...) and diesel for accessing an sqlit... mehr anzeigen
The application in question is my first foray into the rust language and a very simple, CRUD-style web application. It consists of: - a page with list of entries in a database, sorted by fixed criteria, for now - a single JSON API endpoint to retrieve that same list, with optional filters - a form to add an entry to the list - a nearly static about page - two tasks started via cron jobs, one running every few minutes and the other once a day
The rocket framework appealed to me because it offered a similar semantic to flask for python. It comes with several options for templating and state (aka database) integrations. I had picked tera templates, as they are near identical to jinja2 templates that are used all over python application (i.e. in pelican, ansible, flask, ...) and diesel for accessing an sqlite database. The rocket framework was already at version 0.4 when I started out with the project in 2020. It used a synchronous version of hyper, a low-level HTTP library to provide a multi-threaded HTTP server. It lets you compile the entire application into a single statically linked binary, excellent to run in a small "FROM scratch" container image.
In June 2021, the rocket 0.5 release candidate got published, and over the quieter days end of 2021 / beginning of 2022, I dug into upgrading the app. Let's start with the best bits about this process and the end result: - I didn't need to do any changes to the templates, the CSS, the database structures or queries. The rocket routing mechanism worked the same way as before. - rocket 0.5 is now compiling on a stable rust compiler, where 0.4 required the use of the nightly compiler (meaning it used some unstable language features). It got rewritten to use asynchronous functions, which allows it to use the latest hyper library which leverages that mechanism on top of a multi-threaded runtime provided by tokio, an implementation of the green threads concept in Rust.
For this particular application, the average latency was reduced from 67ms to 60ms (as monitored by a zabbix proxy running an a host attached to the same subnet) and the daily cronjob duration got reduced from 52s, using multi-threading, to 22s, using async and the tokio runtime (the earliest, fully linear, implementation of that task took 20 minutes).
While I am happy with these clear improvements, they did come at some cost: - In the end I spent about 7 workdays on this upgrade, spread over 2 weekends and several evenings - git tells me that this consisted of 6188 additions and 4231 deletions, including all the license changes. - The switch to async/await in both rocket and hyper required that many formerly synchronous functions had to be made async as well, as they now needed to call async functions themselves. Rust would offer you the block_on construct to call async functions from sync ones, but that conflicts with the tokio runtime and causes it to fail. The same is true for spawning threads, which I had done extensively in parts of the code that run multiple HTTP(S) requests in parallel. Using tokio to let the async calls be handled automatically did pay off and is indeed faster then my manual optimizations, but the rewrite into async required pretty big code changes. - The newer hyper version lost some features I had been relying on, like shorter connection timeouts (I replaced these with a timeout wrapper on the returned future and an ugly workaround to return an error) and IDN domain names (I now use the URL parser from servo, developed for Firefox). - The unit tests also needed to be made async aware and tokio provides a dedicated macro for wrapping tests. Unfortunately I had to give up on one unit test that I just couldn't get to work under a tokio runtime, while the error couldn't be reproduced on the running service, either manually or scripted. It seems the tokio runtime acts differently when in a unit test, where it keeps telling me it can't launch a runtime from within a runtime. The stack traces seem to go remain outside any of my code, so doesn't even seem to enter any part of the unit test itself. - Due to all these new and upgraded libraries the statically linked binary ended up growing (after stripping and upx-ing) from 2.16 MiB to 2.83 MiB. Sure, compared to a full PHP- or Python-stack it is still at least an order of magnitude smaller, but it's also an increase of 1/3.
I would still recommend rocket to anyone familiar with flask. Creating dynamic web pages or APIs is very straight forward to do. The issues I encountered with the switch to async would not apply to new projects anyway. For existing rocket v0.4 applications, the upgrade is worthwhile if you can use the extra performance or aren't doing weird multi-threaded things in the background, like me. None of the core rocket features broke, only the database abstraction required modest changes. My struggle was mostly with the use of hyper as a client.
For the database side, diesel isn't quite as elegant as ORMs in other languages. It makes it simple to retrieve (lists of) elements from a table and do simple joins. For more complex things you can fall back to writing manual SQL queries and mapping the result onto your structs for convenience (you can mark properties that aren't tied to specific table columns, so you can populate them from calculations on joined tables or such). The database migration mechanism also requires the use of plain SQL.
As a brief follow up, I did eventually manage to fix that last unit test. It was a pretty large end-to-end affair, where I used both rocket's Client, to send form data, and also triggered the cron jobs that use their own rocket instances, but only to extract the database connection. This seems to have been the cause of the tokio within tokio runtime issue. I simply split this into two tests, one that used the form and the other to test all the cron job functions.
The binary acts as a multi-call one, when started with the regular rocket environment parameters it will launch the web service. But if the CRON environment variable is detected it will only execute the requested cron mode and exit. This is not a normal use case for a rocket application, so I can only really blame myself for this quirk.
One thing I did also find out is that my test code did have side effects, because most tests use a shared database (sqlite), so I had to switch to running all tests single-threaded (which is slower, takes up to 10s) as otherwise I got random failures when some tests deleted othe... mehr anzeigen
As a brief follow up, I did eventually manage to fix that last unit test. It was a pretty large end-to-end affair, where I used both rocket's Client, to send form data, and also triggered the cron jobs that use their own rocket instances, but only to extract the database connection. This seems to have been the cause of the tokio within tokio runtime issue. I simply split this into two tests, one that used the form and the other to test all the cron job functions.
The binary acts as a multi-call one, when started with the regular rocket environment parameters it will launch the web service. But if the CRON environment variable is detected it will only execute the requested cron mode and exit. This is not a normal use case for a rocket application, so I can only really blame myself for this quirk.
One thing I did also find out is that my test code did have side effects, because most tests use a shared database (sqlite), so I had to switch to running all tests single-threaded (which is slower, takes up to 10s) as otherwise I got random failures when some tests deleted others data or couldn't add new instances when these were already present. This could obviously be fixed by either providing each test with a mock database or just be more careful designing the tests such that their order doesn't matter (i.e. using unique keys to add to the database for each test.
Not entirely serious question: Why does the Gnome Terminal icon represent an IBM PC DOS prompt instead of UNIX shell prompt? Is that what folks think of first, when they think of a command prompt?
Yes, shell parsing is non-obvious - it does help enormously to understand that the shell takes what you type on the command line after you hit enter, parses it, replacing variables, expanding globs (wildcards) and other language constructs in the process and only then issues a system call, passing the resulting argv structure to the kernel for execution.
Erst nach einigem Verdauen kann ich nun genau benennen, was mich eigentlich so besonders an Googles neuem FLoC Tracking stört: Es kehrt das Prinzip um, dass man Herr seiner eigenen Internet Domäne(n) ist.
Ich gebe Geld dafür aus, meine eigenen Server zu housen und für die Registrierung meiner eigenen Internet-Domänen. Bisher konnte ich so frei entscheiden, ob ich da Werbung und Tracking darauf einbauen will oder nicht. Ich hatte mich stets dagegen entschieden. Das war einfach, ich musste somit auch nichts in meinen Content einbauen und pflegen. Und die Seiten sind erst noch schlanker und ich kann sicher sein, dass diese auch genau so ausschauen wie ich es will und nicht von Werbung verunstaltet werden.
FLoC kehrt dieses Prinzip um: Besucher meiner Webseiten werden ge-trackt, wenn Sie Chrome benutzen und zur Gruppe gehören bei denen das "Experiment" zufällig aktiviert wurde, und ich muss nun aktiv einen neuen HTTP header in alle meine Webseiten einbauen, damit die Besucher nicht getrackt werden! Und hoffen, dass Chrome sich auc... mehr anzeigen
Erst nach einigem Verdauen kann ich nun genau benennen, was mich eigentlich so besonders an Googles neuem FLoC Tracking stört: Es kehrt das Prinzip um, dass man Herr seiner eigenen Internet Domäne(n) ist.
Ich gebe Geld dafür aus, meine eigenen Server zu housen und für die Registrierung meiner eigenen Internet-Domänen. Bisher konnte ich so frei entscheiden, ob ich da Werbung und Tracking darauf einbauen will oder nicht. Ich hatte mich stets dagegen entschieden. Das war einfach, ich musste somit auch nichts in meinen Content einbauen und pflegen. Und die Seiten sind erst noch schlanker und ich kann sicher sein, dass diese auch genau so ausschauen wie ich es will und nicht von Werbung verunstaltet werden.
FLoC kehrt dieses Prinzip um: Besucher meiner Webseiten werden ge-trackt, wenn Sie Chrome benutzen und zur Gruppe gehören bei denen das "Experiment" zufällig aktiviert wurde, und ich muss nun aktiv einen neuen HTTP header in alle meine Webseiten einbauen, damit die Besucher nicht getrackt werden! Und hoffen, dass Chrome sich auch daran hält.
Nein danke, ich will die Besucher meiner Webseiten keiner unnötigen Überwachung aussetzen. Schade, dass ich nun Aufwand treiben muss um dies zu erreichen, anstatt einfach keine Tracker oder Werbung einzubauen.
Started looking into gemini space. Love how it feels - it's like the web ca. mid-90s. UI is back under your control and you can focus on reading the content instead of getting the site to work (because either you have issues with noscript turned on and sites requiring JS to display text or you have it disabled and have to click through modal windows informing you of cookie settings, sign up for that newsletter, etc. to get to the content).
I'm using Castor and wanted to merry it to my Gnome desktop, so clicking links in Firefox/Chromium opens them in Castor. Oh, and I got a gopher client for free with it as well. Was bummed when Firefox dropped gopher support. Here's how to register the gemini protocol in Gnome (and build castor):
While grep and sed are commoly used, awk fills a valuable niche when processing structured text, avoiding multiple pipes or more complicated regex extractions. Here is a handy flowchart to pick the ideal tool for your text processing task:
A plaintext chart and a simple example making use of several awk features can be found here:
Small tools to use as (non-sharable) sketchpads - if you need a quick spreadsheet calculation or text editor and are to lazy to open a terminal. Good example of #minimalism and very #privacy minded.
Maybe a concept we could evaluate for use in our fediverse software as well:
This forms a relative reputation system. As uncomfortable as it may be, one man’s terrorist is another man’s freedom fighter, and different jurisdictions have different laws - and it’s not up to the Matrix.org Foundation to play God and adjudicate. Each user/moderator/admin should be free to make up their own mind and decide which reputation feeds to align themselves with.
Falls Du einen hast und das Display noch funktioniert - Quietscht Deiner auch so, beim Aufklappen? Bei meinem habe ich deswegen versucht das Scharnier etwas zu ölen, aber es ist immer noch recht laut. Auch habe ich den Bolzen wie im PoFo-Wiki beschrieben bearbeitet, um hoffentlich die Lebensdauer des Bildschirmkabels etwas zu verlängern.
Für mein erstes Projekt in Rust suche ich noch eine geeignete freie Lizenz. Da es sich um einen Dienst handelt, welcher hauptsächlich als Webseite genutzt wird, hatte ich an die AGPL gedacht. Mir fiel aber auf, das die meisten darin verwendeten Bibliotheken MIT und Apache lizenziert sind, was ich eher von Software kenne, die auch kommerziell genutzt werden soll. Für Empfehlungen die für oder gegen eine der genannten Lizenzen sprechen bin ich dankbar. #FollowerPower
Weil es mal echt an der Zeit ist: Danke an Euch alle, die Ihr das Netz mit Euren Instanzen verschiedenster freier Software auch und gerade in dieser Zeit so nützlich macht. Danke für alle Helfer und Maintainer des freien Internets. Danke #Fediverse #floss #bleibtwachsam #decentralisation
Neue Services: - https://test.dssr.ch/ - Infos zur eigenen Verbindung mit dem Webserver auf (IP, Verschlüsselung, etc.) - https://meet.dssr.ch/ - video meetings, etherpad, screensharing Hinweise und Verbesserungsvorschläge sind herzlich willkommen!
Letzte Woche habe ich endlich meine fast 10 Jahre alten KVM Server gegen neue ausgetauscht. Aus Bequemlichkeit setze ich auch weiterhin Proxmox ein.
Die alte Anlage bestand aus zwei Pizza-Schachteln mit je Quadcore-Xeon, 24 GiB RAM and 4 x 1 TB SATA HDDs im Software RAID-5 und DRBD. Die neue Anlage besteht aus drei Einschüben, welche sich ein 2HE Chassis teilen mit je 2 x Hexacore Xeon, 32 GiB RAM und 3 x 500 GB SATA SSDs im Software RAID-5 und Ceph. Beide Anlagen konnten durch die Replikation des Storages live Migration und Hochverfügbarkeit für die VMs nutzen.
Die neue Anlage ist neben Gigabit Ethernet (einmal DMZ für die VMs und einmal LAN für die Clusterkommunikation) auch noch mit zwei Infiniband Adaptern ausgestattet (einer onboard und der andere as PCIexpress Karte). Damit habe ich noch ein drittes Mesh-Netzwerk gemacht, welches 40 Gigabit pro Link für die Storage-Replikation nutzen kann. Diese Bandbreite erreiche ich zwar mit iperf nicht (10 Gb/s auf dem den Infiniband-IP-Stack) aber die SSDs würden unter id... mehr anzeigen
Letzte Woche habe ich endlich meine fast 10 Jahre alten KVM Server gegen neue ausgetauscht. Aus Bequemlichkeit setze ich auch weiterhin Proxmox ein.
Die alte Anlage bestand aus zwei Pizza-Schachteln mit je Quadcore-Xeon, 24 GiB RAM and 4 x 1 TB SATA HDDs im Software RAID-5 und DRBD. Die neue Anlage besteht aus drei Einschüben, welche sich ein 2HE Chassis teilen mit je 2 x Hexacore Xeon, 32 GiB RAM und 3 x 500 GB SATA SSDs im Software RAID-5 und Ceph. Beide Anlagen konnten durch die Replikation des Storages live Migration und Hochverfügbarkeit für die VMs nutzen.
Die neue Anlage ist neben Gigabit Ethernet (einmal DMZ für die VMs und einmal LAN für die Clusterkommunikation) auch noch mit zwei Infiniband Adaptern ausgestattet (einer onboard und der andere as PCIexpress Karte). Damit habe ich noch ein drittes Mesh-Netzwerk gemacht, welches 40 Gigabit pro Link für die Storage-Replikation nutzen kann. Diese Bandbreite erreiche ich zwar mit iperf nicht (10 Gb/s auf dem den Infiniband-IP-Stack) aber die SSDs würden unter idealen Bedingungen sowieso höchstens 3 x 500 MB/s, also 1,5 GB/s oder 12 Gb/s liefern. In der Praxis messe ich bisher 200 - 300 MB/s wenn ich z.B. VMs kopiere. Hat also noch Luft nach oben.
Leider ist einer der beiden alten Server vor einigen Monaten ausgefallen und mehrere Reparaturversuche ergaben, dass da was mit dem Mainboard nicht mehr stimmte. Nach so vielen Jahren Dauereinsatz verüble ich das dem Server nicht und es war die Gelegenheit endlich mal über eine Neuanschaffung nachzudenken.
Einige der VMs (z.B. mein Minecraft-Server) lagen auf den lokalen, nicht-DRBD Partitionen des ausgefallenen Servers und waren daher auch nicht Hochverfügbar ausgelegt. Nachdem ich letzte Woche die neue Anlage im RZ eingebaut und alle VMs migriert habe, hatte ich endlich wieder Zugriff auf diese. Da diese monatelang ausgeschaltet waren, habe ich die erst mal gründlich durchgepatcht. Etliche VMs mussten noch von Debian Jessie auf Stretch upgegradet werden.
Was mich dabei geärgert hat, war der höhere Arbeitsspeicher-Verbrauch. Etliche VMs liefen noch mit 256 MiB RAM und konnten auch problemlos geupgradet werden. Nach dem Reboot liefen Sie anfangs gut, aber je nach Anwendung schlug dann bald mal der OOM (out of memory) Killer zu. Aus Performance-Gründen betreibe ich die VMs alle ohne Swap. Soweit ich das sehen konnte ergibt sich der höhere Verbrauch einerseits aus einem etwas grösseren Kernel und dem Wechsel vieler Systemtools auf Python 3, welches scheinbar deutlich verschwenderischer mit dem Speicher umgeht als Version 2.
Was waren das noch für Zeiten, als eine VM mit 128 MiB RAM mehr als genug war um einen Bind oder apt-cacher zu betreiben.
looks more like
as opposed to
elrido
Als Antwort auf elrido • •The binary acts as a multi-call one, when started with the regular rocket environment parameters it will launch the web service. But if the CRON environment variable is detected it will only execute the requested cron mode and exit. This is not a normal use case for a rocket application, so I can only really blame myself for this quirk.
One thing I did also find out is that my test code did have side effects, because most tests use a shared database (sqlite), so I had to switch to running all tests single-threaded (which is slower, takes up to 10s) as otherwise I got random failures when some tests deleted othe... mehr anzeigen
The binary acts as a multi-call one, when started with the regular rocket environment parameters it will launch the web service. But if the CRON environment variable is detected it will only execute the requested cron mode and exit. This is not a normal use case for a rocket application, so I can only really blame myself for this quirk.
One thing I did also find out is that my test code did have side effects, because most tests use a shared database (sqlite), so I had to switch to running all tests single-threaded (which is slower, takes up to 10s) as otherwise I got random failures when some tests deleted others data or couldn't add new instances when these were already present. This could obviously be fixed by either providing each test with a mock database or just be more careful designing the tests such that their order doesn't matter (i.e. using unique keys to add to the database for each test.
Boring details start around this point in the commit.