Sun, 28 May 2023

2:59 PM - New Browser: Raphael

Midori 9.x, the default browser in MidnightBSD, went EOL.? The 10.x version of the browser is electron based and doesn't support many operating systems.? That put us in a pickle.? We need a default browser and webkit browsers have been the most consistent with working on MidnightBSD.? Therefore, we forked Midori 9 as Raphael.? It's on github at https://github.com/midnightbsd/raphael/

All browsers don't have to be chromium based.? That's a bad thing!?

We've also made progress on the firefox-esr port and getting that to build.? It's close to working but still some hurdles.? If we can get that fixed, we have two?decent options.

If you're interested in helping with the Raphael browser or fixing the firefox ports, we are open to pull requests!

()

Tue, 16 May 2023

1:02 PM - Short term goals

Here's a list of some short term goals we're hoping to get done in the next few months:

  • mports: fix firefox-esr port

  • Add llvm12 or 13 to mports and update mesa libraries using it

  • Update perl in current?

  • Look into updating CPAN included in stable/3.0 perl to fix a CVE

  • Import bug fixes / improvements from FreeBSD 12-stable branch from September-December 2022 into midnightbsd current.?

  • Look into updating the mport manager to work with newer libmport versions

()

Fri, 12 May 2023

10:05 AM - Project status update

First, we've had several contributions over the last few weeks.? Thanks, new contributors!??

In current, we imported 2?months' worth of bug fixes from FreeBSD 12-stable.? ?We've also made several changes to the firstboot script.? For folks running current, please test the new firstboot script changes.? If you want to run it, just delete /etc/fbreceipt and run /etc/rc.d/firstboot start (after installing the latest version)?

We've also had a few server outages lately. Here's what's going on:?
1. We had a botched upgrade to our primary virtualization server.? This was running VMware ESXi.? A CPU upgrade killed a motherboard as the CPU wasn't actually supported, as I misread the support list.? The motherboard was replaced.? Additionally, some memory modules had failed in that.? Those were replaced.? We are in the process of migrating VMs to a bhyve environment.??

2. We upgraded our primary network switch.? The new switch is an aruba instant on 1960?12XGT and replaced an aging buffalo bs208 8 port nbase-t switch.? Since the new switch doesn't have 2.5G support, we had to upgrade the network card in our file server to an intel x540.? (it was using a realtek dragon 2.5G)?

3. While upgrading the network switch, we uncovered a thermal issue with another server.? We'll have to bring that down to replace some fans soon.? It's our other virtualization server used primarily for package builds.

()

Fri, 5 May 2023

9:34 AM - Project Update

project update:

We're still restoring VMs from our recent server issue. Some services are back up, but we still have a few like Jenkins, OpenGrok, and build nodes for our package cluster to restore.

We've been running package builds on a single server lately. The latest amd64 run has had a few issues that we're working through.

mport package manager has received several updates in recent weeks. It now supports an audit command that lets you check for CVEs against a copy of the NVD data.
Usage:
mport audit
mport audit -r
mport -q audit

The first version prints a list of all CVEs with descriptions for each package.

The second includes a list of packages that depend on this vulnerable port so you can also update those.

The third doesn't give details about the vulnerabilities and just prints a list of vulnerable packages with package name and package number using the "global" -q aka quiet flag.

This isn't included in MidnightBSD src git yet as we're working through a few bugs. You can check it out and try it now though git clone https://github.com/midnightbsd/mport.git

()

Thu, 27 Apr 2023

Fri, 21 Apr 2023

9:42 AM - ZFS Replacing a bad disk

First, identify the failed disk. You can run zpool list to identify the failed drive.

zpool status

If you were clever, you named your disks with meaningful labels. If not, it might be trial and error. If it's ada0, you know it's in the first SATA port, for instance.

Remove the old disk. Confirm the disk is gone in the BIOS after booting the system.

Now put in the new disk.

It's recommended you boot into single-user mode. (see the option on the midnightbsd boot menu)

Now verify the OS detected the new drive.

camcontrol devlist

You should see the new drive. If you placed it in the same sata port, it is likely the same name. For our example, ada0

gpart create -s gpt ada0

gpart show

gpart add -a 4k -t mnbsd-zfs -l mydisklabel ada0

gpart show

Now if you run zpool status again you should see one drive marked as offline with a long number identifier rather than a label or device name (ada0)

zpool replace mypoolname longnumberfromzpoolstatus gpt/mydisklabel

If you didn't use the label flag above, replace that last part with ada0

This will start the resilvering process, which can take several hours. You can run zpool status to check the status of this process.

While you can boot the system at this point, and it will work, it will run slowly. If possible, let it resilver in single-user mode. It will go a lot faster.

Note there is a better guide on this here:?https://dan.langille.org/2015/08/03/replacing-a-failed-drive-in-zfs-on-freebsd/?The big difference is the partition type above (mnbsd-zfs vs freebsd-zfs)

()

9:40 AM - Access to packages available

If you're an end user and you want to know what software is available for MidnightBSD, the best place to go is the?app store. You can also use the mport search command to look for available packages, or in mports use make search key=

For software developers, there are several ways to get the data for consumption. Our package build cluster software has several endpoints available including:

https://www.midnightbsd.org/magus/api/latest

https://www.midnightbsd.org/magus/api/run-ports-list?run=328&status=pass

The latest endpoint gives you a list of packages available in the most recent magus runs that have been blessed (published). The other one lets you look at a specific run and filter on status.

There are multiple APIs available in the app store as well. Consult the midnightbsd-app-store repository on GitHub for details on those.

Finally, there are the actual sqlite3 databases generated by blessed package builds per arch and os-release.
http://index.mport.midnightbsd.org/amd64/3.0/index.db.bz2

()

9:38 AM - Using the MidnightBSD security advisory api

We've updated the security advisory api recently and it now has a new endpoint that lets you lookup by CPE.?

https://sec.midnightbsd.org/api/cpe/partial-match?cpe=cpe:2.3:a:openssl:openssl:1.0.1d:::::::*

This URL will lookup CVEs that may impact openssl 1.0.1d?

We're considering possibly using this with mport to provide security notifications via a new command.? Right now we have the security-advisory-client in mports that hits different endpoints within this API to provide auditing data.

()

9:34 AM - Server outage

We've had a bad hardware upgrade and some of our infrastructure is down.? This includes the midnightbsd app store, jenkins and portsnap updates.? We also use this system for a number of magus package builds.?

A replacement motherboard should arrive soon to fix this.

()

Tue, 4 Apr 2023

10:38 PM - MidnightBSD 3.0.1

3.0.1 was tagged in our git repository and we've started building ISOs for it.? It includes several security updates such as OpenSSL 1.1.1t, doas 6.3p9, a fix for a telnetd vulnerability.? It also includes some cleanup work on rc.d scripts, fixes for periodic scripts that were incorrect around ntpd and restoring msearch/mport db backups.? ?We also updated mport to 2.2.9 which fixed the mport mirror list command.

()

Thu, 23 Mar 2023

11:30 PM - MidnightBSD 3.0

3.0 i386 packages haven't been built for the release yet.  There are some older ones available but without the full desktop environment.  We'll be building those this weekend. 

We're uploading some ISOs but still need to do final testing on them.

()

Tue, 7 Mar 2023

6:27 PM - MidnightBSD project updates

First, the 3.0 release has been blocked for some time due to a few key mports that just refused to build.  Today, we got LLVM 9/10/11 working on MidnightBSD 3.0.   This unblocks us to work on the remaining desktop ports to prepare for the release. 

We've published updated packages for 2.2 amd64, 2.2 i386, and 3.0 i386 over the weekend.  A new package build for 3.0 amd64 is in progress. 

Several bugs have been reported with the mport package manager stemming from some changes made back in December.  Users on 3.0 or 2.2 from git are impacted.  If you have 2.2.7 or older installed, it's not a problem.  We have a workaround committed in 2.2 stable branch to allow folks to install packages.  We're working on a better fix for the 3.0 release cycle. 

We fixed a recently reported security issue in the magus package cluster web interface.  Thanks for the report!

The new tentative plan is to release MidnightBSD 3.0 by the end of April.  This gives us time to get mports in shape and do additional testing.  

We've added some documentation on settings to use when creating virtual machines of MidnightBSD guests on the MidnightBSD wiki on Github (under src). 

Known issues: 
Some options don't appear to be working on some mports.  
Perl ports on some 3.0 installs aren't working properly.  
OpenJDK 8 is broken on 3.0
Avahi and boost broken on 3.0 and the biggest blockers to release at the moment.

()

Mon, 13 Feb 2023

3:39 PM - MidnightBSD 2.2.7 release

We released a minor update to MidnightBSD which contains the following changes:

tzdata 2022g

less 551

ISOs are on the main site and starting to copy to mirrors.

()

Fri, 3 Feb 2023

12:53 AM - MidnightBSD 3.0 status update

The 3.0 stable branch is pretty much ready.  We may update sqlite3 yet. 

The delay in the release has been due to issues with several mports.  We're still working through those problems, but one can't ship a desktop OS without a working desktop.

()

Sat, 15 Oct 2022

Tue, 7 Jun 2022

Tue, 23 Mar 2021

1:40 PM - mport package manager

Several bug fixes have been made on the mport package manager. 

It's now using a sha256 hash for verification of files rather than md5. 

It gracefully ignores missing files on a mport update.

mport install can now update dependencies it needs to install a package.

()

Sat, 27 Feb 2021

1:17 AM - A comparison between mports and FreeBSD ports

MidnightBSD mports started as a refactor of the FreeBSD ports with some influence from OpenBSD many years ago.  For the first few months of the project, we actually had a "ports" that was literally a checkout of FreeBSD ports with a bunch of changes to make things build. It was horribly unreliable and a huge mess.  Chris Reinhardt did a big refactor along with input from Phil Peria and I.  We started moving features into the extensions directory, which FreeBSD is now doing with their Uses directory. We also broke up a bunch of the bsd.port.mk and bsd.*.mk files into components as smaller chunks to work on and pull them in as necessary. Then the mport package manager prototype in perl happened as a replacement for pkg_tools (pkg_add, pkg_delete, etc).  Chris ported that prototype mostly to C and then I finished the port and wrote the driver program for it.  Chris got it to a point it could install and deinstall packages from the mports tree.  

One of the big design goals we all agreed on as well as our then security officer Adam Karim, was that we wanted to install the package into a directory, which we called FAKE_DESTDIR, and to force all installed mports to be packages and then installed by the tool.  This cut down on installation errors from packages that we frequently had as everything was tested this way.  It was a bit slower in mports, but the benefits outweighed the risks. This wasn't really our original idea and it was inspired by OpenBSD ports at the time. Still, it worked out really well.  As we were trying to adapt FreeBSD ports, we made a bunch of decisions that are still a difference from how FreeBSD ports work today. 

FreeBSD eventually introduced STAGEDIR which aside from being a better name, also works similarly to FAKE_DESTDIR with a few differences.  When we did the change, many ports did not have or honor a DESTDIR directory.  We found that the ones that did would just work if we made the do-install and post-install targets include it in make invocations.  In contrast, FreeBSD often has to add STAGEDIR everywhere to get it to work.  There are cases their way is better such as dealing with ruby gems or similar funky ports.   We have some mechanisms for those problems too.  FAKE_OPTS has a trueprefix option that with change the behavior of the do-install and post-install targets. Sometimes you will see ports that are double appending the DESTDIR  (/usr/mports/mycat/my/port/fake_install/usr/mports/mycat/my/port/fake_install/usr/local) type of nightmares. This works around that.  In some cases PREFIX includes both the DESTDIR and the PREFIX from a makefile sense. This is only true for do-install, post-install, do-fake, post-fake.  So if you have a newer target like ones based on options, it will not apply and you will need to use FAKE_DESTDIR.  If you actually need the PREFIX value without DESTDIR, it's available as TRUE_PREFIX.  

Another difference is the order that different parts of the framework are loaded. This can cause weird bugs if you're trying to copy a freebsd port over.  It's very evident with some of the python ports. Different modules will sometimes fail to save the options selected using the options framework.  It's a combination of the loaading order and the name that is used to distinguish different port options.  In midnightbsd, port names must be unique or it really breaks the package manager index. OPTIONS mostly work the same aside from that bug. 

Fetching distfiles is another area that can differ slightly.  Most of the same features work, but there can occasionally be difficulties fetching from github or gitlab due to the subtle differences in how package names are used in the framework as well as features we're supporting.  Sometimes one just has to add the DISTFILES explicitly or set the git tag to get it to fetch right.  MidnightBSD like FreeBSD includes fetch and libfetch.  Both mports and mport use these to get files.  The MidnightBSD version also includes the OSNAME in the user agent string so you can actually tell it came from MidnightBSD in logs. 

mport package manager doesn't support LUA scripts like pkg.  Most of those trigger operations in the PLIST files. pkg-plist by default.  You will see basic commands like @dir or @mode that work in both.  Some have been augmented in mport with the use of hacks that substitute the commands for @exec or @preexec/@postexec commands in package plists. In other cases, the business logic is in the mport package manager and requires a minimum version. 

on that topic, we recently created a port to install new versions of the package manager.  That meant we needed to put it in it's own repo.  We're bumping the version number based on updates, but mport itself doesn't know about that. Instead it reports the OS version and it's bundle version.  That is the version of the package format and sqlite database schema in use. It gets bumped when we add breaking features.  This used to go out with new OS versions and we're still working on how to manage that going forward. We may add version numbers to the package manager to make it easier to determine in the future. 

We also are looking at adding UCL support to mport. 

Another area where things are quite different is how we build packages, where to find information for them and how to detect security issues in installed packages.  We have magus for package building which is a set of perl scripts included in the mports repository currently along with a postgresql database.   There was some work to rewrite this in java a few years ago but it hasn't been completed.  We also have a few endpoints that return JSON in there for services like repology to use as well as the MidnightBSD app store which is a searchable list of packgages.  the app store also powers metadata in the graphical mport package manager in gtk3.  Then we have the sec.midnightbsd.org security advisory site which is mostly there to work with a perl script in mports called security-advisory.  It will check installed ports against the website REST api for security issues. it's a bit slow because it fetches vulnerabilites for ALL versions from the API so as to not tip off our server what release you're using for security reasons.  We may know IP x is curious about apache httpd but we don't know what version.  We have considered adding some kind of range to this as apache in particular is very slow to process.  Magus has a web interface but it's optional.  You can actually do a package build just with the perl client that runs on a build node, and the indexer.  We then have a C program called bless that creates a sqlite3 database from the postgres database data for a given run.  We then vacuum it and bzip2 it and put it on the index site.  Finally, we mark it blessed=true in the database which is a signal for the app store and repology that those files are LIVE.

()

12:46 AM - BastilleBSD

We recently added a new port, mports/sysutils/bastille that allows you to manage containers. This is a port of a project that originally targetted FreeBSD, but also works on HardenedBSD. 

You can see the getting started guide at the link below. 

https://bastillebsd.org/getting-started/

A few notes on using this with MidnightBSD.  Our port supports bootstrap, create, stop and destory.  It may work with ZFS but we have not tested it.   If you are using MidnightBSD 2.0.x amd64, you likely want to use a version of 2.0.3 in your create statement.  MidnightBSD versions don't include RELEASE or other strings at the end so it's just a straight up version number.  

The getting started guide also recommends using PF.  The instructions will work on MidnightBSD just fine, but you do need to disable IPFW which is enabled by default on MidnightBSD first.  

The update/upgrade function does not work in MidnightBSD right now because it requires binary update support. We do have the binary in 2.0 as it was planned to add it but the server side work isn't done yet. 

There's also a bug in the port but it shouldn't affect operation on MIdnightBSD, just if you tried to use that code on FreeBSD with pkg integration.  This came up when we upstreamed it and should be fixed in that PR now, although we didn't bother to fix the port since it's not used on other systems.

()

Wed, 24 Feb 2021

10:40 AM - MidnightBSD 2.0.5

New 2.0.5 release tagged in git.

Happy 15th anniversary to MidnightBSD!

Fixes: pam security issue Updates: mport 2.0.5 tzdata 2021a Now uses sysrc for firstboot script.

We will do ISO builds later for this one.

()