MidnightBSD Developer Journal

If you're trying to run a recent elasticsearch such as 6.x on midnightbsd, you may find it doesn't startup.

Try adding the following to /usr/local/etc/elasticsearch/elasticsearch.yml:

xpack.ml.enabled: false

bootstrap.system_call_filter: false

MidnightBSD 1.2 release is now available. It's a security and bug fix release.

Most of the focus was on mports and base system third party libraries.

A lot of people have been curious what our plans are for this year. I've also had a lot of bug reports lately.

First, the plan is to release 1.1 in the next few weeks. This is mostly a security update with new versions of several popular software packages. It will be a small update from 1.0. As a result of this work, plans for package manager changes have been pushed to 1.2 along with the related installer work.

In terms of ports, we plan on looking into the flavors feature available in FreeBSD and other BSDs ports systems. Some of the plumbing has already been put into the package manager to support this. However, it will take a bit more work as there are modifications needed for our package cluster software to handle it.

The package cluster software is getting rewritten currently.

At the time of the 1.0 release, there were a lot of outdated packages. We've since fixed several and updated many gnome related ports. Qt5 updates are still on the todo list. It was also determined that there are problems with python ports as more things are transitioning to python 3.x. This requires the flavors work or a bunch of py3 ports to be created.

On the browser front, epiphany and midori were updated last month along with some of the webkit ports. The firefox port is partially done, but stalled on badly needed compiler updates and patches.

Work was started on porting LLVM properly, but complications arose upstream. We didn't get a hard no, but a soft no along with needs for a build cluster node for their project. If someone is willing to donate this or time to help with getting upstream patches, that would be quite helpful. Proper LLVM/CLANG ports are needed at a starting point to get Rust and other languages working more easily. This is a top blocker for really solving long term browser support issues.

We need help getting patches upstreamed with various projects.

MidnightBSD 1.1-CURRENT now has OpenSSL 1.0.2p rather than the 1.0.1 verisons.

Ideally, we will need to get 1.1.1 in, but that will take a lot more work in userland.

Anything built against the system OpenSSL should be rebuilt. Many ports should not be affected.

In light of other changes in the last 24 hours, it's ideal to rebuild world completely if you're running current.

mksh R56C and OpenSSH 7.5p1 are now in current. The latter removes support for the legacy v1 protocol.

Several bugfixes were done related to MSI-X allocation and for intel gigabit network cards. (em/igb)

Take advantage of our third party security advisory site. http://sec.midnightbsd.org/#!/ It has a searchable REST api and shows software from all software even windows :) It powers our security-advisory-client port

Perl client code: https://t.co/0feWsHnf3v

Backend source:https://t.co/4JitOuMG5U

New feature in current mport package manager:

mport config get lets you see a setting configured.

mport config set lets you set a setting.

This allows you to configure the region that you want to download packages from.

Hadoop 1.x was just added to mports in devel/hadoop. This took some time to port.

devel/glib20 was recently updated.

The 1.0 release is finally available. Still buidling packages for i386 and plan to do an amd64 package build later in the week.

The single largest issue with the release process has been the web server performance. The CPU is overloaded and has been at solid 100% for several days. The server has a core i7 7700 in it. I'm trying to figure out what to buy as an upgrade so that we don't continue to have this issue going forward. As it's actually blocked in multiple processes, a 6 or 8 core chip might be an improvement for the workload.

The 1.0 ISOs are on the FTP server. The release notes are nearly written. Waitnig on some updated packages and we'll be able to call this thing.

One of our servers was setup in MidnightBSD 0.7 and had an older ZFS configuration. Over time, the drivers were replaced with 4k advanced format disks. Since ZFS won't let you change the ashift setting on an existing pool, the drivers were not running at optimal performance. MidnightBSD 1.0 current warns about this sutation.

How to fix it?

In this case, there were 2 drives in a mirror and a cache disk as well as another backup pool. First, we performed a backup to the other pool just to be safe using zfs send -R mypool@snapshot | zfs receive -F backup/mypool, then we removed one of the disks from the pool using zpool detach ada4p1. We then blew away the partition and created a new gpt partion that was 4k aligned. gpart add -t mnbsd-zfs -a 4k ada4

Next, we created a new pool called tank with the one drive. We then ran zfs send and receive to copy data from mypool to tank.

Once this data is copied, we went into single user mode (shutdown now) and exported both pools. We then imported the new pool tank. zfs export tank; zfs export mypool; zfs import tank. This allowed our mount points to kick in.

Next, we repreated the process of repartitioning the second disk, ada3. We then add it by doing zfs attach tank ada4p1 ada3p1 to include the second disk in the mirror. Finally we did zpool add cache tank ada1 to re-add our ssd cache drive.

The drive will need to rebuild which can take some time.

This approach prevents any data loss.

We're currently working on a large merge of FreeBSD 10-stable (from late may) into MidnightBSD. A large portion of this work is done, but there are still a few loose ends.

• Symbol adjustments might be made to libc

• Several utilities are not connected to the build yet.

• There are problems building some of the release targets such as the uefi memstick

• A few third party apps were etiher newer or older and not updated. For example, our svn is still 1.8.x.

• There were bugs in the boot loader code due to a bad merge. Most if not all of this was fixed.

• Some architectures may still be using freebsd partition types, some utilities might not be switched over yet.

We also removed the sensors framework during this migration. There have been some locking problems in 0.8.x and it requires a rework.

Also, we now have bhyve. The good news is that it's possible to run FreeBSD on MidnightBSD in bhyve. The trick is to use a freebsd userboot file. You can compile one from FreeBSD 10.x on midnightbsd or use the new port. Another option that may work is using the grub port.

Initial testing of MidnightBSD on a Ryzen 7 1700 CPU with an Asus PRIME x370-pro motherboard indicate that there are some issues that prevent the system from booting.

There are two problematic areas:

1. The system timer detection code is not finding a timer that will work .
2. The Intel NIC included on this motherboard is not detecting properly and causing a panic.

You can check out MidnightBSD with git svn.

For developers
git svn clone svn+ssh://laffer1@stargazer.midnightbsd.org/home/svn/repos/src/ -T trunk -b stable -t releases
(replace with your username of course)

For folks looking to fetch the code, you can just use the http protocol instead. Use the URL:
http://svn.midnightbsd.org/svn/src/

MidnightBSD 0.9 current now includes the doas(1) utility, an alternative to sudo. Learn more about this utility http://www.tedunangst.com/flak/post/doas-mastery

I am experimenting with MidnightBSD on AWS. It's possible to take the memstick image of 0.8.5 and modify it to enable the network interface, sshd and a test user to ssh in and then create a AWS AMI. So far, I've gotten it to boot to a login prompt.
The EC2 steps are as follows:
ec2-import-volume ~/memstick -f RAW --region us-east-1 -z us-east-1a -s 10 -d "MidnightBSD 0.8.5" -O "${AWS_ACCESS_KEY}" -W "${AWS_SECRET_KEY}" -o "${AWS_ACCESS_KEY}" -w "${AWS_SECRET_KEY}" -b "mnbsd-vm"
find out if it's done converting
ec2-describe-conversion-tasks -O ${AWS_ACCESS_KEY} -W ${AWS_SECRET_KEY} --region us-east-1
create a snapshot (get volume id for last arg from previous command)
ec2-create-snapshot -O "${AWS_ACCESS_KEY}" -W "${AWS_SECRET_KEY}" --region us-east-1 -d "MidnightBSD 0.8.5" vol-0d149b0dfc22962b1
check on snap
ec2-describe-snapshots -O "${AWS_ACCESS_KEY}" -W "${AWS_SECRET_KEY}" --region us-east-1
register ami (snap argument shown from last command)
ec2-register -n "MidnightBSD 0.8.5" -O "${AWS_ACCESS_KEY}" -W "${AWS_SECRET_KEY}" --region us-east-1 -a x86_64 -d "MidnightBSD 0.8.5 AMD64 Test" --root-device-name /dev/sda1 --virtualization-type hvm -s snap-0a7bb785269bfb08e

MidnightBSD 0.8.5 RELEASE ISOs now available on our primary ftp server for 32bit and 64bit processors. (i386 and amd64) It includes all the fixes for mport package manager, and security updates to openssh, openssl, sqlite, etc.

I'm happy to announce the availability of #MidnightBSD 0.8 release! It is now available on our primary FTP server and uploading to our mirrors. You can download it from our website.

You may also read about the changes in the release notes. http://www.midnightbsd.org/notes/

MidnightBSD 0.8 amd64 packages have been updated.

Note that xorg is included!

There are 2,884 packages available (more than i386 now) including part of qt5 and lumina desktop environment.

Also, if you have issues installing packages, make sure you have the latest libmport and mport tools from SVN (0.8 stable).

The 0.8 release has been delayed due to bugs found with the package manager. As this is a critical part of the system, the decision was made to delay it.

We currently have developers debugging the problem.