MidnightBSD Developer Journal

I just added a new port, wpop3d, to the cvs repository. The port is a pop3 daemon. (obviously)

I patched MidnightBSD for the recent OpenSSL and BIND security vulnerabilities. This should keep us in sync with FreeBSD on security for the year to date.

I commited a patch to the ATA driver which corrects a few possible deadlock cases. Its not quite as good as it could be, but it should at least prevent some problems. It needs further testing as my development machine only has IDE optical drives. This may correct some of the problems noticed when deleting a large directory (rm -rf) or other highly intensive disk operations. I'll test upgrade another system when I get more time. Archite will also be testing the patch tonight.

This is the first real src commit in awhile. I've been busy with ports lately. Once the patch is tested, its strongly recommened you build a new kernel from it.

I'm in the process of making changes to the perl port after feedback from users. I'll post when its clear.

I've also had reports the time zone is not set properly in the installer. I have a feeling I will be working on the installer quite a bit soon.

On a side note, I noticed a post about localhost performance recently. FreeBSD is working on the low performance and determined its related to preemption. If you compile a new kernel without it, performance will improve in some cases.

The xorg port joins the small list of ports that we've tested and patched to work with MidnightBSD.  It is an old version, but it works. (6.8.2) 

Working: (note i have libtool15 installed)
perl 5.8.8, imake6, xorg 6.8.2, aterm, windowmaker, zsh, libtool15,  bash2, atk, glib 2.6.x, gtk+-2.6.10, pango 1.8.2

Failing:
libtool13, firefox (configure script failure)

I posted a snapshot of the ports on the ftp server.  It will be mirrored by ISC eventually.  I also started putting distfiles up.  The ports tree doesn't use it yet, but you can manually fetch the source if needed.

mergemaster was patched over the weekend.  It now correctly handles /etc changes. 

Archite created a list of security areas where we can improve in the default install. 

Most of my weekend was spent researching ports.  I posted a perl port update that allows perl 5.8.8 to build with default options.  Threaded perl is not guaranteed to work, etc.  Quite a few things depend on perl building including the x11 port. 

Archite is working on adding sudo to the default install.

We've received a lot of attention from GNUstep fans.  I think my choice to include GNUstep is the right one. 

I think I must clarifiy my position on licensing.   Anything going into the kernel must be BSD licensed.  Userland consists of software falling under several licenses.  Creating a GUI environment on all BSD licensed software does not seem possible.  I feel it is ok to use BSD or GPL software in an os environment, but do prefer BSD licensed software.  I do not have a problem with binary blobs in drivers, but wish to avoid them when possible.  I think OpenBSD has done some amazing work with their OSS drivers for WIFI, but as a desktop BSD I can not be so strict.  I do hope code sharing between MidnightBSD and the other BSDs will remain possible in the future.

Here is an informal list of software I am considering for MidnightBSD:

GNUstep (almost in the bag)
Etoile (including their window manager)
Firefox
Thunderbird
aterm
The gimp
gaim
A selection of GNUstep basded software.

I have not decided on an office application.  OpenOffice isn't exactly my favorite and its rather large.  It also requires java for some functionality. 

I've also received a lot of questions about the installation of MidnightBSD.  The installer is hard to use and its a stumbling block.  It will be replaced, but most likely not for .1 release.  I'm 50/50 between a graphical installer using x11 and a command line installer like bsdinstaller.  (DF, etc)

Ports are another hot topic.  I'm at a crossroads with ports.  FreeBSD style ports are tough to maintain.  I would have to fork ports and improve them as I go.  I don't personally have much interest in the ports infrastructure beyond a graphical management utility for end users similar to Fink on OSX or Ubuntu's add/remove features.  pkgsrc from NetBSD is a possibility down the road.  Right now it does not build.  The GNU config project who maintains the config.sub and config.guess files has not patched for MidnightBSD yet.  Any software using configure fails.  The patch is simple enough.  If you need to do this yourself for now, I suggest you look at FreeBSD or DragonFly's (especially DF) if its a recent copy of the files.  You'll see the changes you need to make.  Hopefully that project will patch soon. I've sent a patch to perl maintainers to add a midnightbsd.sh hints file to solve problems with that.  FreeBSD's hints file will allow it to build provided you hack out the old entries and adjust version number checks.  We are .1 and yet similar to 6.1 FreeBSD. 

If someone wants to join the project and work on ports infrastructure I'll fork it.  If someone wants to manage package builds and work with pkgsrc, I'll go that way.  Frankly I don't want to work on ports.  I have many other issues like the kernel, userland, and the gui environment to worry about.  In the short term, I'll simply maintain a freebsd ports fork which will have dated and often non-building software. :(

You can check out our ports from cvs using "ports".   portsnap will continue to point to freebsd ports for now and pkg_add -r will fetch from thier stable tree for now. 

A new snapshot was posted on the ftp and http server today. It should be mirrored by isc soon. The torrent still links to the old one and will be fixed in a few days.

Again, I had a focus on the kernel. I fixed a bug with usb keyboards where they'd start repeating keys, especially arrow keys! It started after my kbdmux update. So far so good. It mostly only occured in x11.

Some of the network card code was updated including vge and mii which it depends on. I also added the ifmedia_baudrate function from freebsd and netbsd.

hwpmc was updated but I don't know how well its working yet. In the process, I patched the vm's zeroidle and it seems peppy. I should start doing formal benchmarks with these changes. I can say the interactivlty imporved greatly.

I have one more kernel patch i'd like to apply before .1 release. The remaining work is all in userland. Mostly we need to get some ports working and get x11 installed. x11 will be installed as port to ease in upgrading and dependancy checking. I may ship an old version just to get something working out of the box soon.

userland should be fairly safe from a security perspective now. We are current on patches that have been added to freebsd up to this point. Our ports tree is very dated and some of that software is not safe I would imagine.

I've been running on the em driver code since about 3am.  So far so good.  I'm still not happy with the network performance though.  Further investigation is needed.

I am in the process of setting up a second server to help distribute the load.  There has been some stability problems with my current amd box.   I'm debating about moving cvs over to it.  If so, i'll setup a new third level domain cvs.midnightbsd.org to simplify things.  That system has a via vge gigabit NIC.  I'd like to test the driver and compare its performance to the em's. 

The site was down earlier today for a hardware upgrade.  Additional ram was added to the cvs/web server.  This should solve some of the problem we've been having lately.  Its still recommened you download midnightbsd from the isc server.

Adam patched the ppp(4) buffer overflow also found in FreeBSD today.  Its recommended to update to the latest sources to receive the patch.  If you are using the last x86 iso, please remember that the password format has changed to blowfish and we've switched to OpenNTPD/rdate.  OpenSSH was also updated to the latest version recently.  Another words its a bit of a jump. 

I've found some problems with soft updates today as well.  Large amounts of disk io on raid volumes can cause soft updates to consume all available memory and crash in some cases.  Its not always consistant, but does happen.  If you experience this problem, disable soft updates on the volume.   I'm experimenting with several kernel patches right now that may help a bit with this and other problems.  I do not think these patches will completely fix soft updates. 

There have also been some problems found in ufs recently where deadlock can occur with snapshots. 

If these recent kernel patches work out, i'll do a new iso release soon.  This might be the beginning of some serious kernel changes.  When I started this project, I was mostly concerned with userland changes up front.  I was hoping that I could avoid major kernel changes until after a 1.0 release.  It appears that is no longer the case. 

For anyone tracking FreeBSD, the PPP patch coinsides with FreeBSD-SA-06:18ppp. 

I'm now building leapseconds files in /usr/share/zoneinfo/right to work with rdate's -c flag.  This was commited yesterday. 

The new website project is coming along.  I didn't get much opportunity to work on it today with all the other issues popping up.  We have someone working on a new logo for us and I've made a new temporary logo that will go with the new theme.  I've decided to drop the current color scheme even though its somewhat unique among the other BSDs. 

For those of you who don't know, MidnightBSD was named after my cat.  He loves to sit on computers.  The butterfly never made sense with the name and was just something I could throw together in illustrator. 

On the desktop environment front, fvwm-crystal looks like its going to be our temporary window manager for sure.  WindowMaker.info appears to be down and the freshmeat page is also disabled.  I don't want to track a dead WM.  Regardless of our final choice, I have every intention of including GNUstep with our distro.  There isn't much US interest, but there is a great deal of interest in europe. 

On a final note, my kernel just built so its testing time.

rdate is in the tree but i'm not quite happy with it yet.  I'm going to go back to the code when I get a chance.  I also imported updated timezone files last night.  That's not too bad to do. 

Today while looking at df cvs, i noticed they also imported OpenBSD rdate.  That means eveyrone but FreeBSD is using it provided netbsd still is.  I see a convention.  Of course its in freebsd ports anyway...

I'm toying with adding vesa in the kernel by default.  It doesn't make a lot of sense on servers, but it does on desktops.  There are some cases it would be nice on a server too.  I may need to create a desktop and server kernel config soon or do it by architecture.  I'm not sure what happens on sparc64 with vesa for example.  I need to look into that. 

I played with fvwm-crystal last night.  Now that i've tried it, my opinion has changed some.  Using the dock or start menu (nebula?) style it wouldn't be too foreign to others.  I did have a few crashes and certain mouse gentures could really cause some chaos with my mom.  Her app windows would disappear out of nowhere on her.  That might be fixable with customization, but the crashes are not so easily.  It is a devel version of fvwm 2 as a base after all.  Maybe when it goes stable...

Aside from that, I think we've exausted window manager choices.  WindowMaker is still my favorite for personal use however it also crashed when playing with some gnustep apps.  There could be something strange with my system as well.  I'm also using packages from stable for some of it so that could be the problem.  (fvwm is NOT a package) I'll look at the core files later.

Finally, portability is an issue.  We need to make sure the window manager will actually compile on sparc64, etc.  Firefox will not compile on freebsd last i checked.  I think gtk is broken.  I'm seriously thinking about making my own wm now.  I have an interface in mind already, its just picking the libraries to use with it.  We may just do a release with fvwm and see what people do.  I strongly favor using either gtkmm or gnustep for our graphical apps.  The pc bsd installer code is very clean but uses qt.  Its still a model for a good installer.  I did a test install today and was quite impressed.  We need to get there at least. 

I'm thinking these issues must be resolved for .1 release:

zoneinfo Etc/UTC and rdate fix
rc script cleanups (formatting and any necessary changes)
man page clean up (for additions/changes we've done)
All urls pointing to midnightbsd servers. 
Decide what to do about pkg_add and portsnap along with ports.
Some ports ready.  At least xorg, a browser, and some common libraries.
Some update to the em driver.
Look at other contrib software to determine version status.
Changes to the installer to remove freebsd specific features, etc.

.2 release will need this:
security patch procedure including binary patches (roadmap for it at least)
more ports, port utilities
improvements to installer
any bugfixes and some driver updates
more work on the gui environment

By .3 i want to have the window manager determined and hopefully working.  If i write my own, it would be started and useable to some degree.

The ISC has setup a mirror of MidnightBSD on http://mirrors.isc.org. Special thanks to them. Their mirror contains the entire FTP archive including the recent sparc64 snapshot.

Adam upgraded OpenSSH in the tree yesterday. I'll attemp a new i386 build when I get a chance.

I'm working on migrating to OpenNTPD. I've got about half the work completed. Anyone having problems with configure scripts should simply copy the FreeBSD releated entries in the config.guess/config.sub and replace FreeBSD with MidnightBSD and freebsd with midnightbsd. Also remove any additional entries for GNU "distros" of freebsd. I asked the GNU to patch their scripts. We'll see what happens.

I've had to disable ftp and http a few times. The traffic has been so bad nothing would come in or out. I'm hoping the mirror will help. I appologize to anyone downloading when that occured.

I'll write another entry when the source tree is "safe" to build. If you don't need ntp, you should be able to just delete those directories src/contrib/ntp and src/usr.sbin/ntp and it should work alright.

Finally, we've been looking at countless window managers. Several people have expressed interest in xfce. We haven't made a final determination but xfce, fvwm-crystal, WindowMaker, something custom, or some offshoot work for the etoile project are all in the running. Two of these choices don't exactly fit into my usability requirements. If they were chosen, it certainly would not be perminent. xfce has some nice points, but its also quite mainstream. was hoping to be a bit unique.

I've just posted the first snapshot build for sparc64 on the ftp server.  It was built on a sun ultra 10 3d creator.   It took me about 11 hours to complete the snapshot build. 

I was also working on a new i386 snap, but my build system experienced some problems today as my air conditioning was on the fritz.  I'll attempt a build again in a few days. 

i386 is available via bittorrent, but sparc64 is ftp only for now.

There will also be some changes commited to ksh soon. 

A few people have inquired about the download speed problem from the ftp/webserver. I am running the server on a business cable modem package which does not have a lot of upstream. I did a recent speed test using speakeasy.net's server. 8314kbps down 823kbps up to chicago. I need a mirror quite badly. I sent an application to ibiblio but have not heard anything.

As for the iso image, the md5 for disk 1 does look wrong. I'm going to switch out the network cable and post a new snapshot for x86 and sparc64.

I did download the iso last onto a desktop and did a virtual pc test install. It worked ok.

Note on installtion: Do not use "all" or "xorg" options when it asks you to select relevant packages. The current snapshots do not include ports or documentation (aside from man pages) so those options will cause you to get errors during the install.

pkg_add intentionally points to FreeBSD's server and will fetch 6-stable packages. They still work for now and until I get ports up its a convenient way to get xorg and other items installed. I have not altered the sup files yet either.

The recommended way to get source is to check it out using the new anonymous cvs. (as of yesterday) Information is in the develop section of the website.

MidnightBSD now includes OpenBSD 3.9's ksh which is based on pdksh.  ksh is not included in /rescue and is not static. /bin/sh has not be changed to ksh either. 

I want to import a few more changes and release a new snapshot in the next few days. 

I'll attempt a sparc64 build soon. 

Today I made some changes to sysinstall.  I'm hoping to get ftp install working with the next snapshot.  I also imported the latest sendmail release over the weekend.

Over the weekend, I created a new snapshot iso of MidnightBSD.  It includes 7 security patches and a new version of Sendmail.  It has has some minor fixes with the rc scripts. 

The cvs server was recently moved to stargazer.midnightbsd.org.  I've also moved the website to the new server as well. 

xorg is not currently building.  I haven't had time to look at it yet, but you can simply use the FreeBSD 6 stable packages safely.

I was able to get some snapshot isos built.  I'm throw a link in to them soon.  The sticking point with me right now is what to do about a window manager.  There are quite a few choices and many are GPL licensed.  I'd prefer to use BSD or MIT licensed code when possible.  I don't see much choice though.  Here's my short list: (not that short)

WindowMaker
Sawfish
XFCE
something custom...
OpenBSD's modified twm?

I also considered enlightenment 1.6 and 1.7.  I liked 1.7, but its not stable enough yet.  Afterstep is too weird in recent versions. 

The problem with window managers is that for some they are religion.  Of course ports will contain others anyway, but I want a solid default.  It refects the usability and spin put on my project.

I'm leaning toward WindowMaker.  I prefer it personally and I showed a few users WindowMaker vs Gnome and KDE desktops.  Most described WIndowMaker as simple.  I didn't have them use it though.  Usibility is very important to me.  I don't want KDE because two other BSD projects are exploring that and I'm not sure how receptive Gnome developers would be to including patches I might need.  If I go the gnome route, I might as well call it ubuntubsd or something.  I don't want that.

I'm just going to do WindowMaker in the first few releases and work on something custom long term.  That seems the best. 

Still don't know what toolkit to use though. :)