MidnightBSD Developer Journal

First, a security update was applied to CURRENT for users who turn on the random IP id feature. It is off by default. This feature has a flaw in the random id generation.

See http://www.securityfocus.com/bid/27647/info

Second, we've updated a large number of ports in the last few days. The astro, archivers, accessibility, and net (except samba3) ports have been updated with the latest versions where possible. I'm currently working on the audio category.

Progress on Gnome 2.20 is still going strong as well. ctriv had a bit of a hardware snafu which has slowed the deployment of the new cluster software. I'm hoping to have many new port updates ready when it's back online.

Magus has been down since december for a rewrite. I've discussed this previously. The build master was brought down for a faculty member to take the video card in the last week. It had been up 113 days which is a record for any machines I'm running. The website has not been restored as of yet, but the system was upgraded to 1GB of RAM.

ctriv hopes to complete the software soon.

I've modified the analog port to report several additional BSDs including MidnightBSD, MirBSD and Dragonfly.

CURRENT now has a patch to correct a potential security issue with sendfile. Files were not checked prior to serving which would allow a file that was write only to be served. While this scenario is rare, we decided to fix it anyway.

sendfile is used by many daemons including Apache httpd.

Analysed requests from Fri-14-Sep-2007 11:13 to Fri-15-Feb-2008 10:04 (153.95 days).

Successful requests: 311,495 (10,072)
Average successful requests per day: 2,023 (1,438)
Successful requests for pages: 118,349 (4,210)
Average successful requests for pages per day: 768 (601)
Failed requests: 54,531 (1,623)
Redirected requests: 52,518 (1,814)
Distinct files requested: 41,433 (4,365)
Distinct hosts served: 19,626 (1,680)
Corrupt logfile lines: 15
Data transferred: 1.03 terabytes (21.44 gigabytes)
Average data transferred per day: 6.88 gigabytes (3.06 gigabytes)

I've updated several of the browser ports to correct security issues.

Firefox & linux-firefox
linux-flock
linux-seamonkey
Opera & linux-opera

I'm happy to announce the release of MidnightBSD for amd64 systems. (and Intel equivalents) This release does not include packages for KDE and several other programs like the i386 release, but you can build them with mports. XORG, Samba, WindowMaker, and several other ports are on the isos though.

The packages are also up on the FTP server along with the ISOs.

ftp.midnightbsd.org now uses round robin to speed up downloading packages with pkg_add -r, etc.

If you need to get to our main site, use ftp1.midnightbsd.org

ftp2 and ftp3 are aliases for our mirrors.

There was a minor build problem with sendmail in current. It has been fixed as of a few hours ago. Current has built successfully on an i386 box.

We're also testing a new threading fix for libpthread in relation to fork() calls.

During December, we had to shutdown the build cluster for the holidays. In the mean time, many new computers were purchased and shuffled around at EMU. Systems were moved and disconnected. Today, I've restored access to 3 cluster nodes. Each node currently has 0.1.1 (snap) on it which was basically 0.1.1 release. I'm upgrading the nodes to 0.2. These nodes are labeled build1, build2 and build3. We also have a 0.1.1 node and the build master working.

At the same time, Chris has been working on rewriting parts of our cluster software to work better with multiple architectures as well as support multiple releases concurrently. When the cluster is back online, we'll have 9-10 nodes running on p4 chips with either 0.1.1 release or 0.2 current. The cluster build time will be slower, but we can test both branches. A full build took less than a day with all the nodes. Usually it was done in 10 hours. Based on the increase in ports and less nodes, I suspect it will take a day or so to do a run.

In addition to this work, I'm also planning on setting up two machines to do amd64 builds, and one or two machines for sparc64. Chris already had a sparc on current for this purpose.

The Eastern Michigan University computer science department has been quite generous giving us access to so many machines.

I've been quite lazy about maintaining this developer blog. I'd like to catch people up on happenings in the project.

First, smultron has been working on a new website design for us. It includes a new logo design similar to the logo we're using on the blog and cia.vc. MidnightBSD was named after my cat and this new logo includes a cat as well as the obvious other usage of Midnight. I'm very impressed with his progress on the site. It will be published when it's complete. Originally, we had hoped to publish a new site with the 0.1.1 Release, but well things happen. The new design improves navigation quite a bit.

Chris (ctriv@) is working on libmport. This is the library that will be used with the command line and GUI replacements for pkg_add and friends. He has made great progress with most of the new code in his local subversion repository. He's promised a checkin of that code soon. raven@ has been looking at the libmport code in preperations for the new GUI tools to manage ports. I believe she is planning on using GNUstep. Chris will write the mport command line tool.

I've been working on several things. The build cluster found many ports that are broken and we've been trying to get many ports current or at least patched for critical security holes. KDE was updated over the weekend to 3.5.8. We're behind on many other ports including php, seamonkey (well this one isn't really done anyway), gnome related ports, etc. I committed an update for php5 today, but many of the "extras" are not updated yet. With the build cluster down, I can't test many ports at the moment. Please submit bug reports or email us if you find bad ports. (math/R is known to be broken as well as devel/ncurses)

The build cluster has not been run since mid December. We had to shut it down for the holidays at EMU and ctriv@ has been working on a new version of the software to allow us to test multiple OS versions and architectures. However, his work was stalled on that so that he could finish up libmport during the holidays. I suspect we'll have the build cluster hardware setup by next week and some of the machines upgraded to CURRENT. Our new plan is to run 5 machines on CURRENT and 5 on 0.1.1 for i386. We also want to test CURRENT on amd64 and sparc64. ctriv has a sparc machine for that and I'm hoping to get my netra working as well.

I have been working on the Live CD and installer as well. I'll post an update on that next week.

archite created a wifi network script that looks interesting. He's posted it for the OpenBSD community on undeadly as well. He's also been doing some scripting for things on stargazer.

We did an interview for the NetBSD pkgsrc 10 years celebration, however it hasn't been posted yet. It was a fun interview.

I've been upgrading our server "stargazer" which hosts the website, cvs and other services for MidnightBSD. I recently purchased two seagate drives for a RAID 1 on /home where I store the CVS repository and other files. The motherboard has an onboard intel raid controller (ATA).

That process was fairly painless.

The other issue is the failing video card. The fan on the current card is wobbling and occasionally stopping which has caused problems. I'm not in a position to run headless with the system, so I purchased a new evga nvidia 6100 LE AGP fanless video card from NewEgg. However, the package was stolen at my front door (or never delivered). After spending half my morning on the phone with Fedex and contacting NewEgg, I have a replacement coming in 5-7 days. Both companies have provided courteous customer service, but I'm left wondering what happened to my video card. Was it stolen on my doorstep? Did a driver forget it on the truck? Including shipping, the card was only $43 dollars. I guess at this rate, I could have gone to a local store and bought a card. Of course, then it wouldn't be fanless.

I'm left with a whining video card for another week. In case anyone is curious about the server specs, I'll list them now.

Dell Precision 650 Workstation
Dual Xeon 2.0Ghz
1.5GB PC2700 ECC RAM
1 Seagate 80GB IDE disk (2MB cache 7200RPM) (primary boot disk)
2 Seagate 160GB IDE 8MB Cache 7200RPM
1 IDE CD burner
integrated intel gigabit nic
ATI AIW 9600 XT (soon to be replaced?)
integrated LSI scsi controller (used to have a backup volume (72GB) until the raid...)

Originally, the system was my desktop which explains the ATI AIW card. I used to dual boot BSD and Windows XP on the system. I'm in the process of setting up a backup "server" for data which should be live in the next week. I'm waiting for a new heatsync/fan for the CPU.

The CVS repository is backed up weekly to a system offsite.

devel/cvsps
devel/cvsps-devel
x11/fast-user-switch-applet
x11-themes/gnome-backgrounds
textproc/rarian
misc/getopt
accessibility/at-poke

Normally, I reserve this blog for MidnightBSD status updates and information about the project. Today, someone gave me this link in IRC to a conversation regarding comments about "free" software in the OpenBSD ports system.

Richard Stallman claims the ports tree in OpenBSD contains non-free software. Ports are just a collection of makefiles and patches. There are a few special ports in FreeBSD and MIdnightBSD which contain BSD licensed code, but that is very rare. In general, the ports system allows you to fetch software and build it yourself. All the BSDs also provide binary packages built from the ports tree, but these comments focused on the ports tree.

I'd also like to point out many linux distros have their own repositories and package distribution systems including gentoo (most iike bsd ports) and sometimes third party repositories for extra packages such as Debian and Fedora users have setup.

It is also important to consider the definition of free each side is using in the conversation. Free software has a different meaning to a BSD user.

http://marc.info/?l=openbsd-misc&m=119750352332512&w=2

While I haven't been updating on all of our ports activity, I thought I'd share recent progress with GNUstep and Etoile ports.  A change in current to bring in pcc has caused a problem with any objective-c port.  Specifically the inclusion of libgcc_s in the base is conflicting with the port versions.  This only happens on current systems and only during the last few months.  (say september or so).  The changes *should* be backed out, however you may have a libgcc_s in /usr/lib causing problems.  If you update to the latest snap and then remove that file, things should work as expected.

Last night I updated gnustep-make to 2.0.2 and gnustep base.  I was able to build gorm and ProjectCenter successfully on these new versions.  Today I'm working on fixing several Etoile ports that have not weathered the recent change in naming of shared libraries as well as improvements to bsd.gnustep.mk and bsd.mport.mk.  In short, I'm making the ports work again.

For variety sake, I've recently imported a good share of gnome into the system.  We're not switching from our goal, but wanted to diversify the ports system.  A few problems with GNUstep + the gcc changes + updating x.org has caused us headaches and some developers and users wanted a backup. 

GNUstep has been working on 0.1-RELEASE (plus mports update) for some time.  We are currently building gcc 4.1 as a dependancy as it seems to be the most recent, reliable compiler for GNUstep on MidnightBSD. 

Etoile-iconkit was fixed just moments ago. 

The current recommended configuration for working with GNUstep on MidnightBSD is
0.1.1-RELEASE
gcc 4.1 (default port)
gnustep (as of last night) 
Etoile (if you want to play with it)
Gorm + project center or project manager
whatever apps you like.

We're still running the AMD64 build.  With one machine it is not as fast as it could be.  The real issue is that magus does not have support for 2 core systems. We can't manually exec a second instance as they will clash.  We're looking into ways to allow for concurrency on powerful systems . This was not a design consideration since we were working with old intel boxes and thinking about old single cpu suns. 

We've had a few bugs with this run.  Specifically the ssh tunnel is dying once in a while.  I modified the script to start the tunnel to sleep 5 seconds and then try to open a fresh tunnel.  If it happens at the right time, we're fine.  Since making that change, it's only occured once where the tunnel dropped. 

We may have to move the master elsewhere to avoid this in the future.

One port (pear) has continuously failed and should be fixed.  It attempts to ask for a file to patch :)

Overall, considering magus is still "alpha" software I'd say it's working quite well.  We've been able to fix a large number of ports we didnt' know where broken or unfetchable.  I'll admit that it's been distracting though.  We haven't done much outside of ports in the last month or so. 

The drivers for ati, nvidia, and intel graphics adapters have been updated to the newest available versions.  I've also updated the joystick driver to 1.3.0. 

There was a power outage that lasted more than 4 hours taking down the MidnightBSD website, CVS, etc.  Service has been restored.

For the first time, we're running a second architecture on the build cluster. I'm testing CURRENT on amd64 against the most recent mports snap used by the i386 run. This setup is a bit odd as the only machine is at home.

The current build cluster is housed at Eastern Michigan University in one of the Computer Science department labs. Since the master is also there for performance reasons, and behind a firewall, I've setup a ssh tunnel to proxy the mysql connections to the master. (it's actually 2 tunnels) I'm a bit concerned about one of the tunnels dropping, but it's still an interesting experiment. The packages are getting stored locally instead of going to the master as usual.

We did manage to get through most of archivers so far. Even though this machine is the only node building, it's also a Pentium D 805 with 2.5GB of RAM and SATA instead of a P4 1.4Gzh with 512MB and IDE.

We're happy to announce the release of MidnightBSD 0.1.1. The software is available on our FTP server and will be on the mirrors shortly.

The packages for 0.1.1 are on the FTP server, but the Latest and category symlinks have not been put in place on the FTP server yet. You can manually download packages to use with pkg_add or give it an exact url to the file with pkg_add -r We're planning on fixing this in a few days. These packages were generated on our new mports build cluster called magus. There were near 1000 packages generated for this release on i386.

I'm planning on making an amd64 release, but have not started that build yet. It will probably be another week before that is done as I have to build packages too.

What's new:

Updated to bzip2 1.04

mports collection is included with the release. This greatly simplifies getting additional software for MidnightBSD.

OpenSSL security issue corrected.

libpthread output intended as an error is written to stderr instead of stdout.

find has new "birth" time options.

pkg_install tools like pkg_add know about mports.


Late breaking information:

This release includes x.org 6.9. It will be the last release with x.org 6.9. Users who want to use x.org 7.3 should update their mports tree and uninstall all ports depending on X11. It is recommended that you delete /usr/X11R6 and create a symlink to /usr/local for /usr/X11R6. This will allow all the ports to work. The updated ports tree has many new software packages including most of Gnome 2.18.3. Since this mports tree was created (0.1.1's), over 300 ports were added to MidnightBSD. 1500 ports are building correctly on our build cluster.

Special Thanks:

Eastern Michigan University's Computer Science Department donated the computing resources for the magus build cluster. This allowed us to build packages in 10 hours for this release.

The next release:

0.2 will be the next release barring any need for a later 0.1.x release. It will include replacements for pkg_add and a new installer option. Expect it in 2008.