MidnightBSD Developer Journal

Yesterday, I committed some new files in src/nrelease in CURRENT. cvs update -d your source tree, then do a make buildworld, buildkernel, installkernel, reboot installworld, mergemaster process. This approach works if you're tracking CURRENT.

Now, go into /usr/src/nrelease. Run make buildiso and make mkiso. This will create a new directory /usr/release, install the buildworld into it and then create an ISO.

If you wish to customize the ISO, go into src/nrelease/root. You can add files and tailer files for the CD there.

I've posted two Live CDs on the FTP server. Both are based on MidnightBSD 0.2 CURRENT. You can download a Live CD for i386 or AMD64. The current approach uses acd0 for booting so most elements are mounted read-only. These Live CDs are not intended for installation, although it might be possible to fdisk/bsdlabel the system and copy the contents of the CD with a few changes to /etc/fstab.

Long term I'm hoping to get this working with X11 + GNUstep and create an installer for MidnightBSD.

This is based on the process used to create DragonFly ISOs.

(amd64) http://www.midnightbsd.org/ftp/MidnightBSD/snapshots/amd64/0.2-070720-LIVECD/mbsd.iso

(i386) http://www.midnightbsd.org/ftp/MidnightBSD/snapshots/i386/0.2-070721-LIVECD/mbsd.iso

A buffer overflow issue was discovered in tcpdump which is shipped with MidnightBSD. A patch has been committed to CURRENT (0.2)

http://secunia.com/advisories/26135/

http://www.tcpdump.org/

Some people wonder about MidnightBSD commit activity. Now there is an easy way to follow the action.

http://cia.vc/stats/project/midnightbsd

You can also view commits from many different projects on #commits on freenode.

I just set this up a few minutes ago.

There was a new version of firefox released (2.0.0.5).  I've updated our linux version.  I also checked on some of our other browser ports.  links1 was about 5 years out of date.  links and lynx also had newer versions.

Unrelated, I also fixed a bug that was causing current to fail to build.  I'm testing some changes to cpio to fix the make release problems.  I have a few more options if that is unsuccessful. 

The 0.2 snap does not work.  There seems to be a problem with cpio
/stand/cpio malformed number...

ugh.  Well I'll be looking at this problem later. 

Update: it does appear to be cpio related. http://www.gnu.org/software/cpio/

I've created an i386 0.2 snap without ports or packages.  The snap has yet to be tested, although I'm preparing to do so.  Presuming this snap works as expected, it should be much more compatible with the changes to the mports system. 

This snap includes userland updates to openssh and cvs as well as bug fixes and improvements.  There is support for additional sound hardware.  You can get a feel for ctriv and wintellect's work on mports. 

I've had a lot of questions about MidnightBSD's future plans.  We have not provided a comprehensive roadmap or even a weak one in some time.  Here's a brief summary of our current situation and where we are headed plus some of our possible plans.

First, we have nearly completed 0.1 for release.  I am building packages for 0.1 on i386 right now.  There are several problems with mports exposed in the builds for 0.1.  We should have work arounds before the release.  0.1 will not include a new installer, but some of the mports changes will be included. 

Security patches present in 0.1 and 0.2 for everything known except the recent theoretical scheduler attack that effects practically every OS except Mac OS X.  I don't see us moving off of a tick based kernel anytime soon.  It's in the back of my mind though.

Most of the changes in 0.2 so far are in the mports infrastructure or userland.  There were a few subtle updates to drivers including firewire.  I expect more driver work in the next month.  No one has reported any hardware that doesn't work in MBSD but does in other systems barring the limits of Xorg 6.9.

One of our developers has been working on migrating to Xorg 7.2.  I don't want to commit this change to a specific release, but it will happen before 1.0.  We will also include an x11 based installer, current GNUstep environment, and a GUI mports management system for 1.0 release.

0.1 release will not be an ideal desktop system.  It is a new project working out release engineering, testing, and proving the changes we've made from FreeBSD are correct.  It should be fairly stable, but not feature complete as a desktop.  (think CLI)  We are toying with a way to install GUI packages as a hack until the new installer is done. 

0.2 release will include the critical ports of the mports tree.  Depending on ctriv and wintellect's progress, we hope to ship command line tools and possibly a GUI if caryn and alex get that part done.  If not, we will be very close to done with major mports work.  We also plan on having a new installer for desktop installation, plus any userland + kernel work during the development cycle.  (cvs, ssh, and quite a few other things have been updated in current already)

0.3 should complete the mports and installer work based on feedback from 0.2 and anything that was not completed during the previous release.  More kernel and userland improvements

0.4 will focus on the desktop experience.  We plan to get documentation in order,  determine the final packages for 1.0 release, and work on integration.  This will be point we make the judgement call on Etoile.  Prior to this release, we plan on using WindowMaker and slim.  We could revisit Etoile later, but not for 1.0.  (more userland + kernel stuff)

0.5 will be the beginning of the server release.  The desktop version of MidnightBSD is the focus of the project, but we've decided to ship a server installer with separate ISOs.  People in the project use MBSD as a server and it would be helpful for us to have it.  The installer will be a bit different for this release as we plan on allowing users to select packages (think httpd, mail, ftp, etc)  as well as add more control over partitioning and things required in a server OS.  I don't plan on making the server a priority as FreeBSD is quite capable as a server as is all of the mainstream BSD projects.  (Free, Net, Open, DF)    You could compare this to Mac OS X server versus Mac OS X client.  They are the same but the server had additional tools and services.  Die hard security fans wouldn't run a GUI on a server so we don't consider this to be a mission critical type of thing.  It's for a computer lab or small business environment where the other BSDs fail to target and Linux or windows get used. 

... (whatever else we need to do before release)

1.0 Release
Full desktop environment with installer, GUI package management, command line environment and package management, documentation, client/server, and GNUstep integration with many GNUstep based applications.  There will be a web browser, some office productivity software, etc.  (mozilla based, open office?)  We'll also have a server version. 

That covers what we'll be doing with basic desktop stuff which is what most people want to know.  However, there are a lot of things we'd like to see under the hood too. These are projects with which we have no specific timetable but want to see added.

1. Enhanced disk encryption for portions or the entire disk.  (subset might be like an OS X home directory with encryption)

2. Something similar to OS X style dmg files.  A transparent disk image system.  There are many things we could use to implement this. 

3. An alternative scheduler more suited to desktop SMP usage.  Multicore is here to stay.  We need to grow up to it. 

4. Disk schedulers.  Disk IO is notoriously slow in FreeBSD and MidnightBSD.  There are a number of factors including locks, scheduling, poor UFS2 performance, etc. 

5. Enhanced wifi configuration and support.  Wifi should be much more transparent than trying to enable wpa_supplicant in rc.conf and hacking out a config file for it.  The network stack is not optimized for wifi or the newer FIOS/cable modem packages available in some parts of the world.  This means we need a self tuning stack to handle both extreme cases.

6. Support for Intel Macs.  This is a combination of power management, fans control, drivers, and either Mac EFI 1.x support or a hack around it.  (i.e. use the emulation)  MBSD does not boot on a Mac Pro due to the keyboard probe timeout and other issues.

7. Cleanup of the threading libraries.  The FreeBSD project has changed to libthr from KSE in current.  We need to evaluate what is best for us and pick a final default and make sure it is standards compliant.  Changing threading libraries every other release is very painful with ports and third party software development for your OS. 

8. Integration of BSD licensed replacements for GNU licensed tools.  Unlike some projects, we are much more content with other licenses, but for flexibility it is preferred to use BSD licensed software in the system.  As much as possible we'd like to keep everything below the GUI layer a BSD licensed app.  I do expect to use a lot of GPL/LGPL code during the course of the project as well.  Both licenses serve different needs.  This also includes importing code from OpenBSD, NetBSD and to a lesser degree DragonFly.  Those projects have more actively targeted userland improvements.  We also hope to develop many of our own.

9. Security.  We have a great deal of plans to improve security including a default firewall, improvements to the existing firewalls.  (pf upgrade, changes to ipfw)  I've mentioned the disk encryption and we hope to look at other sections of the code for enhancement.  Switching to gcc 4.x would also be helpful on this path.

I'll write more on this later.

We inherited libarchive from the FreeBSD project. A security issue effects both FreeBSD and MidnightBSD. We have applied the patch to CURRENT and the 0.1 branch.

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

=============================================================================
FreeBSD-SA-07:05.libarchive Security Advisory
The FreeBSD Project

Topic: Errors handling corrupt tar files in libarchive(3)

Category: core
Module: libarchive
Announced: 2007-07-12
Credits: CPNI, CERT-FI, Tim Kientzle, Colin Percival
Affects: FreeBSD 5.3 and later.
Corrected: 2007-07-12 15:00:44 UTC (RELENG_6, 6.2-STABLE)
2007-07-12 15:01:14 UTC (RELENG_6_2, 6.2-RELEASE-p6)
2007-07-12 15:01:32 UTC (RELENG_6_1, 6.1-RELEASE-p18)
2007-07-12 15:01:42 UTC (RELENG_5, 5.5-STABLE)
2007-07-12 15:01:56 UTC (RELENG_5_5, 5.5-RELEASE-p14)
CVE Name: CVE-2007-3641, CVE-2007-3644, CVE-2007-3645

For general information regarding FreeBSD Security Advisories,
including descriptions of the fields above, security branches, and the
following sections, please visit .

I. Background

The libarchive library provides a flexible interface for reading and
writing streaming archive files such as tar and cpio, and has been the
basis for FreeBSD's implementation of the tar(1) utility since FreeBSD 5.3.

II. Problem Description

Several problems have been found in the code used to parse the tar and
pax interchange formats. These include entering an infinite loop if an
archive prematurely ends within a pax extension header or if certain
types of corruption occur in pax extension headers [CVE-2007-3644];
dereferencing a NULL pointer if an archive prematurely ends within a
tar header immediately following a pax extension header or if certain
other types of corruption occur in pax extension headers [CVE-2007-3645];
and miscomputing the length of a buffer resulting in a buffer overflow
if yet another type of corruption occurs in a pax extension header
[CVE-2007-3641].

III. Impact

An attacker who can cause a corrupt archive of his choice to be parsed
by libarchive, including by having "tar -x" (extract) or "tar -t" (list
entries) run on it, can cause libarchive to enter an infinite loop, to
core dump, or possibly to execute arbitrary code provided by the
attacker.

IV. Workaround

No workaround is available, but systems which do not read tar or pax
extension archives provided by untrusted sources are not vulnerable.
Note that while these issues do not affect libarchive's ability to
parse cpio, ISO9660, or zip format archives, libarchive automatically
detects the format of an archive, so external metadata (e.g., a file
name) is not sufficient to ensure that a file will not be parsed using
the vulnerable tar/pax format parser.

V. Solution

Perform one of the following:

1) Upgrade your vulnerable system to 5-STABLE, or 6-STABLE, or to the
RELENG_6_2, RELENG_6_1, or RELENG_5_5 security branch dated after the
correction date.

2) To patch your present system:

The following patches have been verified to apply to FreeBSD 5.5, 6.1,
and 6.2 systems.

a) Download the relevant patch from the location below, and verify the
detached PGP signature using your PGP utility.

# fetch http://security.FreeBSD.org/patches/SA-07:05/libarchive.patch
# fetch http://security.FreeBSD.org/patches/SA-07:05/libarchive.patch.asc

b) Execute the following commands as root:

# cd /usr/src
# patch < /path/to/patch
# cd /usr/src/lib/libarchive
# make obj && make depend && make && make install
# cd /usr/src/rescue
# make obj && make depend && make && make install

NOTE: On the amd64 platform, the above procedure will not update the
lib32 (i386 compatibility) libraries. On amd64 systems where the i386
compatibility libraries are used, the operating system should instead
be recompiled as described in


VI. Correction details

The following list contains the revision numbers of each file that was
corrected in FreeBSD.

Branch Revision
Path
- -------------------------------------------------------------------------
RELENG_5
src/lib/libarchive/archive_read_support_format_tar.c 1.26.2.8
RELENG_5_5
src/UPDATING 1.342.2.35.2.14
src/sys/conf/newvers.sh 1.62.2.21.2.16
src/lib/libarchive/archive_read_support_format_tar.c 1.26.2.7.2.1
RELENG_6
src/lib/libarchive/archive_read_support_format_tar.c 1.32.2.5
RELENG_6_2
src/UPDATING 1.416.2.29.2.9
src/sys/conf/newvers.sh 1.69.2.13.2.9
src/lib/libarchive/archive_read_support_format_tar.c 1.32.2.2.2.1
RELENG_6_1
src/UPDATING 1.416.2.22.2.20
src/sys/conf/newvers.sh 1.69.2.11.2.20
src/lib/libarchive/archive_read_support_format_tar.c 1.32.6.1
- -------------------------------------------------------------------------

VII. References

http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2007-3641
http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2007-3644
http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2007-3645

The latest revision of this advisory is available at
http://security.FreeBSD.org/advisories/FreeBSD-SA-07:05.libarchive.asc
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (FreeBSD)

iD4DBQFGlkN5FdaIBMps37IRAl/vAJ4vKkZ9eXBW4PPljvbgALUlAPdxCQCXRMzY
4hKO09Xhj1akwPufFXJS2w==
=sRGA
-----END PGP SIGNATURE-----

I've added three pictures to http://www.midnightbsd.org/ The pictures show MidnightBSD running on my ThinkPad.

We just gained a new committer. Alex is interested in GUI related projects and has interest in the installer and GUI for the mports system.

I checked in a new sound rc.d script that loads the appropriate kernel module for specific sound cards. Not all the PCI IDs are in the associated file. This is based on the FreeSBIE/PC-BSD/DesktopBSD scripts.

If you find a missing PCI ID, let me know. You can disable the script and then load all the sound drivers using loader.conf or try loading the module for your specific card.

So far I've tested this on my laptop with great success. After a little testing, this will be included with 0.1 as well.

Some bugs with the xine and libxine ports were fixed. If you had trouble building libxine before, try now. Let me know if there are any errors.

GNU m4 in devel/m4 has been updated to 1.4.9 from 1.4.4.

GNUstep ports seem to build on my laptop now. I'm having some issues running the TextEdit port, but I'm thinking it's a local issue with my laptop setup. I'm building a fresh system in virtual pc on my windows box to test.

The php5-sqlite port was fixed. It was failing after the PHP 5 update.

The PHP5 port was updated to 5.2.3. The new version includes several security patches.

Fixed an integer overflow inside chunk_split() (by Gerhard Wagner, CVE-2007-2872)
Fixed possible infinite loop in imagecreatefrompng. (by Xavier Roche, CVE-2007-2756)
Fixed ext/filter Email Validation Vulnerability (MOPB-45 by Stefan Esser, CVE-2007-1900)
Fixed bug #41492 (open_basedir/safe_mode bypass inside realpath()) (by bugs dot php dot net at chsc dot dk)
Improved fix for CVE-2007-1887 to work with non-bundled sqlite2 lib.
Added mysql_set_charset() to allow runtime altering of connection encoding.

A recent update to libdvdread broke the mport on i386.  The optimized byte swap patch was removed.  The port now works as intended.

http://gplv3.fsf.org/

FAQ:
http://www.gnu.org/licenses/gpl-faq.html

Slashdot article:
http://linux.slashdot.org/linux/07/06/29/172228.shtml

The most recent 0.1 snap from May has been reported not to work on several systems.  We are investigating the problem.  Please use the March snapshot for now. 

triv has committed several changes to mports to correct recent problems with GNUstep ports. 

I've had some reports of problems with the captcha on the account registration page. I've disabled it for now. If you were having problems registering, please try again.

There seems to be some issues with the GNUstep ports.  I'm discussing some changes with mports that might have caused the problem with another developer.  We hope to get these working soon.