MidnightBSD Developer Journal

There has been some concern raised about the possibility of a backdoor in the IPSEC implementation of OpenBSD.  As OpenBSD's implementation was the basis of FreeBSD's IPSEC code and we use the same code,  I want to make others aware of this issue.

http://marc.info/?l=openbsd-tech&m=129236621626462&w=2

 I have not audited this code so I don't know if this is true, but I find it unlikely.  I will defer to others in the BSD community to audit the code in OpenBSD before taking any actions.  

I have never been approached about adding a backdoor into MidnightBSD. To the best of my knowledge, none exists.

I'm attempting to get a WD EARS drive working properly in one of my servers.  There are a number of issues using one of these new advanced format drives.

1. The default alignment causes significant performance problems.  Some users have reported 1-8Mbps read speeds.

2. sysinstall doesn't let you change the offset in bsdlabel in a convenient way.

3. Many people have problems with these drives, but few people have good solutions under BSD.

Things I've found out so far:

An offset of 1 is helpful (so it's on 64 instead of 63 for the first partition).  Most people trying to use these things are going for one large drive for data.  I need to boot off this thing which means bsdlabel has several entries.  I haven't found anyone trying this yet.  

First, I thought I'd get clever and try gpart/gpt.  I actually got the drive setup and some testing showed decent performance but I forgot that I hadn't ported the boot code yet.  Doh!  

Next, I went to plan b.  I used fdisk as normal and instead focused on making appropriate changes in bsdlabel.  WD says that as long as sectors are divisible by 8 you are ok, but an interesting analysis showed performance improvements by using a block size o 32768, sector size of 4096 (with newfs).  That meant I had to be a little more careful during bsdlabel so that everything lined up nicely.  

DES looked into this for FreeBSD and wrote a handy utility to test called phybs.  It doesn't do performance testing, but you can see the affect on alignment.  

Using that utility, I found that the fdisk setup was slower than gpt for some reason and i suspect things are still not optimal.  However, a quick and dirty test of moving some files around showed it was running better than the horror stories I've read.

One test of copying files from a 7200 RPM seagate HDD to two different green drives (a samsung and the WD) showed that the samsung drive was slightly faster (1MBps).  diskinfo shows the WD drive faster on the inner part of the disk but slightly slower on the outer part.  

I'll post real numbers up later.. it's 3 am.  

here's the results under GPT

./phybs -r /dev/ad8p1

   count    size  offset    step        msec     tps    kBps

  131072    1024       0    4096       18198    7202    7202

  131072    1024     512    4096       18026    7271    7271

   65536    2048       0    8192       10233    6404   12808

   65536    2048     512    8192       11135    5885   11770

   65536    2048    1024    8192       11304    5797   11594

   32768    4096       0   16384        7508    4364   17456

   32768    4096     512   16384        8394    3903   15613

   32768    4096    1024   16384        8789    3728   14913

   32768    4096    2048   16384        8458    3873   15495

   16384    8192       0   32768        5672    2888   23107

   16384    8192     512   32768        5723    2862   22900

   16384    8192    1024   32768        5999    2730   21846

   16384    8192    2048   32768        5867    2792   22337

   16384    8192    4096   32768        5735    2856   22852

# gpart show

=>        34  1953525101  ad8  GPT  (1000.2GB)

          34        2014       - free -  (1031.2KB)

        2048     2097136    1  freebsd-ufs  (1073.7MB)

     2099184    16572032    2  freebsd-swap  (8.5GB)

    18671216     2097136    3  freebsd-ufs  (1073.7MB)

    20768352   268435456    4  freebsd-ufs  (137.4GB)

   289203808  1664320808    5  freebsd-ufs  (852.1GB)

  1953524616         519       - free -  (265.7KB)

Here's some rsync data for the samsung drive:

sent 4858801691 bytes  received 41302 bytes  28497612.86 bytes/sec

total size is 4858055400  speedup is 1.00

51.771u 35.499s 2:50.08 51.3% 459+1906k 41623+36549io 4pf+0w

rsync for wd:

sent 4858801691 bytes  received 41302 bytes  27685715.06 bytes/sec

total size is 4858055400  speedup is 1.00

55.324u 36.006s 2:54.74 52.2% 457+1899k 41572+36276io 0pf+0w

This is not scientific at all..  i was copying tarballs from the last magus run.

fdisk for the drive:

 fdisk -v ad8

******* Working on device /dev/ad8 *******

parameters extracted from in-core disklabel are:

cylinders=1938021 heads=16 sectors/track=63 (1008 blks/cyl)

Figures below won't work with BIOS for partitions not in cyl 1

parameters to be used for BIOS calculations are:

cylinders=1938021 heads=16 sectors/track=63 (1008 blks/cyl)

Media sector size is 512

Warning: BIOS sector numbering starts with sector 1

Information from DOS bootblock is:

The data for partition 1 is:

sysid 165 (0xa5),(FreeBSD/MidnightBSD/NetBSD/386BSD)

    start 64, size 1953525105 (953869 Meg), flag 80 (active)

beg: cyl 0/ head 1/ sector 2;

end: cyl 613/ head 0/ sector 1

The data for partition 2 is:

<UNUSED>

The data for partition 3 is:

<UNUSED>

The data for partition 4 is:

<UNUSED>

Here is a list of things that I could assistance with:

Get perl to cross compile in the base system.  This would allow me to test via tinderbox on non amd64 architectures.

Integration of mdns responder.  I've got a good start on it, but it needs to be finished.  I've got to focus on other parts right now.

Investigate the newer intel drivers in freebsd for network cards.  em was split into three and we should look at porting those changes back.

Upgrade DRI/DRM support in the kernel to either the version in FreeBSD 7.2 or something newer from the Linux kernel (it's MIT licensed).  This would help  us get some ATI cards working.

Port changes to get newer NVIDIA binary blobs working. 

 x11-toolkits/qt4-gui: 180

 In current, we have a new utility called sysrc.  With sysrc, you can manage entries in /etc/rc.conf.  It will overwrite existing values instead of spamming the file like traditional echo foo >> bar approaches.

Please let me know what you think about this utility.  It won't make it into 0.3-RELEASE because I want to test it and consider possible integration options for mports.

There is a security issue in glob(3) in libc that can lead to remote DOS attacks against ftpd and sftp servers.  This affects many vendors.

I've committed patches to CURRENT and 0.3 branches, but 0.2 has not been patched yet.  I'm still determining what the best approach is there.  If you can't wait,  go to 0.3-PRERELEASE.  

The patch on those branches is based on a patch from DragonFly and NetBSD.  You must build libc as well as sftp and reinstall them then restart ftpd and sftp services. 

I've added a new rc.d script called netwait.  It allows the system to use ping testing to wait for a working network interface on startup.  One problem I've found is that certain network cards change link state several times during system startup.  My desktop's re(4) is notorious for it.  

This script allows remote file systems, system daemons like ntpd, and other services to startup properly with a working network connection.

Additional testing is required with several situations:

1. various no and limited network connectivity scenarios 

2. different network cards

3. ipv6 on/off

The script sits between NETWORKING and mountcritremote in rc order.  

I found the script as part of a FreeBSD PR:

http://www.freebsd.org/cgi/query-pr.cgi?pr=151063

Jeremy Chadwick wrote this for FreeBSD prior to the shuffling of the networking scripts in rc.d in FreeBSD CURRENT.   There are two interesting discussions on the FreeBSD mailing lists about this problem.

The BSD Show has an interview with Caryn Holt about MidnightBSD  (raven@).

I've updated every port in dns today.  I've also added several new ports.

bind97  - version 9.7 of the popular dns server

autotrust - download new dnssec root zones automatically

ldns & unbound: a library containing RFC compliant dns resolver code and a bsd licensed resolver

Users of c-ares or ddclient probably want to update their ports now. 

We've started the process to 0.3.  RELENG_0_3 is now available and has been added to our tinderbox builds.  This will become 0.3-RELEASE.  HEAD is now 0.4-CURRENT. 

If you want to continue to use 0.3, use -rRELENG_0_3 with cvs.

Over the weekend, I connected mDNSResponder in src to the build.  mDNSResponder is apple's implementation of multicast DNS. 

Currently, the mdnsd and dnsextd daemons as well as the command line tool dns-sd are available.  I've also included the libdns_sd library.

FIles are located in src/contrib/mDNSResponder and src/apache/

MidnightBSD 0.3-CURRENT now includes Apple's libdispatch.  With libdispatch, you can create  multithreaded applications with ease.  

The current port does not include blocks support.  We plan to add llvm to the base system in the future.  At that time, we'll setup libdispatch to work with blocks.

We have a new mirror for midnightbsd.  The download page has been updated to reflect the removal of the EMU CS department mirror and the addition of the new mirror.  I also updated most of the metalink files for releases as well.  

Another place to get MidnightBSD :) 

I just posted the first AMD64 snapshot to the FTP server for CURRENT since 0.2.  It does not include mports or packages, but will allow you to install MidnightBSD 0.3 to get up to date hardware support and try out features such as mport tools / libmport, and ZFS.  

This month's BSD Magazine features an article by Lucas and Caryn Holt about MidnightBSD installation.

http://download.bsdmag.org/en/BSD_as_operating_system_BSD_08_2010.pdf 

http://jonripley.com/brainfuck/  and  http://en.wikipedia.org/wiki/Brainfuck

brainfuck(1) has been added to MidnightBSD.  Check it out.  

The devel/llvm and devel/llvm-devel mports have been updated to 2.7 and 2.8.  lang/llvm-gcc4 is now at 2.7 as well.  We've also got a binutils 2.20.1 port now in devel/binutils.