The Design and Implementation of laffer1

One of the problems on the web is all the different character encodings.  Computers represent information in different ways.  Some of these approaches handle multiple languages, others do not.  One such encoding is UTF-8.  You can test for UTF-8 in your web applications using this regular expression:

http://www.w3.org/International/questions/qa-forms-utf-8

 

$field =~
  m/A(
     [x09x0Ax0Dx20-x7E]            # ASCII
   | [xC2-xDF][x80-xBF]             # non-overlong 2-byte
   |  xE0[xA0-xBF][x80-xBF]        # excluding overlongs
   | [xE1-xECxEExEF][x80-xBF]{2}  # straight 3-byte
   |  xED[x80-x9F][x80-xBF]        # excluding surrogates
   |  xF0[x90-xBF][x80-xBF]{2}     # planes 1-3
   | [xF1-xF3][x80-xBF]{3}          # planes 4-15
   |  xF4[x80-x8F][x80-xBF]{2}     # plane 16
  )*z/x;

http://www.phpwact.org/php/i18n/charsets

The latest in zend bugs.. strlen reports 27 characters for a 10 length string.

http://ha.ckers.org/xss.html

I've been working on a new web application to download and mix rss feeds.  The idea is to cache feed items into a database and allow mixing of content from many sources.  I have about 1300 rss feeds so far on a variety of topics.  It's rather interesting to do like queries on the tables and find all sort of information.  

My long term plan is to have it index all public just journal entries, and content from other sites like twitter.   (just popular stuff)

There are services like this already, but many of them charge a lot of money for the information.  It's not like a typical search engine because one wants to access the information repeatedly and google tends to block that.  It's an interesting problem we stumbled onto at work.  Most likely content will go pay down the road, but I think there will still be free content as well.  

The first version of the rss fetcher is complete.  It's populated about 444,000 articles so far (rss items), 56000 categories (per rss 2 spec), and 55000 enclosures (file attachments like podcasts and images).  

One service actually charges 75,000 dollars for this functionality. They have a lot more content than I do, but most of it is garbage from twitter.  If you do a keyword search for say clorox, it ends up with posts about bleaching blood out, throwing it on people, cleaning tips and other crazy things.  There's obviously a need for good filters and smart content searching.  I only know of one method to do this right now and I'm not going to buy IBM Omnifind :) 

This is one of the many problem domains I deal with at work.  Thing is, they have no interest in getting the content themselves whereas I see a lot of potential in it.  This doesn't really overlap with work but it's related to what I do right now.  Scary isn't it? 

The biggest hurdles are:

1. Bandwidth.  Downloading RSS feeds takes a long time.

2. Storage capacity.  I'm not sure how long I can retain content. 

3. Blocks.  I might get black listed harvesting so care has to be taken in fetching content.  The java libraries I'm using right now don't honor rss feed intervals, but I'm limiting to at least 60 minutes for now.

4. Legal.  I'm not planning on charging for this data now and effectively i'm acting like a search engine.  I spider content, collect it and cache it for a period of time.  The only difference is how one accesses it.

The next step in the project is creating the website.  I've got help with that phase.

A friend of mine has been working on merging useful patches from Debian and MirBSD for CVS into a new port for mirports.  This is interesting and we might look at it soon.  It adds a number of useful additions to CVS.

MidnightBSD CVS is based on a patchset from DragonFly BSD. It's newer than FreeBSD's archaic CVS version. Newer versions have some issues with CVSWeb, but ViewVC works fine. 

This week I reorganized part of the server room.  We had one 42u IBM rack that my old boss purchased at surplus (U of M) for $50.  It's never been great and he paid to have sides put on it which look quite tacky.  

Last time the CEO was in town, he freaked about how ugly the room was.  He wants to impress clients with this storage room that happens to have servers in it.  It's supposed to be like the "water cooler" for the office.  I don't see why anyone would want to talk in a room with loud computers in it.  

I ended up purchasing two enclosures from Apple. The seal around the front doors sticked, but other than that they were nice.  I used similar units at EMU.  It does cut down the noise and it's better looking than the IBM rack.  I've moved all the external servers (client facing) to the first rack.  The second rack is a bit of a problem.  We have to migrate wiring for all the phone and network ports into the back of it which means redoing all the patch panels.  I figure it's at least an eight hour job and I know he'll hate the room next time he's there anyway.  

Of course, I wonder why I get into these things when I'm supposed to be a software engineer.  Last I knew, that meant I write programs for a living.  

To top it off, one of the guys in germany is working on a stupid idea to put articles in CouchDB and then display them in our portals.  Problem is we import the articles from Joomla into another system and those will not be available for newsletters.  I just wish people would think... 

Today, I got an ad from my antivirus software for a useless utility product.  I hate ads in software I have purchased.  I can deal with it in games when it's product placement to a degree to cover costs like servers for multiplayer.  However, when I bought antivirus software from a SECURITY company, I expect it to be free of bullshit.  

My product expires in October so I'm going to try out the free Microsoft security essentials software then instead of buying an AV program.  I haven't found one virus the entire year which means either I'm at a low thread level or the software didn't work.  Either way, I don't see the point of spending $50 to protect our 3 inferior Windows boxes. 

Intel has a new product line that includes a $50 software upgrade to unlock all the features of your CPU.  This unlock only works in Windows 7.  If you're running any other operating system, it will not function.  This sucks if you're using Linux, *BSD, Solaris, Haiku, EComstation, or even older versions of WIndows.

  http://hardware.slashdot.org/story/10/09/19/0050203/Intel-Wants-To-Charge-50-To-Unlock-Your-CPUs-Full-Capabilities?from=rss

I hope this doesn't happen across their product line because it will mean I'm limited to AMD CPUs.  Maybe my next laptop will have an AMD chip in it. 

I've previously created a wiki document about the process to build a MidnightBSD release, but I haven't talked much about what goes on in version control.  

MidnightBSD uses CVS for our source repository.  Today, I created a new tag RELENG_0_3_BP.  This is the branch point, or start of the 0.3 branch that will become 0.3-RELEASE.  

One creates a tag like this:

/usr/src
# 
cvs update -P -d

The next step is to create a branch point  tag , so that diffs against the start of the branch are easier with CVS:

/usr/src
# 
cvs rtag RELENG_0_3_BP src

And then a new branch tag is created with:

/usr/src
# 
cvs rtag -b -rRELENG_0_3_BP RELENG_0_3 src

 

 

http://scoreboard.keynote.com/scoreboard/Main.aspx?Destination=NTT

The current latency for tier 1 providers on the Internet.

Well i decided today was make-up day for losing a good part of my sunday to work.  I just bought the QuakeCon2010 pack on steam.  It includes fallout 3, quake *, Doom *, Hexen *,  RTCW, Spear of destiny, elder scrolls 3 & 4, heretic, commander keen and call of cthulhu: dark corners of the earth.  I figured that fallout 3 is close to the same price, and i got a lot of old games.  

I got a magus run started on ds9 and i'm answering questions via jabber, otherwise i'm just relaxing today. 

I haven't written about LDAP for a few years.  I've been asked to do a samba setup at work with LDAP.  I decided to document some of the steps required on FreeBSD.

Install samba port (i used 3.4.x)

install smbldap-tools (command line tools to help with user management)

install nss_ldap and pam_ldap ports.  Make sure you can resolve accounts from ldap using id and getent group or else samba will not authenticate.

Configure /usr/local/etc/ldap.conf (and make sure it's linked to nss_ldap.conf)

Modify /etc/nsswtich.conf ; on freebsd 8 remove the compat entries and switch it to files ldap for passwd, group and shadow.  When the compat keyword is used, nothing else works according to nsswitch.conf(5). 

Setup your LDAP directory.  If you configure the smbldap-tools properly, you can use the smbldap-populate command to create all the required parts of the ldap database.

Configure your shares and other settings in smb.conf (although part of this must be done before smbldap-tools works right)

optionally setup swat out of inetd via /etc/inetd.conf

This is by far not complete and should not be considered a "secure" setup but it does seem to work so far. 

In smb.conf you would have something like:

passdb backend = ldapsam:"ldap://ipaddresshere ldap://secondipaddress"
 

ldap user suffix = ou=people
ldap machine suffix = ou=Computers
ldap group suffix = ou=Groups
 

 

 

Yuck.. linux server ran out of disk space. installing new raid volumes now

status of raid 1:

mdadm --misc -D /dev/md3

installation howto from ubuntu:

https://help.ubuntu.com/community/Installation/RAID1%2BLVM

useful site on lvm:

http://www.linuxconfig.org/Linux_lvm_-_Logical_Volume_Manager

yum install cyrus-sasl-plain  is your friend.  THis fixes the mail vrfy blah blah error in the logs when you setup sendmail + saslauthd and you try to get thunderbird to auth properly. 

This bugged me for two days and there are hundreds of incorrect posts about using RELAY rules.  It's insane.

 did not issue MAIL/EXPN/VRFY/ETRN during connection to TLSMTA  fix is again to install the right cyrus sasl auth plugin from yum. 
 

I finally got a chance to beat Half Life 2 Episode 2.  I've been half way through it since last year.  

One of my coworkers pointed out the inherit insecurity of WPA on Wifi devices the other day.  I knew that WEP was insecure, and I figured there were problems with WPA, but I didn't realize how easy it is to break it. 

WPA (and WPA2 -PSK / personal in apple lingo) use a secret key that can be up to 63 characters.  Many people don't use a full length key.  Others use a dictionary word or some other easy to deduce key.  

Apparently, in the protocol for WPA, the initial handshake (communication between your pc/mac and the wifi router) transmits something (well the hash of the key effectively) that can be grabbed in the air.  An attacker can grab this and then run a program offline to determine your WPA key.  He can later join your network.  

At this point, assume all wireless networking is insecure if you weren't already.  

Tips to make it harder to crack:

1. Use the maximum length or as close as you can for the private key

2. Use random characters, a generated key from one of the many online sites is better.  Don't use something obvious like "luke & caryn's wireless".   No, i've never used that.

3. Enable MAC address filtering (not to be confused with Mac computers)

4. Change your key periodically.  You never know who's on your Internet connection.

5. Use AES with WPA if it's available on your router. It's a little harder to crack.

Reasons you should care:

1. Someone can download illegal or offensive content and it came from your ip address.  You could go to jail or have an unpleasant search situation.  

2. You're paying good money for your internet connection, why should your neighbors freeload? Do you let your neighbor steal your cable too?

3. War driving.  Just google it.

 " The flaws in Excel allowed hackers to create tainted spreadsheets that infect a user's PC once they are opened, ..."

http://www.msnbc.msn.com/id/37579736/ns/technology_and_science-security/

This has got to be the dumbest security message I've seen to date.

(shamelessly stolen from http://apple.slashdot.org/comments.pl?sid=1660824&cid=32302054) 

Are you a programmer and willing to shoot only an appendage?

C
You shoot yourself in the foot.

C++
You accidentally create a dozen clones of yourself and shoot them all in the foot. Providing emergency medical assistance is impossible since you can't tell which are bitwise copies and which are just pointing at others and saying, "That's me, over there."

JAVA
After importing java.awt.right.foot.* and java.awt.gun.right.hand.*, and writing the classes and methods of those classes needed, you've forgotten what the hell you're doing.

Ruby
Your foot is ready to be shot in roughly five minutes, but you just can't find anywhere to shoot it.

PHP
You shoot yourself in the foot with a gun made with pieces from 300 other guns.

ASP.NET
Find a gun, it falls apart. Put it back together, it falls apart again. You try using the   .GUN Framework, it falls apart. You stab yourself in the foot instead.

SQL
SELECT @ammo:=bullet FROM gun WHERE trigger = 'PULLED';
INSERT INTO leg (foot) VALUES (@ammo);

Perl
You shoot yourself in the foot, but nobody can understand how you did it. Six months later, neither can you.

Javascript
You've perfected a robust, rich user experience for shooting yourself in the foot. You then find that bullets are disabled on your gun.

CSS
You shoot your right foot with one hand, then switch hands to shoot your left foot but you realize that the gun has turned into a banana.

FORTRAN
You shoot yourself in each toe, iteratively, until you run out of toes, then you read in the next foot and repeat. If you run out of bullets, you continue anyway because you have no exception-handling ability.

COBOL
Using a COLT 45 HANDGUN, AIM gun at LEG.FOOT, THEN place ARM.HAND.FINGER. on HANDGUN.TRIGGER and SQUEEZE. THEN return HANDGUN to HOLSTER. CHECK whether shoelace needs to be retied.

LISP
You shoot yourself in the appendage which holds the gun with which
you shoot yourself in the appendage which holds the gun with which
you shoot yourself in the appendage which holds the gun with which
you shoot yourself in the appendage which holds the gun with which
you shoot yourself in the appendage which holds   ....

BASIC
Shoot yourself in the foot with a water pistol. On big systems, continue until entire lower body is waterlogged.

Pascal
The compiler won't let you shoot yourself in the foot.

Unix
% ls
foot.c foot.h foot.o toe.c toe.o
% rm *   .o
rm:   .o: No such file or directory
% ls
%

Visual Basic
You'll shoot yourself in the foot, but you'll have so much fun doing it that you won't care.

Ada
After correctly packaging your foot, you attempt to concurrently load the gun, pull the trigger, scream and shoot yourself in the foot. When you try, however, you discover that your foot is of the wrong type.

Assembly
You try to shoot yourself in the foot only to discover you must first reinvent the gun, the bullet, and your foot. After that's done, you pull the trigger, the gun beeps several times, then crashes.

Python
You try to shoot yourself in the foot but you just keep hitting the whitespace between your toes.

Etc...

  /**

()