pablotron
--
 
The Coveted Pablotron Gold Star
Thu May 12 19:59:32 2005 :: Link

Ubuntu Linux gets the coveted Pablotron gold star for the week. It's Debian-based, really easy to install, and includes the latest and greatest GNOME. Ubuntu also has fantastic documentation, and it detected most of the hardware on my new laptop correctly (Knoppix didn't).

If Gentoo and Ubuntu are the future of Linux, then I'd say we're in pretty good hands...


Raggle Article, RubyGems Signing, Gah People XML-RPC, and More...
Thu May 12 04:42:59 2005 :: Link

Linux Format, Alonzo's Linux magazine of choice, has a brief article about Raggle in issue 65 (April 2005). They seemed to like it. Guess I'd better fix the UTF-8 munging bug before anyone notices (actually, it's fixed in CVS). I scanned the review — if you're intereste, you can check it out here.

Side projects, side projects. A couple weeks ago, I submitted a patch which adds OpenSSL-based package signing for RubyGems. A patch against RubyGems 0.8.10 is available here (signature). I also wrote up some fairly detailed documentation. It's included in the patch, and also here.

Overall I like the OpenSSL support in Ruby, although I've managed to uncover a couple of gremlins along the way, most notably missing methods in the PKCS12 and ASN1 modules. The single biggest problem though, is the lack of documentation; it's even more sparse than the OpenSSL documentation. I've done enough crypto stuff that I was able to slog through it, but this is just ridiculous. It's 2005. RDoc exists for a reason — use it. I'll resist the urge to comment on the lack of decent RDoc documentation, because I think pragdave has earned a few gold stars. You know, for that whole Pickaxe thing.

The #gah people page now has an XML-RPC interface, which allows you to (say), quickly build a blogroll of IRC people, find people in your state, or whatever other pathological idea you can come up with. Full API documentation (including the XML-RPC endpoint) is available here.

Speaking of web pages, I really really like the final designs on the Ruby-Lang 2005 Redesign Blog. The current ruby-lang.org is kind of an eye-sore, so the sooner they replace it, the better.

On the nifty software front, I've been playing with OpenVPN and Monotone. The former is absolutely incredible; it's easy as piss to configure, completely customizable, fast, uses OpenSSL-based X509 certs for identification (both client and server), and as Windows support. Did I mention it's easy to configure, too? I've been using an OpenVPN to tunnel from my laptop to my file server for the last couple of weeks so I can mount my NFS exports read-write over wireless.

As for Monotone, it looks really promising, but it's unbelievably slow. Actually, "ridiculously, horribly, unbearably slow" might be more apt. I know they're working on speed, so in a couple of months, I'm sure things will be bearable. Once that happens, I may seriously consider switching over...

Oh yeah, Raggle. Work continues on the 0.4 branch. the next stable release (0.4.1) will have the Unicode munging behaving properly (hopefully), and a configurable bookmark system. If I get harassed enough, that might be within the next few weeks :).

As for the development version of Raggle... Well, that's where all the fun is. Here's a high-level diagram of the various components:

Next-Gen Raggle
Engine (Squaggle) Interface
Synapse (libsynapse) SQLite3-Ruby Console Web
libfeed Curl SQLite SQLite Profanity WEBrick
libptime expat       Curses
Legend
  •     Written in Ruby
  •     Written in C
  •     External Dependency

What's new? Profanity has been added to the diagram, and Squaggle has a new dependency, Synapse. Synapse is a C-level library that wraps Curl and libfeed. This arrangement has a whole lot of advantages; here's a handful of them:

  • Speed. C is faster than Ruby, and the Synapse API actually passes commands to a child process, so we also avoid the overhead of Ruby's green threads (and threads in general).
  • Smaller memory footprint.
  • All sorts of great new Curl-related HTTP features: Digest authentication, GSS-Negotiate authentication (eg, Kerberos), NTLM authentication (Microsoft), SOCKS5 proxy support, tunnelled proxy support, SSL peer-verification control, cookie support (including Mozilla cookie jars), and more.
  • A complete language-agnostic RSS/Atom parsing, fetching, and saving interface. Write your own interface in whatever language you want!

I could keep going, but I just saw the time. I've got to take a look at picard's busted hard drive (yeah, another one), and check on kylie's fancy new off-site backups (thanks, Alonzo!). Be sure to check out Sean's article, "How I Implemented Tags".


iPabs Returns!
Mon Mar 28 01:47:33 2005 :: Link

I got another PDA. A friend of mine from Wednesday night poker sells PDAs for Microsoft, and he had a bunch of older demo models just laying around, so he perma-loaned me an HP iPaq 5455. On Friday I went out and picked up a 1 gig SD card, then spent the rest of the weekend setting things up. I've scoured FreewarePPC and installed just about everything I need — emulators (NES, SNES, and Gameboy) and their associated ROMs, an MP3 player, and (of course) and eBook reader. Anyway, I've been out of the PocketPC game for over a year, so I'm open to suggestions. Are there any killer apps I should be using?


Raggle 0.4.0
Sat Mar 19 19:15:40 2005 :: Link

I just posted Raggle 0.4.0. The post on ruby-talk is here, and some screenshots are here. A full list of changes is in both the ruby-talk post, and the announcement on the Raggle page.


Three Language Showdown
Fri Mar 11 04:13:57 2005 :: Link

Earlier this evening I wrote an example of how to read a list of field values to delete from an input file. That's not particularly interesting. The fact that I wrote it in three different languages is, though. So, without any further ado, the world's simplest database application, written in Ruby, Perl, and PHP, for your viewing (comparing, contrasting, commenting, etc) pleasure:

Ruby:

  #!/usr/bin/env ruby

  # load the mysql library
  require 'mysql'

  # all our options go here
  opt = {
    :host => 'localhost', # server to connect to (usually localhost)
    :user => 'USER',      # replace USER with username
    :pass => 'PASSWORD',  # replace PASSWORD with password
    :db   => 'DATABASE',  # replace DATABASE with database name
    :tbl  => 'TABLE',     # replace TABLE with table name
    :fld  => 'FIELD',     # name of field to match against
  }

  # check first command-line argument for the filename
  unless path = ARGV.shift
    $stderr.puts "ERROR: missing filename"
    exit -1
  end

  # check to make sure the specified file exists and is readable
  unless File.exists?(path) && File.readable?(path)
    $stderr.puts "ERROR: missing file or unreadable file: #{path}"
    exit -1
  end

  # read a list of IDs from the file, excluding blank lines
  ids = File.readlines(path).map { |id| id.strip }.grep(/[^\s]/)

  # connect to server, select database
  db = Mysql::connect(opt[:host], opt[:user], opt[:pass])
  db.select_db(opt[:db])

  # build a big friggin list of ids for the query below
  ids_str = ids.map { |id| "'#{id.gsub(/'/, "''")}'" }.join(',')

  # build query
  query = "DELETE FROM #{opt[:tbl]} WHERE #{opt[:fld]} IN (#{id_str})"

  # print the query out on the screen
  puts query
  
  # check to make sure this really what we want to do 
  puts "Execute this query? (y/N)"
  if gets =~ /^y/
    # user entered "y" or "Y", so execute the query
    db.query(query)
    puts "Done."
  else
    puts "Cancelled."
  end

Perl:

  #!/usr/bin/perl

  # turn on warnings and strict interpretation
  use warnings;
  use strict;

  # load DBI module (see [2] below)
  use DBI;

  my %opt = {
    'host' => 'localhost',  # server to connect to (in our case, localhost)
    'user' => 'USER',       # replace USER with username
    'pass' => 'PASSWORD',   # replace PASSWORD with password
    'db'   => 'DATABASE',   # replace DATABASE with database
    'tbl'  => 'TABLE',      # replace TABLE with table name
    'fld'  => 'FIELD',      # replace FIELD with name of field
  };

  # declare vars
  my ($path, $fh, @ids, $id_str, $dsn, $db, $query, $response, $a);

  # get filename
  $path = shift @ARGV;

  # check path to make sure it's legit
  die "ERROR: Missing filename\n" unless $path;
  die "Missing or unreadable file: '$path'\n" unless (-e $path && -r $path);

  # read ids from file, stripping out blank lines
  open $fh, $path or die "Couldn't open file: $!\n";
  @ids = grep { /[^\s]/ } <$fh>;
  close $fh;
  chomp @ids;

  # build DSN string, connect to database
  $dsn = "dbi:mysql:database=" . $opt{'db'} . ";host=" . $opt{'host'};
  $db = DBI->connect($dsn, $opt{'user'}, $opt{'pass'});
  
  # build id list string
  $id_str = join(',', map { $a = $_; $a =~ s/'/''/g; "'$a'" } @ids);

  # build query string
  $query = "DELETE FROM " . $opt{'tbl'} .
           " WHERE " . $opt{'fld'} . " IN ($id_str)";

  # print query out on screen
  print "$query\n";

  # check to make sure user really wants to execute query
  print "Execute this query? (y/N)";
  $response = <STDIN>;

  # check response
  if ($response =~ /^y/i) {
    # user said 'y' or 'Y', execute query
    $db->do($query);
    print "Done.\n";
  } else {
    print "Cancelled...\n";
  }

PHP:

  <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" 
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1.dtd">
  <?php 
  require_once 'DB.php';

  echo "<?xml version='1.0' encoding='iso-8859-1'?>\n"; 

  # options
  $opt = array(
    'host' => 'localhost',  # database server (in our case, localhost)
    'user' => 'USERNAME',   # replace USERNAME with user
    'pass' => 'PASSWORD',   # replace PASSWORD with password
    'db'   => 'DATABASE',   # replace DATABASE with database
    'tbl'  => 'TABLE',      # replace TABLE with table name
    'fld'  => 'FIELD',      # replace FIELD with match field
  );

  # set page title
  $page_title = 'PHP MySQL Sample Code';
  ?>
  <html lang='en'>
    <head>
      <title><?php echo $page_title; ?></title>
    </head>

    <body>
      <form method='post' action='<?php echo $PHP_SELF; ?>'
            enctype='multipart/form-data'> 
         <!-- PHP needs this nonsense :/ -->
         <input type='hidden' name='MAX_FILE_SIZE' value='30000' />
         File: <input type='file' name='csvfile' />
         <input type='submit' value='Delete IDs from file' />
      </form>

      <?php
        # check to see if the file was uploaded
        if ($file = $_FILES['csvfile']) {
          echo "<hr />\nResults:<br />";
          handle_file($file);
        }
      ?>
    </body>
  </html>
  <?php

  #####################
  # UTILITY FUNCTIONS #
  #####################
  
  # 
  # handle file upload
  #
  function handle_file($file) {
    global $opt;

    # check to see if there was an error:
    if ($file['error'])
      die(__LINE__ . ": Error handling file upload: {$file['error']}");

    # no error, go ahead and read the list of IDs
    $ids = array_map('trim', file($file['tmp_name']));

    # build ID string
    $id_str = join(',', array_map('quote_str_cb', $ids));

    # build DSN string
    $dsn = "mysql://{$opt['user']}:{$opt['pass']}@{$opt['host']}/{$opt['db']}";

    # connect to database
    $db = DB::connect($dsn);
    if (DB::isError($db))
      die(__LINE__ . ": Couldn't connect to database: " . $db->getMessage());

    # build/print query string
    $query = "DELETE FROM {$opt['tbl']} WHERE {$opt['fld']} IN ($id_str)";
    echo "$query";

    # execute query
    $err = $db->query($query);
    if (DB::isError($err))
      die(__LINE__ . ": Couldn't query database: " . $err->getMessage());
  }

  #
  # escape quotes in string, and return quoted version of string
  #
  function quote_str_cb($str) {
      return "'" . str_replace("'", "''", $str) . "'";
  }

  ?>

Now before the pedant geek armada nails me to the wall: Yes, I know the new way of calling DBI->connect includes passing { AutoCommit => 1 } as a fourth parameter, and yes, I know that I should be passing the DSN an array to DB::connect, and yes, I know that Ruby has DBI too! And finally, yes, I know a dependency on MySQL is totally 2001, but I wrote these examples to address a specific problem. The original email includes all sorts of extra stuff, including explanations of differences between MySQL and Postgres, and the nuances of their respective regex engines. So bugger off :D.


New Versions of Imlib2-Ruby and Rubilicious
Fri Mar 11 03:45:06 2005 :: Link

I just released new versions of Imlib2-Ruby and Rubilicious. I'm lazy and I don't feel like re-typing (or even copying... did I mention lazy?) the list of changes, so if you're interested you can read the ruby-talk posts here and here (respectively). And here's the download information:


Rockbox on my iRiver H120
Fri Mar 4 04:10:18 2005 :: Link

This was already reported on Engadget and Slashdot, but in case you missed it; Rockbox has been ported to the iRiver H100 series! The bad news: No realtime MP3 playback (yet). The good news: Rockboy, a port of the gnuboy gameboy emulator! It's still a little rough around the edges, so I've started submitting patches. The first patch — which has already been accepted to CVS — adds a menu to Rockboy, and the second patch adds load and save state support (via the menu).

Anyway, Rockbox is great. It boots up about 3 times faster than the stock iRiver interface, and it has a lot more features, too! If you're an H120 owner who's unfazed by flashing firmware and cross compiling, then I highly recommend you give Rockbox a try (once the MP3 playback is ironed out, that is). As my parting gift, here's Bionic Commando running on my H120 (and a couple shots of my menu system as well):


Profanity!
Fri Mar 4 01:38:28 2005 :: Link

The last week or so I've been working on Profanity, a Ruby windowing library for Curses. It's features include relative window geometry, a include an HTML-like markup for styled text, and proper terminal resizes support. I've whipped up a couple of example movies to demonstrate the geometry engine and styled text:

  • profanity-2050228.mpg: An example of the geometry engine handling terminal resizes.
  • profanity-2050228.mpg: An example of window constraints, window gravity, and styled text (notice how styles can be applied to window titles as well as contents).

By the way, if you're having trouble with these movies, try them in MPlayer (I'm usually an Xine guy myself, but it seems to mangle these too). In the interest of staving of the torch- and pitchfork-wielding mob of people who can't get either movie to play, here are a few still shots from each (click the thumbnails for the full image):

profanity 20050228-00 profanity 20050228-01 profanity 20050228-02 profanity 20050228-03 profanity 20050301-00 profanity 20050301-01 profanity 20050301-02

Finally, here is a simple example from CVS) which demonstrates what Profanity code looks like:


require 'profanity'

# create a basic window class
class HelloWorldWindow < Profanity::Window
  # contents of this window
  LINES = [
    "<center>Welcome to <u>Profanity</u>!</center>",
    '<center>Press <b>Q</b> or <b>ESC</b> to quit.</center>',
  ]

  # override Window#draw_contents for our own devious purposes
  def draw_contents
    clear
    puts LINES
  end
end

# create the profanity window manager
wm = Profanity::WindowManager.new

# create geometry, and fix height to 4 lines
geom = Profanity::Geometry.new(0.5, 0.5, 0.7, 0.0)
geom.height_absolute = 4

# set gravity to center of the screen
geom.gravity.x = geom.gravity.y = 0.5

# create our window, show and refresh it
win = HelloWorldWindow.new(wm, geom, 'Hello World')
win.show; win.refresh

# run until the user presses q, ctrl-c, or escape
wm.run { |key| break if [?q, 27, Curses::KEY_CTRL_C].include?(key) }


Guess Who's Back?
Wed Feb 16 00:40:23 2005 :: Link

The blog post hiatus has ended! Here's what's new in the world o' Pablotron. First of all, the main hard drive on vault — my file/database/LDAP/email server — bit the dust last Wednesday. Fortunately the drive just started to fail (instead of dying outright). I had ample room to do immediate backups, and I had an unused 160G drive laying around. I spent most of Sunday afternoon and all of Monday evening, partitioning the new drive and copying stuff back to it. As far as I can tell, the only thing I actually lost was the words file for spamprobe. I don't really consider that much of a loss, since I save all my email (even the cursed spam), so I can easily toss the requisite good and bad corpora at spamprobe to get things going again. Even though I'm short a 100G drive now, the experience overall has been a positive one. Here's some thoughts I had; maybe they'll prevent a week of stress for someone else:

  • Regular backups are just something you do. The ad-hoc backups I've been doing are better than nothing, but they wouldn't have done me any good if the my drive had died outright. Had the circumstances been different, I would have lost weeks, possibly even a month of email. My solution is (rather, will be, once everything is up and running again) an NFS-mounted backup directory on every machine (obviously not for peope who don't like NFS)). Each machine will be responsible for it's own daily and weekly backups, via cron. Depending on how large this data set is, I'll be burning DVDs of the backup directory contents on a weekly or bi-weekly basis. Aside: Richard (richlowe) has been advocating revision controlled config files for quite a while (eg. cvs -d pabs@cvs:/cvs co etc-files/vault); maybe I'll give that for a spin, too.
  • Distribute services across machines. I've got 4 other machines sitting around twiddling their thumbs at the moment. Any of them coud easily be an authentication, database, email, LDAP, or CVS server, but instead they're all sitting around twiddling their thumbs (to be fair, sumo is my IRC /PostGres machine, but that hardly qualifies as a crippling load).
  • Keep extra hardware laying around. As a true geek you're already doing this, of course :). The drive in vault started failing at 1:30 in the morning on a Wendesday morning. I was able to start making backups and moving stuff around right then. If I didn't have the extra hard drive, I would have been SOL for several platter-scraping hours.
  • Losing your spam filter settings means you get to say cool words like "corpora" on your web page.

On the non-catastrophic hardware failure front, I upgraded halcyon to the latest Xorg, then promptly downgraded to the latest stable release. Here's the approximate order of events:

  1. Spent an hour or two configuring, compiling, and installing the latest Xorg.
  2. Ran X, and found out that the proprietary NVidia driver isn't compatible with the latest CVS snapshot of Xorg.
  3. Discovered just how painful the composite extension is without hardware acceleration by foolishly attempting to run X using the nv driver. Hint: Imagine using Netscape Navigator 3.0 on your old Commodore 64 with Photoshop doing an RLE Gaussian Blur on a 100 meg image in the background.
  4. Promptly downgraded to the stable release, cursing both NVidia for their proprietary sillyness, and the bastards at freedesktop.org for having the audacity to make source code changes that inconvenienced me. I spent plenty of time on this step, so go ahead and re-read that last paragraph a couple of times.

Since I spent the majority of a Sunday afternoon recompiling X no less than 3 times, I also took the opportunity to try out the latest Enlightenment DR16 from CVS (yes Kim, I'm one of the few people still using e16). It's got it's own built-in, mostly (semi?) working composite manager, so the neither the patch nor the xcompmgr hackery I describe in this post are necessary any more). The new default theme looks great, too!

Why use other peoples' broken software when you can write your own? Here's the latest on the Pablotron coding front:

  • I've converted the RSS feeds on pablotron.org, paulduncan.org, and raggle.org from steaming loads of standards-incompliant crap to pedantically-correct RSS 2.0. If your RSS aggregator couldn't read my pages before, it probably can now (unless your aggregator is based on the RSS library built-in to Ruby 1.8, but I'll get to that part of the story in a few minutes...)
  • Lots and lots and lots of updates to the next version of Raggle. Some of the changes are even by me! Thomas Kirchner (redshift) has been doing an unbelievable amount of work on the CVS version of Raggle. So much so, in fact, that I feel kind of embarassed calling this latest version mine at all. So I think when it's ready for release, we'll call it kirchneraggle or something more suitable ;).
  • This patch for Ruby which adds wcolor_set support to the built-in Curses interface. Ville suggested it eons ago, and that was the last thing stopping me from porting Raggle from Ncurses-Ruby.
  • A partially working Curses windowing library for Ruby. This isn't in CVS just yet, but don't worry, I've got some new stuff for you to play with. Keep reading...

The big stuff I've been working on lately is core of the future Raggle. Before I begin, here's a high-level overview of how the components interact with one another (yup, a diagram!):

next gen raggle

I've mentioned Squaggle previously, but for those of you sleeping in the back of the class (you know who you are), here's a brief recap. Squaggle is the SQLite-Ruby-based engine for Raggle. It's cleaner, faster, it uses less memory, and it lets me do all sorts of cool things I can't really do with the current engine (fancty delicious-style tagging, fast cross-feed searching, smart/auto categorization, and more). The version of Squaggle in CVS is functional (it even includes a usable WEBrick-based interface.

So what's this new stuff on ye olde diagram? libptime is a C-based RFC822 datetime and W3C datetime parsing library. It's BSD licensed, so you can download version 0.1.0 (signature), and use it to your heart's content. The other new library on the diagram is libfeed, an Expat-based RSS (0.9x, 1.0, and 2.0)/Atom feed parser. Why bother writing an RSS parser in C? The existing Raggle engine is slow, partly from being DOM-based, and partly from being written in Ruby. Don't get me wrong, REXML is a great XML parser, but RSS aggregators deal in volume, and I want to be sure the volume isn't constrained by parsing. I also noticed there wasn't a nice C-based RSS/Atom parsing library. Now there is (well, almost!). If that doesn't convince you, then maybe this will:


pabs@halcyon:~/cvs/libfeed/test> du -sh data/big-pdo-wdom.rss 
15M     data/big-pdo-wdom.rss
pabs@halcyon:~/cvs/libfeed/test> time perl -mXML::RSS -e \
  '$rss = new XML::RSS; $rss->parsefile("data/big-pdo-wdom.rss");'
real    7m56.892s
user    4m31.578s
sys     0m19.939s
pabs@halcyon:~/cvs/libfeed/test> time perl -mXML::RSS -e \
  '$rss = new XML::RSS; $rss->parsefile("data/big-pdo-wdom.rss");'
real    5m57.838s
user    4m28.727s
sys     0m3.703s
pabs@halcyon:~/cvs/libfeed/test> time ruby -rrss/2.0 -e \
  'RSS::Parser::parse(File.read("data/big-pdo-wdom.rss"))'
real    2m30.950s
user    1m46.904s
sys     0m8.610s
pabs@halcyon:~/cvs/libfeed/test> time ./testfeed data/big-pdo-wdom.rss \
  >/dev/null 2>&1
real    0m2.195s
user    0m1.472s
sys     0m0.104s
pabs@halcyon:~/cvs/libfeed/test> time ./testfeed data/big-pdo-wdom.rss \
  >/dev/null 2>&1
real    0m2.010s
user    0m1.475s
sys     0m0.099s

The Perl times were so bad I had to run them twice to be sure. 60 times faster than Ruby and over 100 times faster than Perl; I'd say that's a pretty good start :).

Unfortunately, I have to be awake in three hours, so I'll have to save the rest of the next-gen Raggle description for another day...


#Gah People
Thu Dec 9 16:09:57 2004 :: Link

Last night I finished working on #Gah People, a page for the IRC channel #gah. Stop by and check it out. If you're a channel regular, create an account and add your own information! Hopefully I'm going to get to talk to crypticreign about either collaberating or merging with Planet Gah. Stay tuned.


Navigation

Previous . Next . Display entries.
Page: 1 ... 6 7 8 9 10 11 12 13 14 ... 48 (48 pages)