scrAPI on Snow Leopard

Today I needed to scrape some data from a website and tried to use the trusted old scrAPI to do the job. Grrrr, its not working. Throwing an error:

Scraper::Reader::HTMLParseError: Scraper::Reader::HTMLParseError: Unable to load /Library/Ruby/Gems/1.8/gems/scrapi-1.2.0/lib/scraper/../tidy/libtidy.dylib

After some time on google I didn’t find any fixes for the issue. So I decided to build from source…

I grabbed the assaf’s github repository.

• git clone git://github.com/assaf/scrapi.git

Then tried the tests by running

rake test

117 tests, 346 assertions, 0 failures, 44 errors

Nope errors all over the show… Looking at the original exception message, I checked if the libtidy.dylib exist on the lib/tidy directory. Nope, not there….

So where do I get this library file….

MacPorts to the rescue… Install tidy from MacPorts using the following command:

• sudo port install tidy

Now we need to find where MacPorts installed the files using the following port command:

• port contents tidy

The result:

Port tidy contains:
/opt/local/bin/tab2space
/opt/local/bin/tidy
/opt/local/include/buffio.h
/opt/local/include/fileio.h
/opt/local/include/platform.h
/opt/local/include/tidy.h
/opt/local/include/tidyenum.h
/opt/local/lib/libtidy-0.99.0.dylib
/opt/local/lib/libtidy.0.dylib
/opt/local/lib/libtidy.a
/opt/local/lib/libtidy.dylib
/opt/local/lib/libtidy.la

Now all we need to do is copy the library file to our scrAPI source directory:

• cp /opt/local/lib/libtidy.dylib [your source location]/lib/tidy/libtidy.dylib

Ok, before we speed ahead. Lets just run those test to check if all is fine:

117 tests, 474 assertions, 0 failures, 0 errors

Awesome, we are almost there. Next we need to build the gem using the rake:

• rake package

Make sure you get a ‘Successfully built RubyGem’ message. Now we are ready to install the newly build gem and test scrAPI again.

• sudo gem install pkg/scrapi-1.2.1.gem

And there you go, scrAPI working again.

Easily create strong passwords

Create strong passwords on the command line by typing

openssl rand -base64 6

and the output:

=> 050L+4Kx

Run the command again:

=> /sUIIy8s

Cool, now you should be able to create strong passwords quickly.

TextMate command to annotate your current ActiveRecord Model with the DB Schema

Edit: If you can’t wait until the end, here is the download link.

Who hasn’t used the Annotate Models plugin written by Dave Thomas, of Rails Pragmatic Studio fame? If you haven’t don’t fret… The plugin basically adds a comment block to the top of all of your model classes documenting the current database schema for the given model. I like to have the info in my model files to make it a little easier to work with models.

The rails bundle within TextMate gives you a ‘Show DB Schema for current class’ command you can press to show a tool tip with the database schema (Control + Shift + Command + S). This works great, but I find the command sequence to long and the slight wait to see the schema info breaks my rhythm.

To solve the problem I made a copy of the source files used by the command and modified it slightly to output the info as comments at the top of the file. Now I have nicely annotated model files. This is a real time saver.

The bundle also contain the the Beautify command. This command adds automatic code formatting capability for your ruby code. I found the beautify command from Tim Burks. Cheers Tim…

You can download the bundle here.

Technorati Tags: Ruby, Rails, TextMate, JavaScript, Mac OSX

Ruby, Rails, TextMate, JavaScript, Mac OSX, Subversion

Ruby

• Object.alias_class_method – adds a singleton/eigen/virtual/meta-class to the given instance which you can use to add methods.
• assert{ 2.0 } – An interesting new gem that make your assertion definitions feel more natural.
• Have you killed a design pattern today?
• Timeout code execution – to ensure a snippet of Ruby code doesn’t run for too long you can use the timeout function.
• Loading classes from strings – Using method like Module#const_get, Kernel#qualified_const_get and eval.

Rails

• Using DRb to preserve a Selenium Driver – Speed up your selenium tests considerably
• Integrating Google Maps in your Rails 2.0 application

TextMate

• HTML Bundle Tips – Great tips on using the HTML bundle in Textmate

JavaScript

• Language Integrated Query for JSON – This is really interesting. Shows you have versatile JavaScript is.

Mac OSX

• Install ruby memcached on Mac OSX

Subversion

• Distributed Version Control – A comparison between Darcs, Subversion, Git and Mercurial

Technorati Tags: Ruby, Rails, TextMate, JavaScript, Mac OSX, Subversion

Ruby, Rails, RSpec, JavaScript, Ubuntu, Mac OS X

Ruby

• Micro-patterns in ruby

Rails

• Nginx and Memcache – a 400% boost!
• Rails two dot oh! video
• Tips for Debugging BackgroundDRB
• Better reporting with Sparklines
• Segregated page cache storage
• How to handle incoming emails in rails RESTfully
• Re-using rails controllers is a good thing

RSpec

• Validating RSS feeds with RSpec

JavaScript

• Event Delegation made easy

Ubuntu

• Setting up rmagick on Ubuntu

Mac OS X

• Why DTrace makes Leopard a must-have upgrade

Technorati Tags: Ruby, Rails, RSpec, JavaScript, Ubuntu, Mac OS X

Powered by ScribeFire.

Ruby, Rails, RSpec, Rails Plugin, JavaScript, OS X

Ruby

• Executing Shell Commands in Ruby
• What is a ruby symbol? – symbols explained

Rails

• 7 Deadly [Rails] Sins, Part 2
• Adding/Removing columns are now much easier
• RailsConf 2007 keynote videos
• Is your Rails app XSS safe?
• Feeds for Free – Microformats your Atom feeds.

RSpec

• Prevent those race conditions in RSpec specs
• Handy dynamic rspec tip

Rails Plugin

• Ext Scaffold Generator Plugin for Rails – can be viewed as a drop-in replacement for Rails’ standard Resource Generator.
• refresh_view – Refreshing Rails Generated Views.
• Click To Globalize – is an extension for Globalize plugin, it allows to edit in place globalized labels. The video tutorial for Click to Globalize is now available in high resolution

JavaScript

• Ext JS and Rails, how do they get along?
• Low Pro Behaviours 101
• Low Pro Behaviours 101: Part 2

OS X

• A black OS X Leopard Terminal theme that is actually readable

Technorati Tags: Ruby, Rails, RSpec, Rails Plugins, JavaScript, OS X

Post created with TextMate.

Ruby, Rails, OS X (Leopard), CSS

Ruby

• Hash#tomod – extend your class to except a hash of values in the constructor and then access the values as if they are read accessors on the object. Very enlightening post, I’m sure ActiveRecord::Base implements something.

Rails

• Managing a high performance rails application without tearing your hair out (presentation)
• composedof – Format your data using the Rails composed_of Value Object

OS X (Leopard)

• Exploring DTrace on Leopard

CSS

• What CSS did we learn in 2007

Technorati Tags:

Ruby

,

Rails

,

OS X (Leopard)

,

CSS

Powered by ScribeFire.

Ruby, Rails, Rails Plugins, Mac OS X

Ruby

• Five Ruby Addictions
• What is three-equals (===)?
• Accumulators in Ruby

Rails

• Seeing Rails Resources Clearly
• Troubleshoot your rails deployment – Short term problem analysis and resolution
• will_paginate – view testing tutorial
• How will openID change your site?Ruby, Rails, Rails Plugins, Mac OS X
• A fair proxy balancer for Nginx and Mongrel
• How do those ActiveRecord dynamic finders and attributes work?
• Using call stack information at runtime
• PDF generation with Ruby, Rails and HTMLDoc
• Database optimization tips for your Rails Application
• How to find unused code in Rails
• How to view gem documentation
• CruiseControl.rb Growl notifier plugin

Rails Plugins

• Query stats plugin – a Rails plugin for tracking database queries.
• Smooth Migrations – allows database migrations to continue running instead of aborting on failure.
• QueryTrace – is a great Rails plugin for pinpointing where in your Rails application that slow query is running.

Mac OS X

• Rake command completion in the Bash shell

Technorati Tags: Ruby, Rails, Rails Plugins, Mac OS X

Powered by ScribeFire.

Ruby, Rails, Rails Plugins, Mac OS X, JavaScript

Ruby

• Hash#diff – helpful for determining the difference between two hashes.
• String#each_char – Yields a single-character string for each character in the string.
• Integer#even – helpful when adding zebra striping to views.
• Enumerable#index_by – Convert an enumerable to a hash.
• Enumerable#group_by – Collect an enumerable into sets, grouped by the result of a block. Is helpful for grouping elements of an Enumerable by an attribute or an arbitrary grouping.
• Synonyms vs Aliases
• Testing private methods in Ruby
• Keyword-based message sending (Ruby compared to Smalltalk and Objective-C)
• A crash course in Ruby
• main.rb – a class factory and dsl for generating command line programs real quick.
• Reading an Excel file with Ruby

Rails

• Ruby on Rails 101 Presentation (pdf) – Very comprehensive and informative
• Use JQuery on Rails
• Map IP address to latitude and longitude part 1 & part 2
• HasFinder – is an extension to ActiveRecord that makes it easier than ever to create custom find and count queries.
• Pagecaching Highly personalized web pages – making rails wicked fast.
• Rails has and belongs to many (habtm) gotcha – solving slow queries
• Using Paypal with Rails

Rails Plugins

• Lessjsroutes – Have you ever wanted to use named routes in your JavaScript?
• model_stubbing – create in-memory versions of models for testing.
• candy – a rails plugin which adds some ruby candy to your app.

Mac OS X

• Installing RMagick on Leopard
• Installing ruby, mysql in Leopard

JavaScript

• ProtoType vs JQuery – Your Momma’s so fat 🙂
• Scripteka – Site specializing in Prototype extension libraries and scripts
• Pseudo-custom events in Prototype 1.6
• Lightbox JS – is a simple, unobtrusive script used to overlay images on the current page.

Technorati Tags: Ruby, Rails, Rails Plugins, Mac OS X, JavaScript

Powered by ScribeFire.

Ruby, Rails, TextMate, JavaScript, CSS, Web Usability, Mac OS X

Ruby

• String#constantize = tries to find a declared constant with the name specified in the string. It raises a NameError when the name is not in CamelCase or is not initialized.
• How to build a drawing application using shoes, OS X and ruby

Rails

• RSS in Rails 2.0 how to
• Writing custom validators
• Rails fixtures – tips and tricks
• Partial flash messages in Rails
• Rake task to remove css selectors not in use (needs Dust-Me Selectors firefox add-on)

TextMate

• Tasks (ToDo) bundle
• Minimize parentheses in TextMate

JavaScript

• Prototype 1.6.0, script.aculo.us 1.8.0 and Pragprog book released in three way whammy

CSS

• How to design a website – A fully comprehensive step by step tutorial, from design to HTML & CSS.
• CSS Image Replacement, The how to
• CSS rounded corners generator

Web Usability

• clickdensity – analyse the user experience and identify usability and information architecture issues

Mac OS X

• Installing Freeimage/ImageScience on OS X 10.5 (Leopard)

Technorati Tags: Ruby, Rails, TextMate, JavaScript, CSS, Web Usability, Mac OS X

Powered by ScribeFire.