I’ve been working on a Django-based project since October 2010 that includes some integration with Scrapy. Scrapy is a web scraping framework for Python and due to its design inspiration, Django developers will be quickly familiar with it. I have installed Scrapy no less than 10 times at this point and decided it was time for a script. This post is more of a reminder to me for the next time I install Scrapy, but if it works for you as well – great!.

This script works for Ubuntu 9.10, 10.04 and 10.10.  If you’re on an older release, you’ll need to install some dependencies first and then follow these instructions.

sudo apt-get install curl
curl -s http://archive.scrapy.org/ubuntu/archive.key | sudo apt-key add -
DISTRO=$(lsb_release -cs)
echo deb http://archive.scrapy.org/ubuntu $DISTRO main | sudo tee -a /etc/apt/sources.list
sudo apt-get update
sudo apt-get install scrapy-0.12