Web scraping with Python
Here is a set of resources for scraping the web with the help of Python. The best solution seems to be Mechanize plus Beautiful Soup.
See also :
- ClientTable (see also this comment)
- DOMForm
- Pull parser
- wwwsearch ’s FAQ
- Simon Willison’s weblog on this topic
- Dive Into Python’s tutorial on HTML processing
Off-topic : proxomitron looks like a nice (python-friendly ?) filtering proxy.
- Rapid prototyping with microcontrollers ?...
- Pierre Levy vs Tim Berners-Lee, round 0.1...
- Web scraping, web mashing...
- A Periodic Table of Visualization Methods...
- WikiCalc: Web 2.0 spreadsheets wikified...

December 29th, 2004 at
[...] wling design is based on the John L. Lee’s mechanize framework for python. There are other tools available in Python (see also . And several other approaches are available when you want to [...]
March 14th, 2005 at
Check also Web scraping with Python (part II) where I present a python app that helps you scraping the web.