What's are good deisign patterns to build a web crawler and scraper for a Rails app?

I am tring to build my first web crawler and scraper for one of the features within a rails app and I'd like some advice on how best to design it.

I'll use Mechanize to navigage pages and complete forms to submit then Nokogiri to scrape the data from the resulting served pages. My app will need to run these operations for various different urls each with differring form completion and submission requirements.

How might i best to design/house/encapsulate such code to serve a rails app. Internal Model Controllers without views, rake tasks being called from exiting controllers, separate API?

Any advice and guidance would be hugely appreciated.

Thanks