Extract Summit 2022 Coding Contest

Make sure that you’ve registered your Scrapy Cloud account and have Scrapy Discord access. You’ll need to submit your registration form as well.

Once these are in place, you’re ready to take part in the contest when it launches on 28th September.

Here’s how the contest will go:

1. The URL of the target website will be revealed on Discord. There will be a specification of the item fields that need to be extracted.
2. You must write a spider that extracts all items with the specified fields and run it in Scrapy Cloud.
3. Once the Scrapy Cloud job finishes, you must submit the job ID to a bot in the Scrapy Discord server.
4. The bot will let you know if you have managed to extract all items with complete data.
5. If you failed, update your code and try again with a new Scrapy Cloud job. The bot accepts unlimited job submissions for the duration of the contest.
6. To win, be the first to submit a job that successfully extracts all expected data.
7. The website does not ban clients, so you will not need a proxy. But crawling the website and extracting item data will not be straightforward, so do not expect to get a working spider on your first run.