https://docs.scrapy.org/en/latest/topics/spiders.html#crawlspider has a warning block that includes the "New in version" line.

Both indenting the warning block (not sure what's the best practice for the indentation here; also, XMLFeedSpider below has the same block which should have the same indentation I think) and moving versionadded before it (I think I don't like it) will work but I don't

LET page = DOCUMENT("YOUR_URL", {
	driver: "cdp",
	ignore: {
        resources: [
            {
                url: "*",
                type: "image"
            }
        ]
    }
})

Main examples at Apify SDK webpage, Github repo and CLI templates should demonstrate how to manipulate with DOM and retrieve data from it.

Also add one example of scraping with Apify SDK + jQuery to https://sdk.apify.com/docs/examples/basiccrawler

Feedback from: https://medium.com/better-programming/do-i-need-python-scrapy-to-build-a-web-scraper-7cc7cac2081d

I lost an hour trying to make

Currently, in master, the logs are stored in a non-optional manner.

I think a good improvement for this would be to make the file logging optional, as not everyone has log rotation configured or needs logging (especially ad-hoc users).

Mentioned first here: elixir-crawly/crawly#155 (comment)

Some improvements i can think of is:

• optional switch to enabl

We have different mixins in spidermon/contrib/monitors/mixins directory, but no documentation.