After removing the Python 2.7 support, this section:

https://docs.scrapy.org/en/latest/topics/leaks.html#debugging-memory-leaks-with-guppy

should be removed or merged with this:
https://docs.scrapy.org/en/latest/topics/leaks.html#topics-leaks-muppy

Scraping a Google search results page with HTML links containing attributes href and ping, such as:

<a href="https://en.wikipedia.org/wiki/Go_(programming_language)" ping="/url?sa=t&amp;source=web&amp;rct=j&amp;url=https://en.wikipedia.org/wiki/Go_(programming_language)&amp;ved=2ahUKEwi-yY2t5eTeAhUzNX0KHXbrD7cQFjADegQIDRAB"><h3 class="LC20lb">Go (programming language) - Wikipedia</h3

If one opens the link to the docs provided in README the Readme opens on readthedocs.io. There is no navigation bar to find where one can browse to quick start page or advanced. You can only go there if one searches quick start and click on the page. Then there are navigation links for browsing through the docs.

Jus for the record:
I'm using Firefox (60.9.0 esr) on Windows 10 Pro.

Really gr

Describe the bug
When using the cdp driver, during closing of a browser page, this error sometimes appears.

{"level":"warn","time":"x","url":"x","error":"rpcc: the connection is closing","time":"x","message":"failed to close browser page"}
{"level":"error","time":"x","error":": rpcc: the connection is closing: session: detach timed out for session 5C391DF4E758E985AE3CBAA03774E562","t

SeleniumRequest should use meta to pass arguments

self.wait_time = wait_time
self.wait_until = wait_until
self.screenshot = screenshot
self.script = script

when use scrapy_redis.scheduler.Scheduler that won't be serialized

Scrapy has a setting directive implemented for Sphinx documentation that allows linking to settings while formatting them as code in an easy manner.

Looking at #212, I think Spidermon could benefit from implementing such a directive as well.

CONTRIBUTING.md has some guidelines, but essentially there is simply a lot of stuff that needs filled out in the docs.

Also, if you would like to use another documentation format feel free. Listing everything is something I came up with in early development but it's prob

Use https://github.com/stopstalk/stopstalk-deployment/blob/79911cdaf9e8ff0da963d044b286893144b189ec/modules/sites/codechef.py#L204 to replace
https://github.com/stopstalk/stopstalk-deployment/blob/79911cdaf9e8ff0da963d044b286893144b189ec/modules/sites/codechef.py#L139

See the code and update the docs.

wiki의 내용을 지금 버전 기반으로 업데이트 필요.

최소한 몇 버전때 동작하는 코드인지 명시하고, 코드 앞에 설치 코드를 제공.

혹은 사용자 들에게 자신들의 코드를 기여해주기를 부탁할 수도 있겠다.

Are you submitting a bug report or a feature request?

Feature request/documentation enhancement

What is the current behavior?

The requirements for a user to get up and running are insufficient with regard to the requirements and dependencies. I encountered this experience when trying to resolve #31 on a fresh Win

Datasets with identifiers containing upper case letters are being duplicated in the status.json file contained in the working_dir of the project. This is causing the desired flag in the DIG UI to be reset to zero. Hence, the data is not ingested into the system.

Example status.json:

{
"desired_docs": {
"imfCPI": 0,
"imfcpi": 1
},
"added_docs": {
"imf