After removing the Python 2.7 support, this section:

https://docs.scrapy.org/en/latest/topics/leaks.html#debugging-memory-leaks-with-guppy

should be removed or merged with this:
https://docs.scrapy.org/en/latest/topics/leaks.html#topics-leaks-muppy

If you're using proxies with requests-html and rendering JS sites is all good. Once you render a website pyppeteer don't know about this proxies and will expose your IP. This is an undesired behavior when scraping with proxies.

The idea is that whenever someone passes in proxies to the session object or any method call, make pyppeteer also use these proxies. #265

Scraping a Google search results page with HTML links containing attributes href and ping, such as:

<a href="https://en.wikipedia.org/wiki/Go_(programming_language)" ping="/url?sa=t&amp;source=web&amp;rct=j&amp;url=https://en.wikipedia.org/wiki/Go_(programming_language)&amp;ved=2ahUKEwi-yY2t5eTeAhUzNX0KHXbrD7cQFjADegQIDRAB"><h3 class="LC20lb">Go (programming language) - Wikipedia</h3

问题是这样的，我想爬取商品分页的信息于是我用了个for循环，执行document = Jsoup.connect(domain+reviews+String.format(b, p)).get()改变p的值来改变评论的页码。

但是当爬完第一页后再爬取第二页评论时（没准备爬取一页评论时都会执行这句document = Jsoup.connect(domain+reviews+String.format(b, p)).get();）出现了这样的错误：
java.net.ConnectException: Connection timed out: connect
at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
at java.net.DualStackPlainSocketImpl.s

Tabula API version: 1.2.1.18052200
Filename: 3_2019년_통계부록.pdf
Internal Server Error (500)
    
      
        Request Method:
        POST
      
      
        Request URL:
        http://127.0.0.1:8080/pdf/8a6599b3be99fda826cc0448d74f0f74dfd3d78d/data

lines must be orthogonal, vertical and horizontal

Got this while extracting table
[pdf file](https://drive.google.com/fil

Describe the bug
When using the cdp driver, during closing of a browser page, this error sometimes appears.

{"level":"warn","time":"x","url":"x","error":"rpcc: the connection is closing","time":"x","message":"failed to close browser page"}
{"level":"error","time":"x","error":": rpcc: the connection is closing: session: detach timed out for session 5C391DF4E758E985AE3CBAA03774E562","t

There are several things not accurately documented/outdated:

• -v2 is used the examples but does not work
• # duckduckgo not supported although it is in the list of supported search engines
• To get a list of all search engines --config is suggested but that just fails

My project have routing based on hosts. But web driver make request to http://127.0.0.1:9080.
How can i change host?

I'm trying to type some stuff into a page w/ artoo, and i think simulate() will do the trick.

i've never used simulate(), though, so i have no idea what the syntax is.

the github page for simulate linked in the artoo docs has no documentation, and the only docs i can find are for jquery-simulate-ext.

are there any examples I

Checklist for items that I know need worked on before the ui branch can be merged into the dev branch

• Create documentation
• Add offline unit tests
• Add online integration tests
• Add tests to run_*_tests.sh
• Can we add actual ui tests? (aka Selenium o

The line 279

https://github.com/scrapy/parsel/blob/332b7e87ba046c48f8b17ea3a4064015f1f58ffe/parsel/selector.py#L277-L279

appears to raise a warning in the scrapy build on Python 3.7 for the docs target:

https://travis-ci.org/scrapy/scrapy/jobs/653351006#L345-L350

Warning, treated as error:

/home/travis/build/scrapy/scrapy/.tox/docs/lib/python3.7/site-packages/parsel/selector.py:d

We need to put strings in to a lot of places:

• YAML
• markdown
• html inside markdown
• code blocks
• arguments added to includes like with {% include figure.html caption="..." %} (which are a special case too b/c they actually get processed as markdown...)

It's not always intuitive how ' and " will interact in which domain, and there are often mutiple ways to do it which can be