httpx is a fast and multi-purpose HTTP toolkit allow to run multiple probers using retryablehttp library, it is designed to maintain the result reliability with increased threads.
Resources
- Resources
- Features
- Usage
- Installation Instructions
- Running httpx with stdin
- Running httpx with file input
- Running httpx with CIDR input
- Running httpX with subfinder
- Notes
- Thanks
Features
- Simple and modular code base making it easy to contribute.
- Fast And fully configurable flags to probe mutiple elements.
- Supports multiple HTTP based probings.
- Smart auto fallback from https to http as default.
- Supports hosts, URLs and CIDR as input.
- Handles edge cases doing retries, backoffs etc for handling WAFs.
Supported probes:-
| Probes | Default check | Probes | Default check |
|---|---|---|---|
| URL | true | IP | true |
| Title | true | CNAME | true |
| Status Code | true | Raw HTTP | false |
| Content Length | true | HTTP2 | false |
| TLS Certificate | true | HTTP 1.1 Pipeline | false |
| CSP Header | true | Virtual host | false |
| Location Header | true | CDN | false |
| Web Server | true | Path | false |
| Web Socket | true | Ports | false |
| Response Time | true | Request method | false |
Installation Instructions
From Binary
The installation is easy. You can download the pre-built binaries for your platform from the Releases page. Extract them using tar, move it to your $PATHand you're ready to go.
Download latest binary from https://github.com/projectdiscovery/httpx/releases
▶ tar -xvf httpx-linux-amd64.tar
▶ mv httpx-linux-amd64 /usr/local/bin/httpx
▶ httpx -hFrom Source
httpx requires go1.14+ to install successfully. Run the following command to get the repo -
▶ GO111MODULE=on go get -v github.com/projectdiscovery/httpx/cmd/httpxFrom Github
▶ git clone https://github.com/projectdiscovery/httpx.git; cd httpx/cmd/httpx; go build; mv httpx /usr/local/bin/; httpx -versionUsage
httpx -hThis will display help for the tool. Here are all the switches it supports.
| Flag | Description | Example |
|---|---|---|
| H | Custom Header input | httpx -H 'x-bug-bounty: hacker' |
| follow-redirects | Follow URL redirects (default false) | httpx -follow-redirects |
| follow-host-redirects | Follow URL redirects only on same host(default false) | httpx -follow-host-redirects |
| http-proxy | URL of the proxy server | httpx -http-proxy hxxp://proxy-host:80 |
| l | File containing HOST/URLs/CIDR to process | httpx -l hosts.txt |
| no-color | Disable colors in the output. | httpx -no-color |
| o | File to save output result (optional) | httpx -o output.txt |
| json | Prints all the probes in JSON format (default false) | httpx -json |
| vhost | Probes to detect vhost from list of subdomains | httpx -vhost |
| threads | Number of threads (default 50) | httpx -threads 100 |
| http2 | HTTP2 probing | httpx -http2 |
| pipeline | HTTP1.1 Pipeline probing | httpx -pipeline |
| ports | Ports ranges to probe (nmap syntax: eg 1,2-10,11) | httpx -ports 80,443,100-200 |
| title | Prints title of page if available | httpx -title |
| path | Request path/file | httpx -path /api |
| paths | Request list of paths from file | httpx -paths paths.txt |
| content-length | Prints content length in the output | httpx -content-length |
| ml | Match content length in the output | httpx -content-length -ml 125 |
| fl | Filter content length in the output | httpx -content-length -fl 0,43 |
| status-code | Prints status code in the output | httpx -status-code |
| mc | Match status code in the output | httpx -status-code -mc 200,302 |
| fc | Filter status code in the output | httpx -status-code -fc 404,500 |
| tech-detect | Perform wappalyzer based technology detection | httpx -tech-detect |
| tls-probe | Send HTTP probes on the extracted TLS domains | httpx -tls-probe |
| tls-grab | Perform TLS data grabbing | httpx -tls-grab |
| content-type | Prints content-type | httpx -content-type |
| location | Prints location header | httpx -location |
| csp-probe | Send HTTP probes on the extracted CSP domains | httpx -csp-probe |
| web-server | Prints running web sever if available | httpx -web-server |
| sr | Store responses to file (default false) | httpx -sr |
| srd | Directory to store response (optional) | httpx -srd httpx-output |
| unsafe | Send raw requests skipping golang normalization | httpx -unsafe |
| request | File containing raw request to process | httpx -request |
| retries | Number of retries | httpx -retries |
| random-agent | Use randomly selected HTTP User-Agent header value | httpx -random-agent |
| silent | Prints only results in the output | httpx -silent |
| stats | Prints statistic every 5 seconds | httpx -stats |
| timeout | Timeout in seconds (default 5) | httpx -timeout 10 |
| verbose | Verbose Mode | httpx -verbose |
| version | Prints current version of the httpx | httpx -version |
| x | Request Method (default 'GET') | httpx -x HEAD |
| method | Output requested method | httpx -method |
| response-time | Output the response time | httpx -response-time |
| response-in-json | Include response in stdout (only works with -json) | httpx -response-in-json |
| websocket | Prints if a websocket is exposed | httpx -websocket |
| ip | Prints the host IP | httpx -ip |
| cname | Prints the cname record if available | httpx -cname |
| cdn | Check if domain's ip belongs to known CDN | httpx -cdn |
| filter-string | Filter results based on filtered string | httpx -filter-string XXX |
| match-string | Filter results based on matched string | httpx -match-string XXX |
| filter-regex | Filter results based on filtered regex | httpx -filter-regex XXX |
| match-regex | Filter results based on matched regex | httpx -match-regex XXX |
Running httpx with stdin
This will run the tool against all the hosts and subdomains in hosts.txt and returns URLs running HTTP webserver.
▶ cat hosts.txt | httpx
__ __ __ _ __
/ /_ / /_/ /_____ | |/ /
/ __ \/ __/ __/ __ \| /
/ / / / /_/ /_/ /_/ / |
/_/ /_/\__/\__/ .___/_/|_| v1.0
/_/
projectdiscovery.io
[WRN] Use with caution. You are responsible for your actions
[WRN] Developers assume no liability and are not responsible for any misuse or damage.
https://mta-sts.managed.hackerone.com
https://mta-sts.hackerone.com
https://mta-sts.forwarding.hackerone.com
https://docs.hackerone.com
https://www.hackerone.com
https://resources.hackerone.com
https://api.hackerone.com
https://support.hackerone.comRunning httpx with file input
This will run the tool against all the hosts and subdomains in hosts.txt and returns URLs running HTTP webserver.
▶ httpx -l hosts.txt -silent
https://docs.hackerone.com
https://mta-sts.hackerone.com
https://mta-sts.managed.hackerone.com
https://mta-sts.forwarding.hackerone.com
https://www.hackerone.com
https://resources.hackerone.com
https://api.hackerone.com
https://support.hackerone.comRunning httpx with CIDR input
▶ echo 173.0.84.0/24 | httpx -silent
https://173.0.84.29
https://173.0.84.43
https://173.0.84.31
https://173.0.84.44
https://173.0.84.12
https://173.0.84.4
https://173.0.84.36
https://173.0.84.45
https://173.0.84.14
https://173.0.84.25
https://173.0.84.46
https://173.0.84.24
https://173.0.84.32
https://173.0.84.9
https://173.0.84.13
https://173.0.84.6
https://173.0.84.16
https://173.0.84.34Running httpx with subfinder
▶ subfinder -d hackerone.com -silent | httpx -title -content-length -status-code -silent
https://mta-sts.forwarding.hackerone.com [404] [9339] [Page not found · GitHub Pages]
https://mta-sts.hackerone.com [404] [9339] [Page not found · GitHub Pages]
https://mta-sts.managed.hackerone.com [404] [9339] [Page not found · GitHub Pages]
https://docs.hackerone.com [200] [65444] [HackerOne Platform Documentation]
https://www.hackerone.com [200] [54166] [Bug Bounty - Hacker Powered Security Testing | HackerOne]
https://support.hackerone.com [301] [489] []
https://api.hackerone.com [200] [7791] [HackerOne API]
https://hackerone.com [301] [92] []
https://resources.hackerone.com [301] [0] []📋 Notes
- As default, httpx checks for
HTTPSprobe and fall-back toHTTPonly ifHTTPSis not reachable. - For printing both HTTP/HTTPS results,
no-fallbackflag can be used. - Custom scheme for ports can be defined, for example
-ports http:443,http:80,https:8443 vhost,http2,pipeline,ports,csp-probe,tls-probeandpathare unique flag with different probes.- Unique flags should be used for specific use cases instead of running them as default with other flags.
- When using
jsonflag, all the information (default probes) included in the JSON output.
Thanks
httpx is made with
Probing feature is inspired by @tomnomnom/httprobe work
