Pentesting Notes
  • Home
  • 🌐Web pentesting
    • Content Discovery
    • Subdomain Enumeration
    • Authentication bypass
    • IDOR (Insecure Direct Object Reference)
    • Git repository
    • XSS
    • SSRF
    • CSRF
    • Injection
      • SQL Injection
      • Cypher injection
      • Command injection
      • Server Side Template Injection
      • NoSQL injection
      • XXE
    • FI (File Inclusion)
    • File upload
    • OAuth
    • JWT
    • CORS
    • Prototype pollution
    • Request Smuggling
  • Windows Pentesting
    • Enumerating users (No credentials)
    • Privilege Escalation
    • Post-Exploitation
    • Cross-domain enumeration
    • LDAP port (389, 636, 3268, 3269)
    • SMB port (139,445)
    • MSSQL port (1433)
    • Certificate Authority (CA)
    • Delegation attacks
    • Attacking Kerberos
    • Relay attacks
    • Bypassing Security
    • File Transfer
    • GPO (Group Policy Object)
    • Tools
      • Mimikatz
      • NetExec
      • Crackmapexec (CME)
      • Powerview
      • Bloodhound
      • Impacket
      • BloodyAD
      • Sliver C2
  • 🐧Linux Pentesting
    • Linux Privilege Esclation
    • Escape docker
    • Ansible
  • 🕊️Cross platform pivoting
    • Pivoting
  • ☁️Cloud
    • Kubernetes
    • Azure
      • Architecture
        • RBAC & ABAC roles
        • Entra ID roles
        • Entra ID - Authentication with OAuth and API's
        • Consent and Permissions
      • Service Discovery, Recon, Enumeration and Initial Access Attacks
        • Unauthenticated Recon
        • Password Spraying
        • Azure App Service
        • Azure Blob Storage
        • Phishing with Evilginx
        • Conditional Access
      • Authenticated Enumeration
        • ROADTools
        • BloodHound & AzureHound
        • Storage Accounts (database)
        • MFASweep
        • GraphRunner
        • Azure SQL databases
      • Privilege Escalation
        • Illicit Consent Grant
        • Macro enabled Word-files (Revshell)
        • Add secrets to app
        • Automation Accounts & Function Apps
        • Virtual Machines
        • Key Vault
        • ARM Deployment History
        • Enterprise Application / Service Principal
      • Lateral Movement
        • Entra ID Devices & Primary Refresh Tokens
        • Dynamic Groups
        • Application Proxy
        • Hybrid Identity
  • 🔁Reversing
    • Windows executables and DLL's
    • Linux binaries
    • Java applications
    • Android APK
  • 🛜Wireless networks
    • WPA/WPA2
    • WPS
    • WEP
    • Capative portal bypass
    • Setting up a Rogue Access Point
    • WPA Enterpise (WPA-MGT)
  • ⭐Tips and tricks
    • Tips and tricks
Powered by GitBook
On this page
  • Robots.txt
  • Favicon
  • Sitemap.xml
  • HTTP Headers
  • Wappalyzer extension
  • Wayback machine
  • S3 Buckets
  • Automated tools
  • ffuf
  • dirb
  • gobuster
  1. Web pentesting

Content Discovery

Discovering content on the website is a important step in the reconnaissance phase of web pentesting.

PreviousHomeNextSubdomain Enumeration

Last updated 9 months ago


Table of contents:


Robots.txt

The robots.txt file is a document that tells search engines which pages they are and aren't allowed to show on their search engine results or ban specific search engines from crawling the website altogether.

http(s)://site.com/robots.txt

Favicon

Command:

curl <https://static-labs.tryhackme.cloud/sites/favicon/images/favicon.ico> | md5sum

Sitemap.xml

Unlike the robots.txt file, which restricts what search engine crawlers can look at, the sitemap.xml file gives a list of every file the website owner wishes to be listed on a search engine.

http(s)://site.com/sitemap.xml

HTTP Headers

When we make requests to the web server, the server returns various HTTP headers. These headers can sometimes contain useful information such as the webserver software and possibly the programming/scripting language in use.

Command:

curl <http://10.10.252.141> -v

Wappalyzer extension

Find out the technology stack of any website. Create lists of websites that use certain technologies, with company and contact details. Use our tools for lead generation, market analysis and competitor research.

Wayback machine

Search cached websites and webpages using the wayback machine.

S3 Buckets

S3 Buckets are a storage service provided by Amazon AWS, allowing people to save files and even static website content in the cloud accessible over HTTP and HTTPS. The owner of the files can set access permissions to either make files public, private and even writable. Sometimes these access permissions are incorrectly set and inadvertently allow access to files that shouldn't be available to the public.

http(s)://{name}.s3.amazonaws.com

Automated tools

ffuf

ffuf -w /usr/share/wordlists/SecLists/Discovery/Web-Content/common.txt -u <http://10.10.252.141/FUZZ>

dirb

dirb <http://10.10.252.141/> /usr/share/wordlists/SecLists/Discovery/Web-Content/common.txt

gobuster

gobuster dir --url http://10.10.252.141/ -w /usr/share/wordlists/SecLists/Discovery/Web-Content/common.txt

finding files:

gobuster dir -u 10.10.10.191/ -w /usr/share/seclists/Discovery/Web-Content/raft-medium-files.txt

Find API endpoints using all HTML responses

ffuf -w ~/Tools/SecLists/Discovery/Web-Content/big.txt -u <http://prd.m.rendering-api.interface.htb/api/FUZZ> -mc all -fs 50 -X POST
🌐
https://wiki.owasp.org/index.php/OWASP_favicon_database
Find out what websites are built with - Wappalyzer
Internet Archive: Wayback Machine
Logo
Logo