recon tool suite

Based on the services identified on a network host, we often need to run specific tools to assess the security/configuration of these services. The recon tool suite can help you automate that and analyze/summarize the results.

motivation

Instead of manually running various tools (e.g. testssl.sh, Nikto, feroxbuster, etc.) and having to remember all commands and the necessary options, we can configure them once (see config/scanner.toml) and have the scanner (i.e. scan.py) run the required/appropriate tools based on what the Nmap service scan (e.g. services.xml) has found. In addition to that, the suite also provides a tool to analyze and summarize the results of some scans (e.g. HTTP response headers, various protocol-specific configurations, etc.). This allows for an automated and consistent assessment of specific services (i.e. no longer are testers forced to analyze configurations by hand).

installation

The tools in this suite (i.e. analyze.py and scan.py) require Python 3.11+, and the defusedxml and Jinja modules.

Clone the repository:

# this step is optional; change the directory name to your preference
mkdir --parents $HOME/tools && cd $_

git clone https://github.com/4elta/recon.git
cd recon

To install (and update) the required tools, you can use the provided script:

./required-tools.sh install

The script will install the following tools via package manager if possible, otherwise from GitHub to $HOME/tools:

• (GNU) awk
• BIND 9
• curl
• enum4linux-ng
• IKE scanner
• NFS support
• Nmap
• Nikto
• Python libraries
  • defusedxml
  • dnspython
  • Impacket
  • Jinja
• Remote Method Guesser
• RPC support
• Samba client
• SecLists
• SIPVicious
• testssl.sh
• WhatWeb

Then add symbolic links to the scripts to /usr/local/bin. Please make sure, that the names for analyze and scan don't conflict with any binaries already installed.

sudo ln --symbolic "$(realpath analyze.py)" /usr/local/bin/analyze
sudo ln --symbolic "$(realpath scan.py)" /usr/local/bin/scan

usage

1. run your "standard" Nmap port and service scan(s); make sure to use -sV and -oX services.xml for the service scan
2. run the scanner on the results of the Nmap service scan; be aware that this (like the Nmap scan itself) will send requests to the target system(s)
3. run the analyzer on the results of the scanner; no network traffic will be generated during this step

You can customize the configuration for the scanner (i.e. what tools to run, etc.) by specifying your own with the --config argument (see config/scanner.toml). Similarly, you can modify the recommendations based on what the analyzers will evaluate certain services. Make sure to have a look at the architecture documentation and/or study the provided configuration files.

scanner

% scan -h
usage: scan [-h] [-o path] [-c path [path ...]] [-t number] [-s number] [-m seconds] [-n] [-f key=regex [key=regex ...]] [-y] [-d character] [--ignore-uid] path [path ...]

Schedule and execute various tools based on the findings of an Nmap service scan.

positional arguments:
  path                  path to the Nmap scan result file (e.g. 'nmap/services.xml')

options:
  -h, --help            show this help message and exit
  -o, --output path     path to where the results are stored (default: './recon')
  -c, --config path [path ...]
                        path to the scanner configuration files (default: '/home/user/tools/recon/config/scanner.toml')
  -t, --concurrent-targets number
                        number of targets that should be scanned concurrently (default: 3)
  -s, --concurrent-scans number
                        number of scans that should be running concurrently on a single target (default: 2)
  -m, --max-time seconds
                        maximum time in seconds each scan is allowed to take (default: 3600)
  -n, --dry-run         do not run any command; just create/update the 'commands.csv' file
  -f, --filter key=regex [key=regex ...]
                        only scan specific services that match all provided filters ('key' can be 'host', 'protocol', 'port' or 'service'); existing result files will be overwritten
  -y, --overwrite-results
                        overwrite existing result files
  -d, --delimiter character
                        character used to delimit columns in the 'commands.csv' and 'services.csv' files (default: ',')
  --ignore-uid          ignore the warning about potentially lacking permissions

After running the scanner, the results directory (e.g. recon/) will contain the following files/directories:

• commands.csv: contains information about the executed commands (incl. start time, end time and return code)
• config_<YYYY-mm-dd_HH-MM-SS>.json: the configuration used for the scan; this can be used in re-scans (-c path/to/this/config.json)
• scanner_<YYYY-mm-dd_HH-MM-SS>.log: the debug/error log of the scanner
• services.csv: contains information about the identified services (incl. whether they have been scanned or not)
• <IP address>/: each host has its own directory where the result files of the various tools are stored
  • the result files follow a specific naming scheme: <service>[,<transport protocol>,<port>,...],<tool>.<ext>
• <IP address>.md: this file contains a list of services identified on this particular host

analyzer

% analyze -h
usage: analyze [-h] [-s code] [-S name] [-r path] [-i path] [-l code] [-f code] [-t path] [-o path]

Analyze and summarize the results of specific tools previously run by the scanner of the recon tool suite (i.e. 'scan').

options:
  -h, --help            show this help message and exit
  -s, --service code    service that should be analyzed (choices: ['dns', 'ftp', 'http', 'isakmp', 'ntp', 'rdp', 'smb', 'snmp', 'ssh', 'tls'])
  -S, --scan name       name of the tool/scan whose results should be parsed
  -r, --recommendations path
                        path to the recommendations document (default: '/path/to/recon/<service>/recommendations/default.toml')
  -i, --input path      path to the root directory that holds the results to be analysed (default: './recon')
  -l, --language code   language of the analysis (default: 'en')
  -f, --format code     format of the analysis (choices: ['csv', 'json', 'md']; default: 'md')
  -t, --template path   path to the Jinja2 template for the analysis; this option overrides '-f/--format'
  -o, --output path     path to the directory where the analysis result(s) will be saved

The following analyzers (and parsers) are currently implemented:

• DNS configuration (nase, nmap)
• FTP configuration (nmap)
• HTTP response headers (curl, nmap)
• ISAKMP/IKE configuration (ike)
• NTP configuration (nmap, ntp)
• RDP configuration (nmap)
• SMB (and some AD) configuration (enum4linux-ng, nmap)
• SNMP configuration (nmap)
• SSH configuration (nmap)
• TLS configuration (nmap, sslscan, sslyze, testssl)

The following languages are currently available for the analysis:

• de: Deutsch
• en: English

The analyzer can print its results in Markdown, JSON or CSV. If you need the analysis in a markup format other than Markdown, just pipe the output of the analyzer to pandoc and you are good to go. Below is an example of a conversion to docx:

$ analyze [...] | pandoc --from=markdown --to=docx --output="/path/to/analysis.docx"