OSINT With Datasploit
With all this talk of shifting security left, it's important for developers to know that different tools available for security testing. Today, we check out Datasploit.
Join the DZone community and get the full member experience.Join For Free
I was reading an interesting Motherboard article, Legal Hacking Tools Can Be Useful for Journalists, Too, that includes a reference to one of my all-time OSINT favorites, Maltego. Joseph Cox's article also mentions Datasploit, a 2016 favorite for fellow tools aficionado, Toolswatch.org, see 2016 Top Security Tools as Voted by ToolsWatch.org Readers. Having not yet explored Datasploit myself, this proved to be a grand case of "no time like the present."
Datasploit is "an #OSINT Framework to perform various recon techniques, aggregate all the raw data, and give data in multiple formats." More specifically, as stated on Datasploit documentation page under Why Datasploit, it utilizes various Open Source Intelligence (OSINT) tools and techniques found to be effective and brings them together to correlate the raw data captured, providing the user relevant information about domains, email address, phone numbers, personal data, etc. Datasploit is useful to collect relevant information about a target in order to expand your attack and defense surface very quickly.
The feature list includes:
- Automated OSINT on domain/email/username/phone for relevant information from different sources.
- Useful for penetration testers, cyber investigators, defensive security professionals, etc.
- Correlates and collaborate results, shows them in a consolidated manner.
- Tries to find out credentials, API keys, tokens, subdomains, domain history, legacy portals, and more as related to the target.
- Available as a single consolidating tool as well as standalone scripts.
- Performs Active Scans on collected data.
- Generates HTML and JSON reports along with text files.
YouTube: Quick guide to installation and use
Second, a few pointers to keep you from losing your mind. This project is very much a work in progress, with lots of very frustrated users filing bugs and wondering where the support is. The team is doing their best, be patient with them, but read through the GitHub issues to be sure any bugs you run into haven't already been addressed.
1) Datasploit does not error gracefully, it just crashes. This can be the result of unmet dependencies or even a missing API key. Do not despair, take note, I'll talk you through it.
2) I suggest, for ease, and best match to documentation, run Datasploit from an Ubuntu variant. Your best bet is to grab Kali, VM, or dedicated and load it up there, as I did.
3) My installation guidance and recommendations should hopefully get you running trouble free, follow it explicitly.
4) Acquire as many API keys as possible, see further detail below.
Installation and Preparation
From Kali bash prompt, in this order:
- git clone https://github.com/datasploit/datasploit /etc/datasploit
- apt-get install libxml2-dev libxslt-dev python-dev lib32z1-dev zlib1g-dev
- cd /etc/datasploit
- pip install -r requirements.txt
- mv config_sample.py config.py
- With your preferred editor, open config.py and add API keys for the following, at a minimum. They are for all intents and purposes required, and detailed instructions to acquire each is here:
- Shodan API
- Censysio ID and Secret
- Clearbit API
- Emailhunter API
- Fullcontact API
- Google Custom Search Engine API key and CX ID
- Zoomeye Username and Password
If, and only if, you've done all of this correctly, you might end up with a running instance of Datasploit. Seriously, this is some of the glitchiest software I've tussled with in quite a while, but the results paid handsomely. Run python datasploit.py domain.com, where domain.com is your target. Obviously, I ran python datasploit.py holisticinfosec.org to acquire results pertinent to your author.
Datasploit rapidly pulled results as follows:
211 domain references from Github:
Luckily, no results from Shodan.
Four results from Paste(s):
Pastebin and Pastie results
Datasploit pulled russ at holisticinfosec dot org as expected, per email harvesting.
Accurate HolisticInfoSec host location data from Zoomeye:
Details regarding HolisticInfoSec sub-domains and page links:
Sub-domains and page links
Finally, a good return on DNS records for holisticinfosec.org and, thankfully, no vulnerabilities found via PunkSpider.
Datasploit can also be integrated into other code and called as individual scripts for unique functions. I did a quick run with python emailOsint.py firstname.lastname@example.org and the results were impressive:
I love that the first query is of Troy Hunt's Have I Been Pwned. Not sure if you have been? Better check it out. A reminder here, you'll really want to be sure to have as many API keys as possible or you may find these buggy scripts crashing. You'll definitely find yourself compromising between frustration and the rapid, detailed results. I put this offering squarely in the "shows much promise category" if the devs keep focused on it, assesses for quality, and handles errors better.
Give Datasploit a try for sure.
Cheers, until next time.
Published at DZone with permission of Russ McRee, DZone MVB. See the original article here.
Opinions expressed by DZone contributors are their own.