Tuesday, 3 June 2014


In my previous post I spoke about my first attempt at Python + Elasticsearch + Kibana for cyber intelligence data, well as promised this is my followup as promised.

Today I introduce, Threatelligence.

Its pretty much as I described in the last post, my attempt at cyber intelligence gathering data then further enriching it and then displaying it in a eye candy presentation that makes it easy to view, understand and search through.

I have written the data collector in a way to make it as generic as possible so that others using the tool can easily add their own custom sources of cyber intelligence data to the environment with one single file and a regex (:

Here is a sample, botnet_feeds.ini

When installed correctly, the cron job scripts will:
  1. Fetch all the sources of data, parse, enrich it a little and store in Elasticsearch
  2. Check how many days of data to keep, remove old data from Elasticsearch
 I have included a mini how to add your own custom sources of data, a test script to test custom sources before committing it to the database and pre-configured Kibana dashboards.

When you are up and running you can create custom dashboards or add to the existing ones.

Default Dashboard:

World Map Dashboard:

It can be pretty useful in a number of cases, for instance there is a report/log of a suspicious .exe that a user downloaded. You search parts of the name of the file and find a number of sites hosting the same EXE, you have the urls, ip address and can now block it at the firewall or look for similar logs:

So this is version 0.1, it may be buggy, may be poorly coded but its pretty much working and you can grab a full copy from Github, feel free to fork and improve/add/modify:

Link: https://github.com/syphon1c/Threatelligence

Read through the install, should be straight forward!


Wednesday, 28 May 2014

Cyber Threat Intelligence...my attempt at it...

*** update to this post: http://infosecscapes.blogspot.com/2014/06/threatelligence.html

I recently started playing with various open source threat intelligence frameworks. One of which is the collective intelligence framework (CIF):

  • https://code.google.com/p/collective-intelligence-framework/

I am not going to go to much in to the technical bits but the CIF tool collects data from various open source intelligence feeds which include data such as:
  • IP Addresses
  • Domains
  • MD5 - Suspicious malware 
  • etc. theres more to it
The solution works really well but is limited on the interface side of things, but is really good at collecting, normalising data, sharing data, feeding data to appliances (FW/proxies/snort). I like it and will continue using it. Also the data is not really that rich in information

But I decided to play around and make my own threat intelligence solution type of thing. Since I started playing with Python recently I thought it was a good start. I wrote a threat feed collector that will collect data from the same/similar sources as the CIF tool does. I made it easy and generic enough to be able to add new sources as I find them.

It was a success I am collecting this data really well, but I was at a loss of how to store it, view it, make it useful...I was up all night thinking about a database structure and how not to make it horrible...

Then it clicked to me, intelligence data should not be structured, and so I finally had a reason to play with ElasticSearch:

  • http://www.elasticsearch.org/
In short (because I am only scratching at the surface), ElasticSearch is a fast, schema-less "database". I found it to be excellent at storing large data quickly and efficiently. You must really go and research it. I started with a simple article and took it from there:

  • http://joelabrahamsson.com/elasticsearch-101/

So now I am able to collect a lot of data and easily store it and access it. I took it further by enriching the data that I am collecting by looking up country, geo locations etc.

My next problem was a big one that I thought I would be spending the most time on. Which was the interface to view the data. I was thinking about the mammoth task of creating an interface that will make the data searchable, easy to view, and make it informative and eye candy friendly

....another problem solved by the ElasticSearch team, namely Kibana which recently became apart of ElasticSearch it seems. 
  • http://www.elasticsearch.org/overview/kibana/

Kibana essentially is an Dashboard interface for querying ElasticSearch data and presenting it beautifully. Within no time I was building really pretty dashboards using Kibana.

Here are some examples I built...by the way I did this all over 2 days from start to where it is now! Below you can search for the entire database for specific key data...and its fast!

The next steps are to automate the data collection, add a limit to the amount of days to keep data (you want to stay relevant and not sit with old data, especially when it comes to cyber threats.) and even add some sort of alerting.

I started the project on a saturday and got to this point by sunday and even got to spend time with the family. 

Python + Elasticsearch + Kibana = Threatelligence for the win!

I can now focus on collecting data, normalising + correlating and enriching it further...all in an automated fashion. I will push a release to Github soon.


Sunday, 11 May 2014

Tabs over Curly brackets

Python over Perl...
Last week I was talking to an infosec guy from Australia, and we briefly spoke about programming/scripting languages that we were comfortable in. I mentioned that I am comfortable with C#, bash, PHP, Perl etc. He mentioned that a few guys at his office are into Python and I replied with...."who programs in tabs it makes no sense...curly brackets for the win!" He agreed with me and couldn't understand the whole thing about tabs too...

I mean really, tabs...tabs!

Anyways it got me thinking, perhaps I should try it before I diss it! I usually need some sort of motivation to start something new, especially learning/adapting to a new programming language. So I remembered a few years back, when I worked at SensePost, at the time Roelof Temmingh was working on a research project named Setiri which he demo'ed at BlackHat. Setiri was a smart trojan at the time, it pretty much opened an invisible Internet Explorer which it hooked and used it to communicate to the Command & Controller server. It bypassed all the firewall rules and beat proxies etc. because hey it was IE and IE was allowed out...

Here's a link to a BH presentation on Setiri, click

So I set the challenge to hard and figured if I can do a little something similar in Python then maybe it would grow on me...that is...if Python could even can do it.

I installed the latest version Python (3.4) and started Googling and putting a few lines of code together...and well before I knew it...I was mostly done...and I was only 2 hours into my freshly installed version of Python...surprised...yes! disappointed...hell no!

I am going to say that I am impressed, the original Setiri Trojan was about give or take 1,000 lines of CPP, I managed a Python version which works in the exact same way in about 65 lines of code.

I am not going to call it Trojan, because I do not write them...I am interested in them yes...I like to poke at malware and understand it but just not my cup of tea...so I am going to call it an agent (: And whats really awesome is that I can bundle everything into a standalone EXE (Py2Exe/PyInstaller/PwnInstaller)...so it runs on Windows even when Python is not installed on the Windows machine.

Opening up an  invisible Internet Explorer is a simple module import and 2 lines of code, below is a sample with one or two more lines:

So obviously my version has a bit more to it...but Python really is that simple, for years people have been telling me whatever you can do in Perl or whatever other langauge you can do it in a lot less lines of Python...and I blew them off back then.

Here is just a sample of opening up Internet Explorer in CPP, 5 lines of code before you even browsed off to the command server:


Its so easy...everything is really much easier in Python, I have spent the weekend on and off in my spare time with Python and I am loving it...WHY HAS IT TAKING ME SO LONG?!?!

So I have a full on working agent and server (PHP) which runs really well, but it has spawned off a few more idea's for a testing tool (: So I have some more learning and playing to do! More on that in my next post...when I have something to show...

Sunday, 4 May 2014

oneVault - Updates

Updates on the blog about oneVault have have been few but work on bugs and feature requests have been recently working out well due to a number public holidays and long weekends giving me some time to dev.

There have been a number of bug fixes and speed improvements, but one of the new features is a pre-project scoping section. Someone asked me about getting oneVault involved before the project begins, specifically about pre-project engagement...scoping.

This is sort of my take on it...

Managers can now create dynamic project scope questionnaire templates which can be used over and over again, in the below screenshot we have a basic scope questionnaire template, easily designed in minutes. You can design questions with text, single choice or multiple choice answers, whatever suites your requirement:

When a new pentest requirement comes in the manager can select a scoping questionnaire template and send it off to the client. oneVault will send an email off to the end-client with a dynamic link to the questionnaire which the client will fill out:

The manager will be able to keep an eye on the questionnaires sent out and review completed applications

From here the Manager will be able to create a new project, assign it to a pentester and they will have access to view the scope and work accordingly within the scope.

I am pretty happy with it and its all coming along quite nicely. If you want to beta or play with it get hold of me or send me requests and idea's you would like to see in it.

Thursday, 20 March 2014

Cracking WAP2 Enterprise

In one of my recent assessments I had to look at some wireless networks at a few locations. I had not done a Wireless Security Assessment in ages so it was a good exercise to see whats changed, learn about some new authentication protocols and learn a few new attack vectors.

Everything was run from my Kali installation running on my Macbook via VMware Fusion and I used the Alfa AR 9271 Wireless USB card for WiFi (came with my wifipineapple (:  ). Yes VMware does not pickup your native built-in wifi cards, hence why I used an external USB WiFi adapter which I attached to my guest system which picks it up just fine.

All the used tools:
  • FreeRadius-WPE
  • Hostap
  • Kali
  • asleap
  • John the Ripper
My target networks were running WPA2 Enterprise, essentially users authenticate using ta username and password, in this case their domain credentials to associate to the access points. This is not as simple as cracking WPA or even worse WEP sort of attacks (which was big in my day...yeah im that old nowadays), attacking WPA2 Enterprise requires a semi-different approach. 

The approach taken was to impersonate the access points you are attacking and to run your own RADIUS server which will be used to capture password hashes. The password hashes are then cracked offline using either asleap or as I found out and worked better for me, John the Ripper.

Essentially you run hostap to impersonate the access point and run a patched FreeRADIUS server to handle the authentication, by handle I mean capture hashes of users authenticating to your spoofed access point. 


The first step is to download and compile/run/install  FreeRADIUS Wireless Pwnage Edition (WPE) which has been tweaked exactly for the purpose of capturing RADIUS authentication requests.
  • https://github.com/brad-anton/freeradius-wpe
Just follow the instructions for compiling. There are debian package you can download and simply install. You may need to make a few tweaks, I kept getting errors when trying to run the FreeRADIUS-WPE app, as the directory where the logs are stored did not exist, just a simple mkdir fixed that (:

The logs which contains the captured hashes is located here:
  •      /usr/local/var/log/radius/freeradius-server-wpe.log



Now that FreeRADIUS is installed we make sure hostap is running/installed. If you dont have it installed you can find a copy here:
  • http://hostap.epitest.fi/releases/hostapd-2.0.tar.gz
You will run hostap with a specific config file for a WPA Enterprise access point (wpe-test.conf). Hostap will turn your wifi card into an access point spoofing the access point, be sure to edit the below config to match your requirements:


  • hostapd -dd wpe-test.conf
The above example config file assumes that FreeRADIUS-WPE is listening locally. Also I noticed that my FreeRADIUS-WPE was not logging the credentials correctly when a client authenticated. After looking around I found I had to edit the following file:
  •  /usr/local/etc/raddb/modules/mschap
By adding the following line (This is specifically for WPA2 Enterprise using domain credentials for user authentication):
  • with_ntdomain_hack = yes
Once everything is up and running test the system and ensure all is good and that nothing is broken. Once it seems like all is good you should be able to simply monitor the FreeRADIUS-WPE log files for incoming logs:
  • tail -f /usr/local/var/log/radius/freeradius-server-wpe.log

mschap: Mon Feb  1 12:15:52 2014

 username: domain\testuser
 challenge: ab:bb:dd:c6:a1:13:53:dd
 response: de:14:c8:a8:7e:ce:53:5304:f8:a0:75:ce:53:53:ca:70:35:e1:e6:21:0c:f1:b6
 john NETNTLM: domain\testuser:$NETNTLM$9d0addde98e8474$d09ef8t43db04f8a075ce5353cd993571b621aeeec4


Cracking the captured Credentials

Here you can use either asleap or in my case John. I didnt have a large enough wordlist at the time of the assessment o use with asleap (wordlist based cracking). So I found that john supports the captured WPE hashes (NETNTLM), and this is accommodated by WPE. 

Simply take the NETNTLM hashes and use john to crack.

*Remember this is only for use of networks you are authorised to test and I hold no responsibility should you use this for malicious means.

Some recommendations:
  • Users should be vigilant of the networks they connecting to, in the above attack I found that most mobile devices happily connected to the "spoofed" access point with no issues, however my Macbook and Windows picked up that the certificate did not match and prompted the user with a warning. Although some users continued connection after the prompt (:  So user awareness is important, but there is no patch for human stupidity!
  • I dont think using domain credentials for authenitcation over the wireless network is the best idea, yes its practical but you are potentially broadcasting those hashes to the world and its not that hard to crack them! And who knows where else you can reuse the credentials to obtain further access. So I would suggest seperate set of credentials for authentication

Other WiFi tools for Mac OSX that I came across:

 iStumbler for Mac - http://istumbler.net/
 Kismac 2 - https://github.com/IGRSoft/KisMac2