Recently in Penetration Testing Category

Sometimes when you're testing it's good to be able to quickly get a feel for where to focus your attention or to get an overview of all the ports you've got open, so you can be sure you investigate all of them. Once you've done several scans as part of a job, you end up with a stack of nmap and nessus output files, and it can be hard to keep an eye on exactly what's been found so far, and it's good to have a way to just get the facts.

As a result a lot of testers will have scripts to help parse collate output from common tools like nmap.

They tend not to be the prettiest code in the universe or to produce lovely management friendly reports, but handy nonetheless.

Having set expectations about the quality of this code :) Here's a couple of scripts which may come in useful for testers in managing some of those xml report files.

NMAP Auto Analyzer can parse a single nmap xml file or a directory of nmap xml files and provide a concise report on ports open across them.

Nessus Auto Analyzer somewhat unsurprisingly, does the same job for .nessus report files (v2 only at the moment)

Both have reasonable help files, so should be fairly straightforward to use, any questions/queries welcome either in comments or in e-mail (rorym at mccune dot org dot uk)

We left off last time having created a simple vulnerability database using Ruby on Rails. So the next piece of the puzzle is getting that data into Dradis.

Luckily Dradis has a nice plugin system which is designed to ease the process of importing and exporting data from Dradis, so this isn't too tricky.

Creating the Plugin

As dradis has rails generators for import plugins, we can use that to create the basic structure. First off, obviously, we need a working Dradis installation to work from. There are instructions on the site for the latest svn version here and following those should give you a working version of the latest code.

Once that's done we can enter the dradis server directory an use this command to create the our import plugin.

rails generate import_plugin simple_vulndb

This creates a directory simple_vulndb_import under vendor/plugins and also creates a number of files for us to modify.

Configuring the Plugin

Here we'll just step through the bits that are necessary to get the plugin up and working. there's a number of files that we need to modify to get everything working ok. Most of this is just a modified version of the default vulndb_import plugin which is provided as part of Dradis.

First up is the configuration file in the plugin config directory.

Dradis uses YML config files which is a pretty easy syntax which is parameter : value

Here we can define the hostname, port and path for Dradis to access our vulnerability database. This also provides you the flexibility to change it (for instance if you've got a centralised version of the database as opposed to one hosted locally). The settings below are based on what we configured for the vulnerability database in the last post.

host: localhost
port: 3003
path: /vuln_search.json

with that done we can move on. Next up is the meta.rb file which can be found in lib/simple_vulndb_import/ . Here we just define the name of the plugin and the version information. So for example

  NAME = "Simple Vulnerability Database Import"
  # change this to the appropriate version
  module VERSION #:nodoc:
    MAJOR = 0
    MINOR = 1
    TINY = 0
     STRING = [MAJOR, MINOR, TINY].join('.')

would work fine. Next up the main piece we need to change, the filters.rb file. This is found in the same directory as the meta.rb file.

Creating the Filters

There's two main pieces to how I've set this up. The first is the filters. Essentially if we configure one of these for each of the search_types that we defined in the database (description, OWASP reference, Severity and Test Type) then we'll be able to search by those methods from within Dradis).

Dradis handles filters by creating a module within the filters module that you'll see pre-defined in the filters.rb template.

So for each of our searches we need to create a new module which looks a bit like this.

module TestTypeSearch
NAME = 'Search Database by Test Types'
  def self.run(params={})
    result = Filters::get_records('test_type',params['query'])
    records = Filters::prepare_results(result.body)
    return records
  end
end

what we're doing here is essentially setting up a NAME constant which contains (rather unsurprisingly) the filter name. then defining the behaviour when the filter is run. this is rather short as we're just calling two class methods and then returning the result.

When I was writing this file I realised that I was essentially just writing variations of the same logic four times, so in good ruby practice I tried to DRY up the code and moved most of the logic into the class methods get_records and prepare_results

get_records looks like this

def self.get_records(search_type,query)
  require 'cgi'
  conf_file = File.join(Rails.root, 'config', 'rvulndb_import.yml')
  conf = YAML::load( File.read CONF_FILE )
  http = Net::HTTP.new(conf['host'], conf['port'])      
  res = http.get(conf['path'] + '?search_type=' + search_type +'&query=' + CGI::escape(query))
end

So this method opens the configuration file that we defined earlier (you'll notice that it looks in the config directory under the rails root, so it's a good idea to put a copy in there). Once it's opened that it uses rubys' YAML class to read the file, sets up an http connection to the database mentioned in the config file and executes the query on the database. One thing to note here is the use of CGI::escape. This helps manage any use of characters that aren't allowed in URLs in our query string.

Ok, so after that method has completed we should have an array of 0 or more records that we can setup to be returned into dradis.

Next method up preps the records for input into Dradis

def self.prepare_results(json_data)
  recs = []
  jrec = ActiveSupport::JSON::decode(json_data)
  if jrec.length == 0
    error = Hash.new
    error['title'] = "No records found"
    error['description'] = "The search didn't return any records!"
    recs << error
    return recs
  end
  
  jrec.each do |jr|
    newrec = Hash.new
    newrec['title'] = jr['vulnerability']['title']
    newrec['description'] = Filters::build_description(jr['vulnerability'])
    recs << newrec
  end
  return recs
end

So this code just loads up the JSON data that our query should have returned, checks to make sure that we got some records (and returns an error if we didn't) then creates a hash for each record. There's one more bit of logic to explain in here which is the call to Filters::build_description. For neatness sake I broke that bit out. At the moment it's a pretty ugly text creation, but does the job :)

def self.build_description(note_data)
      <<-eos
Vulnerbility Title
------------------
#{note_data['title']}      
      
Vulnerability Description
------------------------
#{note_data['description']}
        
        
Vulnerability Remediation
-------------------------
#{note_data['remediation']}
        
        
Technical Notes
--------------
#{note_data['technical_notes']}
      eos
end

This just puts together the body of the note description for each finding, as one long string.

There's obviously a lot more that could be done with this (like better error handling and writing tests) but with those files complete, the module should work ok and you should be able to import vulns from your database directly into Dradis using the "import note" feature.

I've put a copy of the code for the plugin up here, in case it's helpful :)

One of the main tools that I've found useful in pen. testing is the Dradis Framework, it's a good way of keeping track of findings and notes during a test and I've also found it's template feature is good for keeping a list of things to remember during a test.

One of the features available in Dradis is import plugins. This lets you create a link to an external information source, such as a OSVDB or a database of vulnerabilities.

Having a database of vulnerabilities or findings can be pretty useful in cutting down the time required for reporting on a test as you can keep standard wordings in place (who really wants to write the same section about preventing XSS more than once!).

So recently I knocked up a simple vulnerability database to link in to Dradis and I thought it might be of use, so here's the process.

Creating the App

We're going to use Ruby on Rails for this as it's nice and easy to develop for (as you'll see) and also that's what Dradis is based on, so makes sense to keep all the coding in the same underlying language. Also rails apps are very portable, they're basically contained within a single directory structure, so it's relatively easy to move them from place to place.

Before starting the application, there's the usual pre-req's. I'm using Ruby 1.9.2 and Rails 3 so having those installed is a good thing. If you're using Linux then it's helpful to have RVM working as some distros don't have ruby 1.9.2 packaged up as yet.

once you've got the pre-req's working, we can start by creating a rails app

rails new vulnlist

This creates a new application called vulnlist and adds all the standard rails files in.

Creating the Scaffold

Once we've got the app created, we can use rails scaffolding to quickly create the basic structure for our app. The web pages that scaffolding creates aren't the most pretty, but they'll do for now.

With the scaffold we can specify what fields we want to create in the database and also what data types those fields are.

rails generate scaffold Vulnerability title:string test_type:string description:text remediation:text technical_notes:text severity:string owasp_reference:string

Once we've completed this we can look at the basic app by setting up the database with

rake db:migrate

Ensuring that all our gems are installed ok with

bundle install

and launching the app

rails server

At this point browsing to http://127.0.0.1:3000/vulnerabilities should show a blank page with our fields in it. From this page we can create new vulnerabilities and edit or delete existing ones.

Now that we've got this basic structure setup it's worth using git to keep a handle on the source code. On Linux the procedure for this is pretty easy

If you've not already got it installed

sudo apt-get install git-core

then in the root of the application

git init
git add .
git commit -m "Initial Commit with Scaffold"

Having git running on the app will make it pretty easy to revert any mistakes made along the way, as long as we've done regular commits.

Setting up the Searches

So far we've got a basic structure in place and can do the basic Create, Read, Update, Delete cycle on our data. However for the dradis integration, what'd be useful is if we could search for vulnerabilities using various criteria and have the results returned to Dradis.

This turns out to be relatively straightforward. First what we need is a new action in our controller. Opening vulnlist/app/controllers/vulnerabilities_controller.rb we can see the existing actions that we've got for the application.

What we need to do now is add a new action to allow for vulnerability searches.

def vuln_search
    case params[:search_type]
    when "description"
      @vulnerabilities = Vulnerability.where("description like ?", "%"+params[:query]+"%")
    when "owasp"
      @vulnerabilities = Vulnerability.where("owasp_reference like ?",params[:query]+"%") 
    when "severity"
      @vulnerabilities = Vulnerability.find_all_by_severity(params[:query])
    when "test_type"
      @vulnerabilities = Vulnerability.find_all_by_test_type(params[:query])
    end
    respond_to do |format|
      format.xml {render :xml => @vulnerabilities}
      format.json {render :json => @vulnerabilities}
    end
  end

This defines a new method called "vuln_search" which takes two parameters, search_type and query. The search type parameter lets us pick from different finders. Rails provides access into the application database via ActiveRecord and this just uses a couple of the finder types for different parameters. Where the query is going to be one of a number of fixed values like "severity" which will be something like high, medium or low, we can just use a standard find_all_by_ approach, but where it's a more free text style search, we use Vulnerability.where and pass in the query parameter that way.

The respond_to block is a really nice feature of rails. By adding in the two lines for :xml and :json rails wires up responses so that we can get the data out in those format, no additional code required.

Now that we've got the basic code in place, we just need to modify the rails routes so that the application knows how to access our new method.

This is done by modifying the vulnlist/config/routes.rb file, and adding the following code

controller :vulnerabilities do
  get 'vuln_search' => :vuln_search
end

At this point, we've got the application basically working. If you put in a couple of test findings, then you should be able to go to http://127.0.0.1:3000/vuln_search.xml?search_type=Severity&Query=High for example and get some XML data back.

Tidying up

Now that we've got the basics working, there's a couple of additional steps that it's worth looking at to tidy some things up.

Selectors

First off, we'd like some of our fields (OWASP Reference, Severity and test type) to be one of a number of defined values. The "proper" way to do this would be to create additional models for these and link them in to the main vulnerabilities controller, but there's a quicker way which is probably going to work well enough for our purposes.

Opening up vulnlist/app/models/vulnerability.rb we can specify some Constant values for these settings

TEST_TYPES = ["Web Application","Windows Server","Unix Server","Wireless","Web Server","Oracle","MS SQL","MySQL","DB2"]
SEVERITY_LEVELS = ["Critical","High","Medium","Low","No Impact"]
OWASP_TOP_10 = ["A1 - Injection","A2 - Cross Site Scripting (XSS)","A3 - Broken Authentication and Session Management","A4 - Insecure Direct Object Reference","A5 - Cross-Site Request Forgery (CSRF)","A6 - Security Misconfiguration","A7 - Insecure Cryptographic Storage","A8 - Failure to Restrict URL Access","A9 - Insufficient Transport Layer Protection","A10 - Unvalidated Redirects and Forwards"]

Then we can modify the form that the scaffolding process created to use these arrays as a select list. The form is found in vulnlist/app/views/vulnerabilities/_form.html.erb. In that file we just need to replace the "text_field" lines for those three fields with the following select lines

<%= f.select :test_type, Vulnerability::TEST_TYPES, :prompt => "Select the test type" %>
<%= f.select :severity, Vulnerability::SEVERITY_LEVELS, :prompt => "Select the severity level" %>
<%= f.select :owasp_reference, Vulnerability::OWASP_TOP_10, :prompt => "Select the appropriate OWASP top 10 reference" %>

This picks up the Constants from our model and helps keep the data consistent.

Localhost Only

As you'll have noticed with this application, there's pretty much no security whatsoever. At the moment it's setup as a personal database only and isn't suitable to be on any kind of network. Adding that security isn't too difficult with rails, however it's not really a problem for the basic use case that we have here. Both the vulnerability list and the dradis installation only need to listen on the localhost.

Configuring rails to only listen on the localhost (as opposed to specifying it on the command line) is a bit hacky, but here's a way to do it based on this post and this dradis change . We need to modify the vulnlist/script/rails file and add the following lines

require 'rubygems'
require 'rails/commands/server'
require 'rack'
require 'webrick'

module Rails

  class Server < ::Rack::Server

    def default_options

      super.merge({

        :Port => 3003,

        :Host => "127.0.0.1",

        :environment => (ENV['RAILS_ENV'] || "development").dup,

        :daemonize => false,

        :debugger => false,

        :pid => File.expand_path("tmp/pids/server.pid"),

        :config => File.expand_path("config.ru")

      })

    end

  end

end

This also moves the application off the default port of 3000, to a new one of 3003 which hopefully shouldn't clash with other services.

Default Routes

At the moment if we visit the root page of our application, now at http://127.0.0.1:3003 we get the default rails welcome page. What'd be nicer is if we were re-directed to the vulnerability listing automatically.

That's easily done with two steps. First edit the vulnlist/config/routes.rb file and add the line

  root :to => "vulnerabilities#index"

then delete the file vulnlist/public/index.html file.

Summary

So at the end of this first part we've created a basic vulnerability database which we can search easily on a number of parameters.

The next step is to create the dradis plugin to hook the two together, which as I'll cover next time is a reasonably easy thing to do.

I've been meaning to do a post on burp for a while, and courtesy of my new years resolution to stop procrastinating, here it is :)

I was thinking of a way to sum up burp, so far the best I've got is "If you're doing web application testing and not using Burp, you're missing out !", it is that useful...

Burp has been around for a while now as one of a group of handy web application proxy tools along with the likes of Web Scarab and Paros. Over time it's developed a huge range of handy features, which make web app. testing a lot more productive.

There's the basics, in terms of intercepting and modifying requests, which all the tools in this category tend to do quite well, but on top of that there's just loads of other things that Burp does.

The Web app scanner is a good targetable way to test for several types of common web app vulnerabilities. As an add-on in the latest beta, there's a wizard which lets you reduce the number of requests that Burps going to make, by removing duplicates and similar requests. If you've used many commercial web app. scanners you'll know that volume of requests generated can be a real problem, so anything that can help bring down the load is useful.

Another stand-out feature of Burp for me, is intruder which lets you pick a specific parameter or set of parameters from a request and easily modify them. As an example pretty much every time I see a URL that looks like request.asp?id=123 , it's going to be worth running burp Intruder over that parameter to see what pops up when you change that value.

One of the things I have found about burp is that it's easy to miss functions that are available and not get the best out of it. Luckily there's a blog which covers a lot of the new features as they come out. Also I need to mention the Burp Tip of the Day series of posts on Cktricky's blog which has loads of good tips on getting more out of Burp.

I've been using Dradis for a couple of months now. It's an interesting piece of software that's designed to help teams of people share information on penetration tests.

That said I find it useful on the tests I do even when I work alone.

Essentially Dradis provides the base environment for users to work in, either from the command line or the rails based web interface, and then pretty much lets them get on with it, so it can be a bit difficult to work out how you want to use it at first, but once you get used to it it's very handy!

What I've done is started to create templates for information that I need to collect during a test and areas to review, that way it reduces the number of times I get into writing the report and realised I've missed something ;o). Once you create the tree structure in Dradis, containing any information you want, you can export it as a single file and then re-use it as a template on future tests (for example one for web application tests, one for infrastructure etc)

you can also import information from other tools (nmap, nikto, nessus and burp at the moment) into Dradis, meaning you've got a single place to view all the information relating to the test.

It's well worth a look, as it's under active development there are new features coming pretty frequently. Also there are guides on the site for prospective developers and as it's all done in ruby it's nice and easy to start coding for!

After encountering some SNMPv3 servers recently and looking into the differences from a pen. test perspective, I thought it may be worth a quick write-up.

SNMPv1 and v2 do not respond when traffic is sent their way unless there is a valid community string in the message, a fact used by scanners like onesixtyone . So traditionally the theory is unless there's a known community string, the service running has a vulnerability or you can get in-line to sniff traffic, there's not a lot to get from SNMP services.

Turns out that SNMPv3 behaves differently from v1 and v2. Firstly the notion of using community strings for authentication is gone, replaced by username/passwords. Second the traffic can be encrypted to limit sniffing opportunites.

However it's not all bad from a testers perspective! unlike earlier versions SNMPv3 will respond to correctly formatted requests and provide some information about itself as part of the reply. This allows confirmation of the servers existence.

To get these responses there's a couple of different tools we can use. Nmap with version detection will confirm that a SNMPv3 service is running and looking at the traffic in wireshark

From this there's a couple of interesting pieces of information. The Engine Enterprise ID field seems to identify the server type that's running, net-snmp in this case and the msgAuthoritativeEngineTime parameter shows the time in seconds since the service was started (according to this page ).

In addition to using nmap, it's possible to use the inbuilt snmp tools to get some information out of the service including possible username enumeration and brute-force password attacks.

Issuing the snmpwalk command with an invalid username like so:

snmpwalk -v 3 -n '' -l noAuthNoPriv -u "invaliduser" 192.168.207.142 IF-MIB::ifName

provides the response snmpwalk: Unknown user name

but if we use a valid username and no password like so:

snmpwalk -v 3 -n '' -l noAuthNoPriv -u "snmpUser" 192.168.207.142 IF-MIB::ifName

we get Error in packet. Reason: authorizationError (access denied to that object)

So it's possible by parsing responses to figure out valid usernames for the service.

Update : Here's a ruby script (should work on linux with snmp tools and ruby installed) which iterates over a list of usernames and a list of IPs and attempts to guess whether the username is valid or not snmpv3enum.rb

A similar technique works with specification of passwords which would allow for brute-forcing those as well, (although that said the snmp tools try to stop people choosing passwords less than 8 characters, so unless a dictionary word is used it isn't too likely to be successful.

Additionally for people who're fond of metasploit, I've knocked up a very basic SNMPv3 scanner. At the moment all it does it take a range of IP addresses and say whether a valid SNMPv3 packet provokes a response from the server, but could be handy. it's here.

There's some good references on setting up and using SNMPv3 here, here and here.

So I got across to Defcon this year (and of course security Bsides)

There were several interesting presentations , here's some notes on some of the ones that I got along to. The CD is up at defcon.org, and you can get the slides for most of the presentations there.

SSL Talks - There were several presentations on ways to bypass or subvert SSL. From my point of view the drive was around 2 themes. The difficulty in correctly parsing SSL certs due to the complexity of the relevant RFCs and also the weaknesses in the issuing processes of some CAs (which is exacerbated by the fact that any CA can issue a "valid" cert. for any domain). Dan Kaminsky's white paper on the subject is here, Moxie's blackhat presentation is here and Michael Zusmans' blackhat presentation is here (the defcon presentations are on the CD linked above).

Metasploit - The Metasploit team had a whole track to themselves on Saturday and there's a load of interesting stuff in there, basically looks like Metasploit will have some coverage for most types of assessment eventually, which is very handy!

Egypts presentation on client-side attacks with Metasploit had some great content on browser fingerprinting (not just relying on User-Agent) and client side attacks with Metasploit.

The Oracle presentation from mc and Chris Gates had some good content on going through the steps of an Oracle attack purely using Metasploit (more information on this on CG's blog

Druid had a good talk on the war-dialer that he's added to the framework (unfortunately the slides for that one aren't on the CD, but I'd guess that they'll be around soon). Another interesting thing in the Metasploit track was the demos of the Mac OSX and POSIX versions of meterpreter which are both in active development.

Other Ruby Tools - There was a bit of a theme of ruby tools across some of the presentations. In addition to all the Metasploit stuff, there was a mobile device fuzzer (although the website for it, www.fuzzit.com isn't up yet). Also there was a good presentation on Dradis from etd , which is a Pen test management and reporting tool written in Ruby and Rails.

Password Cracking - There was a good presentation, from Matt Weir, on approaches to password cracking, speeding up brute-force attacks and picking good dictionary files . There's more informaton on this one on his blog as well as on the presentation slides.

Router Hacking - FX had some more information on how to create reliable exploits on Cisco devices and the difficulties in doing so (amazing to think that Cisco Maintain thousands of discreete monolithic router builds!). Slides from the Blackhat presentation are here and White paper here

USB Hacking - Interesting presentation from a guy from MWR on how they approached an assignment to assess the security of a USB device and the approaches to fuzzing that they took. Slides are up on their site here