Getting Acquainted With MISP

I had an item on my To-Do list to learn and get acquainted with MISP for months now so when I had a couple of free hours and nothing else to do, I decided to finally do it and learn the power and capabilities of MISP platform.

MISP is a free and open-source threat intelligence platform for gathering, sharing, storing and correlating Indicators of Compromise of targeted attacks, threat intelligence, financial fraud information, vulnerability information or even counter-terrorism information. (Link)

Objectives

Install and configure a MISP instance
Subscribe to some feeds to get information into the instance.
Get acquainted with the theoretical aspects of the platform.
Do some scripting using PyMISP
Learn what possible integrations are available.

Installing MISP

There are so many ways one can build a MISP instance, pick your favourite and run it. I went with the automated installer for Ubuntu 20.04. I spun a VPS and ran the script on it. It takes 10-15 mins to install everything needed on the server and start the services. Once the script is done, you can now navigate to your instance URL and login in using the default credentials (which you will be forced to change).

Sidenote: You might need to request an SSL certificate from Let's Encrypt and modify the Apache2 configuration file before you can establish a secure connection to the instance. For local instances using docker, follow the instructions.

Subscribing To Feeds

Feeds are integral parts of using MISP, whether you want to read data published by other organizations around the world, to contribute to that data or even publish your data, feeds are what you will be working with.

Subscribing to a feed is simple. Login in using your account then select Sync Actions from the top menu. In that menu, select List Feeds. From there you can both see available feeds, enabled them, and fetch the data available on them.

I subscripted to two different feeds and fetched the events available at the moment, when I go to the home page, this is what I see (list of events from these feeds).

Deep Dive Into Events

Let's take a deeper look at one event and see what data it tells us and what fields are available for those publishing similar events. It's important to know that the structure of MISP events is defined in the MISP Core format and can also be extended through MISP taxonomies opening the possibility for all kinds of data representation.

The picture above shows the metadata for one event which tells us quite a lot about the event. Let's break it down:

A couple of IDs
The organization that created the event?
The user who created the event?
Tags (quite useful).
Dates, lots of them.
The threat level for the event
Number of attributes
Number of sightings.

When you scroll down, you get the event content which could be lots of data depending on the source organization, event type, and the data available. Here is a snippet from the data section for the event we are discussing.

There are many things to try and even more things to learn about the platform, in the next section I will sum up my basic understanding for how MISP works (in theory). I will write more posts on specific features or use cases in the near future.

Events are the building block upon which the entire platform relies, everything else is used to enhance, extend, categorize and modify these events.
An event can either use the core format or any of the taxonomies for extended attributes.
You can attach a galaxy to an event which will link the information about that galaxy and make it easier to categorize the events or use them.
Events can be grouped into feeds that can be kept private or shared with others.
If a user is reading events from a feed and notices an attribute they have seen or detected in their infrastructure, they can add a sighting to that event which will validate the data and help others filter noise and unverified events.

This is the most simplified explanation I can think of at the moment and I will keep refining it as I learn more about the internals of the platform.

Tinkering With PyMISP

I am a sucker for any good library that makes my life easier and helps me automate things, PyMISP definitely sounds like one of these libraries.

PyMISP is a Python library to access MISP platforms via their REST API. PyMISP allows you to fetch events, add or update events/attributes, add or update samples or search for attributes.

Installing PyMISP is straightforward and well documented, all you need to do is run the following command:

pip3 install pymisp

Ones that's done, you are ready to start having fun. There are a bunch of examples included in the PyMISP repo, I ran one of them just to test everything works nicely, and it sure does. The response was a bit too verbous but it suffices for the purpose at hand.

The code for this last.py script is simple and elegant, here it is with some added comments:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

""" Import stuff """
from pymisp import ExpandedPyMISP
from keys import misp_url, misp_key, misp_verifycert
try:
    from keys import misp_client_cert
except ImportError:
    misp_client_cert = ''
import argparse
import os

if __name__ == '__main__':

# Do some arguments parsing
# I deleted code for this part, you can check it in the repo

    # Define a misp object
    misp = ExpandedPyMISP(misp_url, misp_key, misp_verifycert, cert=misp_client_cert)
    # Fetch the events using the limits provided by the user
    result = misp.search(publish_timestamp=args.last, limit=args.limit, page=args.page, pythonify=True)

    if not result:
        print('No results for that time period')
        exit(0)

    if args.output:
        with open(args.output, 'w') as f:
            for r in result:
                f.write(r.to_json() + '\n')
    else:
        for r in result:
            print(r.to_json())

As I said, this is a well-written library that makes automation a breeze. I will definitely use it to build my upcoming integration between MISP and G-Suite.

Integrations & Possibilities

MISP is open and extendible, two characteristics that make for endless possibilities to how you can use the platform to share data and collaborate with others for the greater good. You can find a long list of tools and systems that either use or integrate with MISP here, it's truly amazing.

One idea I will work one in the upcoming days is exporting information from productivity suites into MISP (e.g. phishing detect alerts from G-Suite, ...). I think it can be a great module specially considering the huge move towards cloud productivity services following the COVID-19 pandemic start back in March.

Objectives Check

Install and configure a MISP instance ✔️
Subscribe to some feeds to get information into the instance ✔️
Get acquainted with the theoretical aspects of the platform ✔️
Do some scripting using PyMISP ✔️
Learn what possible integrations are available ✔️

Links

Cover Image from MISP Project

#Labs #Tools #MISP