Boxplots – Data Visualisation

Box and Whisker Plots or boxplots, are a hugely useful data visualisation tool to clearly compare algorithm configuration performance results (or experiment data with multiple dimensions). However, using Python’s Matplotlib library to implement them suitably for comparisons by groups used to be tough. To make them attractive and clear you had to stitch together documentation and examples and more examples and grids and line colours and axis labels and some very hacky legend use case, etc.. each taken from across the matplotlib site and beyond. So I wrote a couple of scripts to simplify grouped boxplots that can be directly reused..

Here’s the Grouped Boxplot (on left) and Ungrouped Boxplot (on right):

The code to create the plots is below.

Continue reading “Boxplots – Data Visualisation”

Combine PDFs into single file in LaTeX

Occasionally, you will probably need to combine a set of PDF files into a single PDF file

LaTeX can do it – But it can be a pain to get the correct appearance and find the correct API parameters for scaling, margins, number of pages per PDF, etc. Below is a quick snippet that should just work.

Continue reading “Combine PDFs into single file in LaTeX”

RDA – Artificial Immune System (AIS) for Anomaly Detection

The Receptor Density Algorithm (RDA) is an Artificial Immune System (AIS) anomaly detection algorithm modelled upon how T-cell receptors respond to antigen, originally modelled by Owens et al in 2009. A recent small project has been to investigate its applicability as an AIS anomaly classifier for our CARDINAL-Vanilla AIS self-healing architecture.

Github repo | Version 0.1 implementation of RDA in python 2.7.

Intelligent Robotics PhD Research Student at Aberystwyth University (2011 to 2015)

As my PhD time at Aberystwyth’s Intelligent Robotics research group draws to a close  I will be moving my content, code, papers and presentations from this new Wordpress blog and new domain name

The last four years have been such a fun time, full of learning, friendships and a developing love-hate relationship with the Welsh countryside and weather! This photo is of our climb up Snowdon, Wales’s tallest mountain at 1000m, with my partner Wikanda Promkhuntong and Tzu Chueng, our kind friend and visiting scholar from Taiwan, back on a beautiful July day in 2013.

It’s also nice to go back over past work, here are some interesting bits of code that I will migrate in the coming weeks:

  • The open source — Weka package RSARSubsetEval — feature selection package using rough set theory, for selecting the most informative columns from datasets for supervised learning classifiers.
  • The Receptor Density Algorithm — RDA python implementation —  for anomaly detection, which is an artificial immune system model algorithm based on a T-cell receptor, originally by Owens et al in 2009 at York.
  • The reformatted CSIC 2010 HTTP datasets network security datasets for machine learning algorithms. Specifically for web penetration testing and web application firewalls. We have used them in feature building, distributed system testing and network-based security intrusion detection/ prevention systems (IDS/IPS).
  • … and of course details of — CARDINAL-Vanilla — my PhD thesis architecture. It’s an applied proof-of-concept decentralised self-healing architecture for network-based security in industrial control system (ICS) and supervisory control and data acquisition (SCADA) networks. There’s also a Youtube video of an early version of the CARDINAL-E architecture classifying and self-hardening on two workstations reading network traffic packet data.