Utilities
DNA FAT – DNA Frequency Analysis Tool
by sharpe on May.20, 2010, under Molecular Biology, Utilities
DNA-FAT performs a very rapid indexation of the uniqueness and repetitative buildup of fasta format DNA sequences using user assigned window sizes.
The two output files contain the following:
- The number of unique sequences with the preset window size and the number of sequences present, any number of multiple times.
- Output of the actual sequences themselves and their number of repetitions in the input DNA sequence.
One of the possible uses of the program is to evaluate the required DNA sequencing length of the new next-generation sequencing technologies such as Illumina’s Genome Analyzer and ABI’s SOLiD platform, to predict a high number of unique matches when performing techniques such as RNAseq.
Here is a screen-dump of dnafat in action:
Below is an example of an output file using the genome sequences Staphylococcus Aureus USA300 (NC_007793):
- Extracted sequences report: dnafat sequence report (76)
- Analysis report: dnafat results report (72)
DNA-FAT can be downloaded here: dnafat (62)
Progamme idea by Marc Stegger (SSI), written by sharpe.
FISH Version 0.0.3 Beta Released
by sharpe on Aug.31, 2009, under Malware, Security, Utilities
FISH version 0.0.3 beta with numerous improvements is now available for download. Expect to see trememdous performance increases in this release as well as many other improvements including the following:
Version 0.0.3 (second public release):
- This version is at least 30 times faster than version 0.0.2
- Implemented a new hash function
- Optimised the iterative function used to iterate files
- Time stamps are now included in database files as well as reports
- It is now possible to limit file selection to a specific size (calculating the digest of a 7GB vm image is a really bad idea)
- Restructured database files
- Restructured report files
- Introduced more logging to ensure that the user knows exactly which files were skipped (if they are too big according to the configuration)
If you find any bugs or have any ideas for improvement, please don’t hesiate to let me know.
FISH can be downloaded here: fish 0.0.3 (98). This is still a beta release so use at your own risk.
NASTI – Nickos And Sharpes Tool for Identifying potencially malicious files
by sharpe on Jun.12, 2009, under Malware, Security, Utilities
Recently whilst analysing the PSP2-BBB banker trojan, I discovered that this particular trojan, as well as many others, downloaded malicious payloads to the currently logged on user’s “Temp” and “Temporary Internet Files” directories, located in the “Local Settings” directory, on a Win32 system, and saved them as temp files with a random four character name plus the .TMP extension (e.g. 23D4.TMP). Knowing this, it would be nice to be able to identify executable files with inconsistent file extensions (e.g. executable files with .TMP as the file extension), as this would aid in the identification of potentially malicious files and thus speed up the analysis.
For the sake of illustration, if we look at one of the examined TMP files in a hex editor (presented in the following screen dump), we can clearly see that it is a valid Portable Executable (PE) file:
NASTI’s configuration enables the user to specify signatures for known file types and associate file extensions to this signature. When NASTI runs, it compares the current file’s extension with those supplied in the configuration file. If they do not match, then the signature should not be present in the file. If it is, the file is flagged.
Here is a screen dump of NASTI in the middle of a scan:
NASTI can be downloaded here: nasti (50). This is a beta release so use at your own risk.
IFAU – Index Frequency Analysis Utility
by sharpe on Jun.04, 2009, under Reverse Code Engineering, Utilities
IFAU is a simple utility that was written to analyse binary files and count how many times each hex byte is present in a given file. This is useful when attempting to identify frequently used op codes. The results are written to a hash which is dumped to an XML report as well as a graph upon completion.
A sample XML report might resemble the following:
<?xml version=”1.0″ encoding=”iso-8859-1″ ?>
<bytes count=”10″>
<byte value=”11″ count=”1″ />
<byte value=”23″ count=”1″ />
<byte value=”32″ count=”1″ />
<byte value=”38″ count=”1″ />
<byte value=”4a” count=”1″ />
<byte value=”69″ count=”1″ />
<byte value=”87″ count=”2″ />
<byte value=”ad” count=”1″ />
<byte value=”df” count=”1″ />
<byte value=”fa” count=”1″ />
</bytes>
As mentioned above, this information is also presented in a graph. A large number of different bytes can result in the graph being difficult to read, which is why the previously presented XML report is also generated.
As an illustration, a graph for a small binary file might resemble the following:
IFAU can be downloaded here: ifau (46). This is a beta release so use at your own risk.
File Identification Toolkit Signature Generator
by sharpe on Apr.29, 2009, under Reverse Code Engineering, Utilities
FIT-sig (File Identification Tool-kit Signature Utility) is a tool designed to be used to aid in the identification of common signatures, of varying lengths, in two given files of any type (e.g. binary and non-binary files). FIT-sig was originally written to help when identifying Portable Executable packers/protectors.
The general idea is that you take two different files (e.g. executables) of varying sizes and pack them with the same packer/protector and feed them to FIT-sig. The result is an XML report containing signatures common for both files. FIT-sig can also be used to find similarities in other files such as images (e.g. GIF, BMP, and JPEG) and PDF documents. The length of the desired signatures can be specified in the configuration file as can the number of signatures required. For further information regarding how this tool is used, please consult the configuration file.
It can be downloaded here: fit-sig (40). This is a beta release so use at your own risk.
FISH – File Integrity Utility Version 0.0.2
by sharpe on Mar.25, 2009, under Malware, Security, Utilities
FISH is a free file integrity tool that creates a file signature database (default db.xml) for the file types (ie. exe, dll, sys) specified in the configuration file (config.xml). This signature database is then fed to the FISH (sigdb in the config) at a later date to check for the presence of new, deleted and altered files. A brief summary is written to the console and a report (default report.xml) is created, highlighting changes.
I use FISH on numerous servers to perform nightly file integrity checks. The results are written to the report which I have placed on the desktop. The path for the report can be specified in the configuration file (1).
FISH can be downloaded here: fish 0.0.2 (33). This is a beta release so use at your own risk.
1: If the path use specify in the configuration file contains reserved characters, remember to place the path in a CDATA section in the configuration file: <![CDATA[C:/Reports & Logs/FISH/report]]>
SAJO – Automated MSSQL data-mining tool
by sharpe on Mar.25, 2009, under Security, Utilities
SAJO is an automated MSSQL data-mining tool. Via the presence of a single SQL Injection vulnerability, SAJO can gather all table and field details including ids, name lengths and names. After all preliminary work is done, SAJO proceeds to dump all data in the first record in each table.
The cool thing about SAJO is that it uses the binary search algorithm when gathering ASCII characters used in table names and data which makes it very fast. It’s pretty sexy and does a tremendous job. I have included a log that was created on Foundstones Hacme Bank web application as well as a small application that I tested with, to illustrate what it is capable of.
Using this tool is quite simple. You must define a link to the vulnerable file (A) as well as a parameter (B) including a default value for the parameter (C). A value must be included as a fingerprint which is some text string that is returned by the web application when A?B=C is requested.
SAJO is still under development so minor bugs may exist. SAJO can be dowloaded here: sajo (36). Use with discretion.
LAIT – Log file Attack Identification Tool
by sharpe on Mar.25, 2009, under Security, Utilities
Recently whilst working on a project, I required a tool that was able to read a large number of large log files and identify attacks in them. Loads of web server log analysis tools exist but I haven’t come across one that focuses exclusively on identifying attacks.
LAIT a generic tool that is able to search any log files and identify web application attacks, attacks that are specified in the configuration file. I have added some really nice signatures for some of the OWASP Top Ten categories but have not added attack signatures for them all. If you manage to create some good signatures, I would be really pleased to hear from you. The signatures are based 100% on regular expressions so you can get as fancy as you like. You only need one category in the configuration but it’s a good idea to add multiple, as LAIT is able to write out individual XML reports for each category which is nice if there are hundreds or even thousands of hits.
LAIT can be downloaded here: lait (36). This is a beta release, use at your own risk.
DNA Extractor – Utility for Invitrogens Vector NTI
by sharpe on Mar.25, 2009, under Molecular Biology, Utilities
DNA Extractor is an automated utility for extracting specifically formatted DNA sequences from data files such as those available at the National Center for Biotechnology Information. This can be done for both genomes and plasmids alike, as long as they keep the specific format. Genome and plasmid locations are read from genename files having the format specified in this file and actual DNA sequences are read from genedata files (specified in the XML configuration file).
By adding the desired details concerning which files to search, to the XML configuration file, the locally stored data files are searched for the gene locations (i.e. nt position 336..2798) of interest and all valid gene sequences are extracted from the specified file types (specified in the genedata element in the configuration file) and written to the output file (specified in the output element in the configuration file).
This format is specifically designed as input data for the Vector NTI programme suite for further analysis.
It can be download here dna-extractor (33). This is a beta release so use at your own risk.


