Puttering around with BlackBerry forensics — Part 1
I’m guilty of sitting on source code which I should have released a long time ago. I make excuses to myself that I didn’t release any of it because I was waiting for someone to come along and prove to me that there was a better way of doing things. I guess the bottom line was that I was just lazy and procrastinating.
Today, I’m releasing source code to three of my projects that I’ve been incubating. 1) bbt 2) evt2sqlite 3) ConParse Take! Build! Enjoy! At some point or another, I may just split them up into their own repos. For now, they all live in the bb-tools repository down at github. Update: I’ve split them up now
In today’s post, I’ll cover the first tool, bbt.
bbt
bbt is a python script that analyzes the thumbnail cache from a BlackBerry. The purpose of a thumbnail cache in any system, is generally to speed up the browsing of large numbers of graphic or video files. Instead of presenting a static icon to the user, a small thumbnail of the picture or frame of the video file is shown. Apparently this is a good thing, because you can see an icon of the image that you’re clicking on and will hopefully be able to find the file you’re looking for quicker. Typically, the Operating System will find and shrink pictures found in directories of the filesystem. These shrunk pictures will then be placed inside the thumbnail cache.
When conducting a digital forensics analysis of a computer, looking for these thumbnail caches often provide clues as to what files may have existed before they were deleted off the file system. The thumbnail cache is important enough to warrant its own entry on the Forensics Wiki (albeit only the Windows thumbnail cache is spoken about). The principle generally remains the same when extended to the BlackBerry device as well. So, bottom line: being able to analyze this file is useful.
bbt will do just that for thumbnail cache files found on BlackBerry devices. There are two types of thumbnail caches on the BlackBerry device: 1) BBThumbs.dat format 2) key/dat format.
Format 1 is pre OS 5.0 and the key/dat format is post OS 5.0 The key/dat combination is interesting because it uses two files to keep track of thumbnails. They look something like thumbs86x86.key and thumbs86x86.dat (the 86×86 indicates the size of the thumbnail – 86 pixels by 86 pixels). I’ve noticed quite a few interesting things in these files and no doubt, you will too after you look through the source or play around with them long enough:
- The BBThumbs.dat header is 0×24052003 (which is a hex number)
- The thumbs.dat file header is 0×22062009 (hex again)
- The thumbs.key file header is 0×08062009 (hex)
This is pure speculation, but if you took those hexadecimal representations and looked at just the numbers, don’t they look like dates?
- 0×24052003 –> 24 05 2003
- 0×22062009 –> 22 06 2009
- 0×08062009 –> 08 06 2009
Maybe birth dates of the file format itself or someone significant to someone who wrote it? Dunno.
Another interesting observation of the key/dat thumbnail cache is that it not only stores image thumbnails, but also stores details of all types of media including ‘wav’, ‘mp3′, and ‘mid’. It doesn’t store any content inside it though. The only content stored inside the files is image data.
I’m not actually going to tell you what is found inside the thumbs files in this post, but instead, I am going to tell you how to run bbt. bbt is a python script and as such will require that you have python installed on your system. I’d always recommend cloning my repository on github so that you can easily pull any updates. You may also want to sign up for a github account and watch the repository so that you will be notified of any commits I make. You could also fork the project and get to work on it yourself.
azazel:Device sheran$ ~/github/bb-tools/bbt/bbt.py Usage: bbt.py [options] -h, --help: This cruft -k, --key <bbthumbs key file>: Process post OS5 thumbs.key file (requires thumbs.dat file in same directory) -b, --bbthumbs <old bbthumbs file>: Process pre OS5 BBThumbs.dat file -x, --extract: Extracts the thumbnails into directory specified by -o -o, --output <output directory>: Directory to extract thumbs to (used only with -x) azazel:Device sheran$
The output above is what you will receive if you run bbt without options. As of the latest release (0.3b), the most magical thing you can do with this tool is to extract the thumbs into a specific output directory. Additionally, bbt will parse out information about 1) What thumbnails are stored in the file (filename) for BBThumbs.dat files or 2) Where at what offset in a ‘dat’ file a specific record id is stored. Here’s some example output when parsing a key/dat pair:
azazel:Device sheran$ ~/github/bb-tools/bbt/bbt.py -k thumbs116x116.key -x -o out *** Processing thumbs116x116.key on 2011-07-22 21:50:48.156899 Record ID C620B80A is at offset 7 in 'dat' file // [1306132653179.jpeg] Record ID DB0B7CA3 is at offset 25930 in 'dat' file // [1306492410606.jpeg] Record ID D2EC23E3 is at offset 52123 in 'dat' file // [1306732433796.jpeg] *** thumbs116x116.key has 9 records *** Processed 3 records azazel:Device sheran$
When you parse a key/dat file combination, you need to make sure that both the ‘key’ and ‘dat’ file are in the same directory. When you run bbt, you will point it to the location of the ‘key’ file. From the output above, you can see that it has discovered 3 records, corresponding record ids and offsets where they are stored in the ‘dat’ file. Also, the filename of the thumbnail is provided. What do the offsets mean? Well, if you were to take the numbers and open up the ‘dat’ file in a hex editor, then you would land on the location where that specific record began. This is what it looks like:
The highlighted portion is the first part of the record with the correct starting offset. You may also notice that the ‘key’ file supposedly has 9 records but only 3 were processed. This happens because the ‘key’ file holds 9 record ids and 9 offsets, but only 3 of those actually match up in the ‘dat’ file. One assumption that can be made is that the files were deleted from the ‘dat’ file, but the ids and offsets still remained in the ‘key’ file.
bbt also has the ‘-x’ option which allows you to extract the thumbnails that are inside either the BBThumbs.dat file or the key/dat files. You do this by specifying the ‘-x’ option along with the ‘-o’ option to tell bbt where to extract the thumbnails to. You will need to make sure that the output directory specified by the ‘-o’ option does not already exist.
For now, that’s as much as you’re going to get out of bbt. Some features that are planned in the roadmap for bbt are:
- HTML Reporting
- Identification of Exif data within thumbnails
- Completely parsing some of the header and record bytes that are as yet unknown
I’ll cover the other tools in subsequent posts. For now, though, the tools are all live in the github repository. All of the tools contain a basic README doc that tells you how to get started with each of the tools.
Nice work Sheran! As always you never cease to amaze me with your parsing skills! – Shafik
Hi
Love your app but I get an error after only a few files have been extracted.
Traceback (most recent call last): File “bbt.py”, line 158, in main() File “bbt.py”, line 150, in main process(keyfile,outdir,extract) File “bbt.py”, line 70, in process rec = dfile.record(thumbs[key], key) File “C:Python27bbtmodulesDatFile.py”, line 48, in record header = struct.unpack(“>B4sIIIQIB”,self.dat_file.read(30))struct.error: unpack requires a string argument of length 30
if you can please help.
Looks like you’ve got an incomplete file. The error generally occurs when there’s not enough data to be read. If you want me to take a further look, send across the file you’re trying to process. Use the email address in the header of the bbt.py file.
it is the same files from the same phone. It is 2 back up’s that I made 1 month apart , the one month’s is just fine but the other gives me that error. It looks like somwhere it want to read something in the key file that is not in the dat file. Is there not a way to skip files on errors or something. It extract the first 10 files then gives me the error.
If you don’t mind , The files can contain some sensitive and private data so it will not be possible to send it to you.
I can understand if you don’t want to share the file. I will see if i can replicate the issue.
Hi I e-mailed you the files , to sheran@zensay.com . Help will be appreciated.
regards
Amazing. Please somebody pull my jaws up.