Reverse Engineering CoD: Ghosts FastFiles — Part 2: The Format Finally Cracked

In Part 1 I got as far as the outer deflate decompression and mapping the auth header, RSA-2048, SHA-1 block hashes, the LO region, but the actual asset data inside the zone was still opaque. I knew assets were there, I could see them in the hex, but I couldn't reliably locate or decode them. This post covers how that finally changed, with a lot of credit owed to primetime43 who picked up the research and pushed it across the finish line with me.

---

How it started again

Someone found the first blog post and reached out wanting to continue the work on Ghosts. That conversation pulled me back in, which then pulled in my friend primetime43. primetime43 has his own open source FastFile tool CoD-FF-Tools which already handles the older CoD titles. A few days prior he decided he wanted to take a crack at helping me crack Ghosts, which he hadn't dug into before.

This is one of those situations where two people each had half the picture, and the answer only appeared when you put them together.

primetime43 joins the dig

I've been building my own tool, a closed source FastFile editor and compiler in Rust, that already handles the older titles: CoD4, WaW, MW2, BO1. It can walk the asset pool, extract raw files, and surface them for reading and editing. The outer decompression for Ghosts was also working from Part 1. What wasn't working was the inner zone, the asset walking kept landing in the wrong place.

Where Ghosts diverged from the older games is the inner asset layout. On the older CoD titles the asset pool structure is well understood and you can walk it directly from the XFile header. Ghosts (IW6) changed things enough that the direct walk kept misfiring. primetime43 jumped in and started digging into the zone bytes independently, and what he came back with was the pattern that unlocked everything.

The pattern he confirmed for Ghosts asset location is this:

[FF FF FF FF] [compLen u32 BE] [decLen u32 BE] [FF FF FF FF]
              followed by the asset name (null-terminated ASCII)
              followed by the zlib stream

Four bytes of FF pointer placeholder, the compressed length as a big-endian 32-bit integer, the decompressed length as a big-endian 32-bit integer, another four bytes of FF placeholder, then the name, then the zlib stream. The zlib magic bytes (78 DA, 78 9C, 78 5E, 78 01) are reliably identifiable and confirm you've landed on an asset.

The outer decompression was already working from Part 1, skip the first 0x20024 bytes (outer header + auth header + LO region) and walk the raw deflate blocks:

// from src/compiler_core/ghosts_zone.rs
pub fn decompress_outer(data: &[u8]) -> io::Result<Vec<u8>> {
    let mut pos = BLOCKS_OFFSET; // 0x20024
    let mut zone = Vec::new();
    let mut n = 0usize;

    while pos + 2 <= data.len() {
        let src_size = u16::from_be_bytes([data[pos], data[pos + 1]]) as usize;
        pos += 2;
        if src_size == 0 { break; }

        let mut dec = flate2::read::DeflateDecoder::new(&data[pos..pos + src_size]);
        let mut chunk = Vec::with_capacity(BLOCK_OUTPUT_SIZE); // 0x10000 = 64 KB
        dec.read_to_end(&mut chunk)?;
        pos += src_size;
        zone.extend_from_slice(&chunk);
        n += 1;
    }
    Ok(zone)
}

The asset header: what we were actually looking at

This is the thing that had been blocking progress. I'd been looking at the right bytes in the hex editor but misreading the field layout. The asset header in Ghosts is:

Offset  Size  Field
+0x00   4     FF FF FF FF  — pointer placeholder (runtime address, not file data)
+0x04   4     compLen      — compressed byte count (u32 big-endian)
+0x08   4     decLen       — decompressed byte count (u32 big-endian)
+0x0C   4     FF FF FF FF  — pointer placeholder
<name>        null-terminated ASCII string
<zlib>        standard zlib stream (78 DA / 78 9C / 78 5E / 78 01)

A concrete example from bl_ps3_mp_boneyard_ns_war.bin:

Bytes:   FF FF FF FF 00 00 0A E6 00 00 94 00 FF FF FF FF
                     └─compLen─┘ └──decLen──┘
                        = 2790      = 37,888

So the header is saying: "compressed body is 2,790 bytes; when decompressed it expands to 37,888 bytes." You can stride by compLen from the start of the zlib stream to reach the next asset header. That stride is the primary way to walk the asset list. No pool table needed for the inner navigation once you've found the first asset.

Pattern scanning: how primetime located assets in Ghosts

For the older CoD titles primetime's tool could walk the asset pool table directly. For Ghosts it falls back to a scan: look for the [FF FF FF FF][plausible u32][plausible u32][FF FF FF FF] pattern where "plausible" means the values are non-zero and in a sane range for compressed/decompressed sizes, then check whether the bytes after the name are a valid zlib magic. If they are, you've found an asset.

This is less elegant than a table walk but it works, and for a format you're still mapping it's actually safer because you're not depending on a correct XFile header parse to get started. Once you've confirmed the pattern on a few assets you can switch to stride-based navigation for speed.

The key insight primetime confirmed: after finding one asset via scan, every subsequent asset is exactly at current_zlib_start + compLen. The stride is exact. You don't have to scan the whole zone, one scan to find the first asset, then stride from there. With that confirmed I implemented it into the tool.

Here's the fallback scanner in my tool, it runs when the stride lands somewhere unexpected:

// from src/compiler_core/ghosts_zone.rs
fn find_next_header(zone: &[u8], start: usize) -> Option<usize> {
    let mut i = start;
    while i + 13 < zone.len() {
        if zone[i..i+4].iter().all(|&b| b == 0xFF) {
            let comp_len   = u32::from_be_bytes(zone[i+4..i+8].try_into().unwrap()) as usize;
            let decomp_len = u32::from_be_bytes(zone[i+8..i+12].try_into().unwrap()) as usize;
            if zone[i+12..i+16].iter().all(|&b| b == 0xFF) {
                // plausible sizes + printable first char of name
                if comp_len > 0 && comp_len < 0x400000
                    && decomp_len > 0 && decomp_len < 0x400000
                    && i + 16 < zone.len()
                    && zone[i + 16] >= 0x20 && zone[i + 16] < 0x7F
                {
                    return Some(i);
                }
            }
        }
        i += 1;
    }
    None
}

And the main asset walker that ties it together. Stride first, fall back to scan if the stride lands somewhere wrong:

// from src/compiler_core/ghosts_zone.rs (simplified)
for (idx, &type_id) in pool.iter().enumerate() {
    // Header: [FF FF FF FF][compLen u32 BE][decLen u32 BE][FF FF FF FF][name][zlib]
    pos += 4; // skip leading FF FF FF FF
    let compressed_len   = u32::from_be_bytes(zone[pos..pos+4].try_into().unwrap());
    let decompressed_len = u32::from_be_bytes(zone[pos+4..pos+8].try_into().unwrap());
    pos += 8 + 4; // skip lengths + trailing FF FF FF FF

    // null-terminated name
    let name_start = pos;
    while pos < zone.len() && zone[pos] != 0 { pos += 1; }
    let name = String::from_utf8_lossy(&zone[name_start..pos]).to_string();
    pos += 1; // skip null

    // inner zlib decompression
    let zlib_start = pos;
    let content = decompress_zlib(&zone[zlib_start..], decompressed_len);

    assets.push(GhostsAsset { type_id, name, compressed_len, decompressed_len, content });

    // advance: stride by compLen, fall back to scan if stride looks wrong
    let stride_pos = zlib_start + compressed_len as usize;
    let looks_good = stride_pos + 16 < zone.len()
        && zone[stride_pos..stride_pos+4].iter().all(|&b| b == 0xFF)
        && zone[stride_pos+12..stride_pos+16].iter().all(|&b| b == 0xFF);
    pos = if looks_good { stride_pos } else { find_next_header(zone, pos).unwrap_or(zone.len()) };
}

The .bin files: a weird detour

While we were poking around one of the smaller patch FFs, patch_mp_boneyard_ns.ff to be more specifc, something odd turned up. The top-level asset pool for that file contained only mptype and aitype entries. No rawfiles at the top level. But we knew the map had associated scripting data somewhere.

The mptype asset decompressed to a large blob, and inside that blob there were filenames that looked like mp/constbaselines/bl_ps3_mp_boneyard_ns_war.bin. These .bin files are embedded rawfile blobs packed inside the mptype struct... they don't appear as top-level pool entries.

(Screenshot curtosy of primetime43)

We spent a while staring at the .bin content itself trying to figure out what format it was. The bytes had a strange repeating structure... lots of E5 values, then sequences that seemed to walk through parts of the alphabet. It looked like it could be delta-encoded or some kind of run-length scheme. After enough staring the working theory settled on: it's map state data, probably constbaseline save data (the constbaselines path prefix is a hint), and it's binary-packed game structs rather than something with an obvious compression signature. Not something you'd reverse without knowing the in-memory struct layout for the MapEnts type.

The detour was useful though because it confirmed that small patch FFs have a two-tier structure: the outer zone has only mptype/aitype pool entries, and the actual map scripting rawfiles are embedded one level deeper inside the mptype payload. Any tool that only walks the top-level pool will miss them entirely.

mptype assets and embedded rawfiles

This turns out to be a consistent pattern across the small patch FFs in the BLUS31270 update pack. A survey of 331 files from that pack breaks down roughly like this:

The larger patch FFs (those with many rawfile pool entries) expose everything at the top level and walk cleanly. The small ones, anything that's mostly mptype/aitype have their rawfiles buried inside the mptype struct. Extracting those requires parsing the IW6 MapEnts in-memory struct layout, which is a separate problem we haven't solved yet. For now those files are noted as partially parsed.

The tool: what it can do now

So my tool now handles Ghosts PS3 FFs end to end for the assets it can reach:

• Outer decompression — both Variant A (multiple 64 KB deflate blocks, large FFs) and Variant B (single block, small FFs)
• Asset pool parsing — reads all type IDs correctly from the XFile header pool
• Per-asset walking via the [FF FF FF FF][compLen][decLen][FF FF FF FF][name\0][zlib] header format
• Inner zlib decompression per asset
• Text assets (rawfile, scriptfile, localize, stringtable) surfaced in the Raw Files tab for reading and editing
• Hex viewer for any asset
• Asset Pool tab showing all assets with type name, size, and a parseable flag
• 329 of 331 files from the BLUS31270 update pack load correctly

The two files that don't load are common_alien_dlc_updated_mp.ff and common_core_dlc_updated_mp.ff. These use a completely different outer container, the IWffS100 magic appears at 0x86C4 instead of 0x24, and there are no deflate blocks anywhere in the file. Compression method unknown. These are set aside for now.

The full format map

Pulling together everything from both research phases, here's the complete picture of a Ghosts PS3 FF:

File offset   Size         Region
0x00000       36 bytes     Outer header
                           — "IWff0100" magic, version 0x22E, flags, file size

0x00024       8144 bytes   Auth header
                           — "IWffS100" inner magic
                           — 20-byte SHA-1 subheader hash
                           — 256-byte RSA-2048 signature (Activision signing key)
                           — 32-byte zone name (ASCII, null-padded)
                           — 244 × 32-byte master block hashes (SHA-1 + 12B zero pad)

0x01FF4       48 bytes     Padding (residual bytes, purpose unclear)

0x02024       114688 bytes LO region (14 × 8KB chunks, all bytes < 0x80)
                           — engine reads this at load time, not needed for extraction

0x20024       rest         Compressed zone blocks
                           — stream of [u16 BE srcSize][raw deflate payload] blocks
                           — each block decompresses to exactly 65,536 bytes
                           — no encryption

Zone layout (after decompression)
0x000         64 bytes     XFile header
                           — zone size, block size slots, scriptStringCount, assetCount

0x040         N × 8 bytes  Asset pool
                           — [FF FF FF FF][00 00 00 XX] per entry (pointer + type ID)
                           — pool ends at first non-FF-FF-FF-FF entry

after pool    variable     Per-asset content, back to back
                           — [FF FF FF FF][compLen u32 BE][decLen u32 BE][FF FF FF FF]
                           — null-terminated ASCII name
                           — standard zlib stream (78 DA / 78 9C / 78 5E / 78 01)
                           — stride by compLen from zlib start to reach next asset

The known type IDs for IW6 — these are what the 8-byte pool entries resolve to:

// from src/compiler_core/ghosts_zone.rs
pub fn ghosts_type_name(id: u32) -> &'static str {
    match id {
        0x01 => "physpreset",    0x03 => "xmodel",
        0x05 => "material",      0x07 => "technique_set",
        0x0D => "image",         0x15 => "sound",
        0x1F => "menu",          0x21 => "localize",
        0x23 => "weapon",        0x28 => "aitype",
        0x29 => "mptype",        0x2C => "rawfile",
        0x2D => "scriptfile",    0x2E => "stringtable",
        _    => "unknown",
    }
}

For reference, the older titles (CoD4, WaW, MW2, BO1) have their own type ID enums and the IDs shift between engine versions, which is why a CoD4 pool walker can't directly read a Ghosts zone. The tool maintains per-game type tables for each supported title.

Two compression layers, no encryption at either. The RSA signature is verification-only. The zone data itself is not AES encrypted or otherwise ciphered, contrary to some earlier theories. Raw deflate + standard zlib, nothing more exotic than that.

What still doesn't work

Embedded rawfiles inside mptype structs. The small patch FFs store their map scripting data inside the mptype decompressed payload. Getting those out requires parsing the IW6 MapEnts struct, which means mapping the full in-memory layout of that type on PS3. Not done yet.

The two multi-zone DLC files. Different outer container, different offset for the inner magic, no deflate blocks visible. Haven't figured out what they're doing.

Writing / patching. You can read everything that's exposed. Writing a modified FF back to disk and having the engine accept it is a different problem... the RSA-2048 signature covers the content, and without Activision's private key you can't produce a valid signature. On RPCS3 there may be ways around this (the emulator's signature verification can potentially be patched), but that's a separate investigation.

PC and Xbox One Ghosts. Those use the IW6 x64 engine with 64-bit pointer zones. The pointer width changes and the zone layout differs. The outer container might be similar but the inner format will need its own mapping.

Closing

The first post ended with "knowing when to stop." This one ends differently. The format is cracked, at least for the PS3 retail version and the asset types we care about. The path from a .ff file to readable GSC scripts and game data is clear and implemented.

The thing that actually got us there wasn't any single clever insight. It was primetime43 doing parallel research independently and being willing to share what he'd found. His confirmation of the [FF FF FF FF][compLen][decLen][FF FF FF FF][name\0][zlib] asset header pattern and the compLen stride that follows from it was the specific piece that unlocked the inner zone. He's also been doing his own implementation work on the format separately, so if you're researching IW-engine titles it's worth checking out what he's up to: github.com/primetime43.

The remaining open problems: mptype struct parsing, the DLC container format, write support. When any of them get solved there will be a Part 3.