Collectible Card Game Headquarters

spirolone wrote:Maybe I have some infos usefull. I'm pretty sure that structure is same that PKZip central directory one ( https://users.cs.jmu.edu/buchhofp/foren ... pkzip.html ) and I attach an Excel sheet with values for a Data_000.zed file. As you can see there are all infos usefull about single files in archive (you didn't see compressed size cause there is a 2-bytes integer with lenght of file name). In a standard zip file we should have an "End of central directory record" and a local file header at the beginning of any file, but there are only redundant data and we can easily construct them. I tried to do so, but it was not sufficient. Now, I'm doing some tests ad then I'll update you...

Yeah i already checked that link yesterday and i also found that the structure we can see is of a PKZip central directory, i checked your excel file and on the flags column you always have "256" but the value here can be :

Flags General purpose bit flag:
Bit 00: encrypted file
Bit 01: compression option
Bit 02: compression option
Bit 03: data descriptor
Bit 04: enhanced deflation
Bit 05: compressed patched data
Bit 06: strong encryption
Bit 07-10: unused
Bit 11: language encoding
Bit 12: reserved
Bit 13: mask header values
Bit 14-15: reserved

Any idea about this?

Yes, I noted that strangeness in flags bits: 256 = bit 8, and it should be unused. In normal zip files, if encrypted, bit 1 is set, and I checked that value is "1" and not "256". I think that programmers use that bit to indicate that there is a non-standard "encryption"...

spirolone wrote:Yes, I noted that strangeness in flags bits: 256 = bit 8, and it should be unused. In normal zip files, if encrypted, bit 1 is set, and I checked that value is "1" and not "256". I think that programmers use that bit to indicate that there is a non-standard "encryption"...

Umm.... You may have your byte order flipped (little-endian vs big-endian) as bit 8 is bit 0 of a different byte.

RiiakShiNal wrote:Umm.... You may have your byte order flipped (little-endian vs big-endian) as bit 8 is bit 0 of a different byte.

I tried to create an encrypted zip file and those two bytes are "01 00" instead of "00 01" like ours...

Someone has already tried some software to recover broken zip archives?? We could test some of them also ont old 2014 wad and onto ZED(and XOR ZED) just to understand if there is already an utility that can read the central directory and rebuild the files...i just downloaded ZIP RECOVER 3.1 i am trying it onto XOR DATA_000.ZED...

NeoAnderson wrote:So do you also think the data is not using any Encryption? why there are references to private key, RSA,....???

Maybe encryption is used for multiplayer at some point, I don't know.

spirolone wrote:Now, I'm doing some tests ad then I'll update you...

We're counting on you!

EDIT: If the uncompressed size of DATA_000's Header.xml really is 2736 Bytes, I'm wondering what's in it... I would have expected it to be just like this:

DotP2014 DATA_CORE @header.xml with DATA_CORE substituted with DATA_000 | Open

Code: Select all

<?xml version="1.0"?>

<WAD_HEADER>
   <ENTRY platform="ALL" source="DATA_000/DATA_ALL_PLATFORMS/" alias="Content" order="1"/>
   <ENTRY platform="TOUCH" source="DATA_000/DATA_TOUCH/" alias="Content" order="2"/>
   <ENTRY platform="PC" source="DATA_000/DATA_PC/" alias="Content" order="3"/>
   <ENTRY platform="X360" source="DATA_000/DATA_X360/" alias="Content" order="3"/>
   <ENTRY platform="PS3" source="DATA_000/DATA_PS3/" alias="Content" order="3"/>
   <ENTRY platform="IPAD" source="DATA_000/DATA_IPAD/" alias="Content" order="3"/>
   <ENTRY platform="ANDROID" source="DATA_000/DATA_ANDROID/" alias="Content" order="3"/>
   <ENTRY platform="ANDROIDLOWRES" source="DATA_000/DATA_ANDROIDLOWRES/" alias="Content" order="4"/>
</WAD_HEADER>

...but this is about 700 Bytes.

EDIT 2: if I make a ZIP file with a .METADATA/Header.xml (by the way, Windows doesn't let me call a directory with a "." as first character, I have to rename it when it's already inside the ZIP file), the central directory will also list the .METADATA/ directory. Why doesn't this happen in the ZED files' central directories?

thefiremind wrote:EDIT 2: if I make a ZIP file with a .METADATA/Header.xml (by the way, Windows doesn't let me call a directory with a "." as first character, I have to rename it when it's already inside the ZIP file), the central directory will also list the .METADATA/ directory. Why doesn't this happen in the ZED files' central directories?

I'm starting to think that first 256 bytes of central directory is "encrypted" not only with a Xor. Indeed first bytes of .METADATA/Header.xml is too strange and thay don't have "PK" signature. Likely in that 256 bytes there are other 3 entry (txt file with list of files in archive, .METADATA dir, and another). The problem is to find how those bytes and bytes in single compressed files is encrypted...

However I verified that Magic 2015 loads last 8 bytes of Zed files, then bytes from offset and then bytes of single files. It starts with Header.xml and AppInfo.xml (for DATA files), and then a lot of tdx files. It loads 8KB avery time (16KB with AUDIO and no limit with MOVIE).

spirolone wrote:I tried to create an encrypted zip file and those two bytes are "01 00" instead of "00 01" like ours...

... And where does it say that they are using standard zip files? We already know they have changed things which is why none of us has figured out how to decrypt and unpack anything. So we can't really afford to make assumptions about what is and isn't correct at this point.

spirolone wrote:I verified that Magic 2015 loads last 8 bytes of Zed files, then bytes from offset and then bytes of single files. It starts with Header.xml and AppInfo.xml (for DATA files), and then a lot of tdx files. It loads 8KB avery time (16KB with AUDIO and no limit with MOVIE).

If I substitute last 2 bytes of DATA_010.ZED with different values that have the same xor result, the game doesn't load. Could it be that those last 8 bytes are used for something more?

RiiakShiNal wrote:

spirolone wrote:I tried to create an encrypted zip file and those two bytes are "01 00" instead of "00 01" like ours...

... And where does it say that they are using standard zip files? We already know they have changed things which is why none of us has figured out how to decrypt and unpack anything. So we can't really afford to make assumptions about what is and isn't correct at this point.

True, indeed I'm not saying they use standard zip files. But we saw a similitude between their format and standard zip format and we were asking ourselves what flags might mean. However, I didn't understand, then, what are you assuming about little or big endian...

thefiremind wrote:If I substitute last 2 bytes of DATA_010.ZED with different values that have the same xor result, the game doesn't load. Could it be that those last 8 bytes are used for something more?

Very interesting! You are right: there are 4 bytes potentially with usefull info; but I didn't understand how we can use them. I assumed it can be a "key", but maybe they could be only "check" bytes...

spirolone wrote:However, I didn't understand, then, what are you assuming about little or big endian...

Little endian - used mostly in Intel machines.Little endian number is ordered from the least significant byte (LSB) in low memory address to the most significant byte (MSB) in high memory address.
Address offset Data
0 byte0
1 byte1
2 byte2
3 byte3

Big endian - used mostly in Motorola machines and Interent Protocols ( network byte order ).
Big endian number is ordered from the most significant byte (MSB) in low memory address to the least significant byte (LSB) in high memory address.
Address offset Data
0 byte3
1 byte2
2 byte1
3 byte0

There is also another one Middle endian
Numerous other orderings, generically called middle-endian or mixed-endian, are possible. On the PDP-11 (16-bit little-endian) for example, the compiler stored 32-bit values with the 16-bit halves swapped from the expected little-endian order. This ordering is known as PDP-endian.
data, architecture
Neither big-endian nor little-endian. Used of perverse byte orders such as 3-4-1-2 or 2-1-4-3, occasionally found in the packed decimal formats of some minicomputer manufacturers

Now looking the structure of our XOR DATA_000.ZED file i can see :
50 4B 01 02 3F 00 14 00 00 01 08 00 85 9D F0 44 that should be :
SIGNATURE : (50 4B 01 02) = Central directory file header signature
Version : (3F 00) = 0x003F should be : upper byte: 00 -> 0 - MS-DOS and OS/2 (FAT / VFAT / FAT32 file systems) and lower byte: 63 -> 6.3
Version needed : (14 00) = 0x14 = 20 -> 2.0
Flags : (00 01) = 0x0100 = 4 enhanced deflation
Compression method : (08 00) = 08: deflated
File modification Time : (85 9D) = 0x9d85 = 10011 101100 00101 --> Hour(10011) : 19 - Minutes(101100) : 44 - Seconds(00101) : 05 = 19:44:05
File modification date : (F0 44) = 0x44f0 = 0100010 0111 10000 --> Year(0100010) : 34 (1980+34= 2014) - Month(0111) : 07 - Day(10000) : 16 = 07/16/2014

Enhanced deflate
The "enhanced deflate" method is similar to the original deflate but operates on larger chunks of data at a time, often resulting in improved compression. It can be particularly useful for compressing large files containing large amounts of highly compressible data such as large text files and text-based database files.
5.6.1 The Enhanced Deflating algorithm is similar to Deflate but uses
a sliding dictionary of up to 64K. Deflate64(tm) is supported
by the Deflate extractor.

This make sense if you consider that there are a lot of Text based files inside the ZED file.

thefiremind wrote:If I substitute last 2 bytes of DATA_010.ZED with different values that have the same xor result, the game doesn't load. Could it be that those last 8 bytes are used for something more?

Maybe that the HASH of the file is changed and the game cannot recognize it, or somekind of CRC check???

NeoAnderson wrote:Maybe that the HASH of the file is changed and the game cannot recognize it, or somekind of CRC check???

If I change some characters in the <Files> section inside MOVIES_000, the game doesn't complain at all. Then again, maybe DATA_xxx ZEDs are treated differently, who knows.

thefiremind wrote:

NeoAnderson wrote:Maybe that the HASH of the file is changed and the game cannot recognize it, or somekind of CRC check???

If I change some characters in the <Files> section inside MOVIES_000, the game doesn't complain at all. Then again, maybe DATA_xxx ZEDs are treated differently, who knows.

My first thought could be that DATA_xxx.ZED should contains also functions and config files needed to make the game initialization, so if something goes wrong here the game cannot load, about MOVIES_000 probably if a video file cannot be loaded the engine simply skip it, but will be not a critical error.

awesome that there seem to be some progress with getting into 2015, but since I have no idea what the heck you guys are talking about, are we close to getting an answer to the problem ?

drleg3nd wrote:awesome that there seem to be some progress with getting into 2015, but since I have no idea what the heck you guys are talking about, are we close to getting an answer to the problem ?

We're closer. From what I can tell, there's no good way to tell a timeline or progress or anything until it finally clicks since we don't even know with certain if anything is genuinely encrypted, merely obfuscated, or both. The read speed suggests to me (as was mentioned earlier by someone else, though I don't remember who) that it's probably not encrypted, but it could also be a less secure quick encryption method or something. I could be wrong though. Sure, I'm following along very closely, but I know very little about the topic (hence my silent observations haha).