Creating an xdelta3 patch

From Gitaroo Pals
Jump to navigation Jump to search

So you've created a hack of Gitaroo Man— changed some files, moved other files around, whatever. The result is an ISO file with your changes. And now you want to share it, but there are a couple problems:

  • Size: ISO files of this game tend to be a few gigabytes.
  • Legality: Yar har fiddle-dee-don't. Sharing full ISOs is illegal. Even if you modified your ISO, it still contains original copyrighted data that you are not allowed to redistribute.

So, what do you do? The solution is to create an xdelta3 patch file, one which contains only the changes you made. By itself, an xdelta3 patch file isn't playable. But if you apply it to a copy of the original ISO, it will spit out a copy of your hacked ISO, changes and all. So if you give your xdelta3 patch to someone who owns the original ISO, they can get your hack too.

Required stuff

Getting xdelta3 3.1.0

  • Windows: easiest way is to download the 3.1.0 beta
  • Mac: easiest way is to install Homebrew package manager then run brew install xdelta
  • Linux: If it didn't come pre-installed, you can probably get it from your package manager (e.g. sudo apt install xdelta3). Be sure check your version with xdelta3 -V.

Creating the patch

xdelta3 is a command-line program.

xdelta3 -e -B <SIZE_2x_ISO> -s <ORIGINAL_ISO> <HACKED_ISO> <your_hack.xdelta3>

Things to replace:

  • <SIZE_2x_ISO>: replace this with the size of the original ISO (in bytes) multiplied by 2
  • <ORIGINAL_ISO>: the filename of the original ISO
  • <HACKED_ISO>: the filename of your hacked ISO
  • <your_hack.xdelta3>: whatever you want to name the patch file it outputs

Explanation of arguments:

  • -e: encode (create a patch, as opposed to decode/apply a patch)
  • -B <SIZE_2x_ISO>: search the whole original ISO for data that was moved around in the hacked ISO. (see the -B argument)
    • Often you can omit this for PSP hacks created with UMDGen, saving a lot of RAM. (again, see the -B argument)
  • -s <ORIGINAL_ISO>: use ORIGINAL_ISO as the source
  • <HACKED_ISO>: use HACKED_ISO as the destination (aka target)
  • <your_hack.xdelta3>: creates an xdelta3 patch file with this name

Note for PSP hacks created with UMDGen: Often you can leave out the -B argument, this saves a lot of RAM (see the -B argument for explanation)

Appendix

Why xdelta3 version 3.1.0?

First of all, a later version is even better, it just didn't exist when I wrote this. If it does exist, get it!

Why not an older version? Because 3.1.0 introduces the ability to specify a -B (source buffer/window) greater than 2 GB.

The -B argument, or why you need a lot of memory

tl;dr:

  • The -B argument makes xdelta3 load the entire original ISO into memory...
  • so it can create a small patch even if files have been moved around in the hacked ISO.
  • Oh, and ImgBurn and such will move all the files around, so use this!
  • Why? To shrink the patch size and avoid distributing copyrighted data.

The longer explanation

There aren't many tools for editing an ISO's contents in-place, so often one will:

  • extract all the files from an ISO,
  • edit some files to create the hack, and then
  • rebuild the ISO from those files using a program like ImgBurn.

The problem with this is that often the files are out of order compared to the original. A naive patch-creation program would create a massive patch containing lots of copyrighted data duplicated from the original, while a smart patch-creation program would know to look in different places of the original ISO and rearrange the data.

That's where the -B argument comes in. By specifying a source buffer size that is double the size of the original ISO, then as the patch-creation process progresses through the original ISO, it will look forwards & backwards the length of the entire ISO file.

However, this will load the entire ISO into memory during the patch-creation process. This takes up a lot of RAM, and probably a lot of extra time too. (For me, this took like 10 GB of RAM and froze my computer solid for 10 seconds before proceeding with creation.)

Exception: PSP hacks created with UMDGen may not need this. UMDGen is really good at preserving the original file order. That means in most cases, you can leave out the -B argument entirely. However, if you do things like copy models or songs to other parts of the ISO, you should still use this argument.

xdelta3 vs (old) xdelta

So here's the situation.

  • There's xdelta3, which we use.
  • And then there's a much older version simply called xdelta.

Don't accidentally download and use xdelta when you meant to use xdelta3! Otherwise you may have problems like command line arguments not working as expected, or patch files that fail for no apparent reason when used with xdelta3.

If in doubt, run <name of your xdelta program> -V to check the version number, it should be 3.1.0 or higher.