Mobile textures (SA/VC) - Revision history

Squ1dd13: Seems unlikely

2021-06-02T11:56:33Z

Seems unlikely

Squ1dd13: More bold

2021-06-02T11:56:05Z

More bold

Squ1dd13: Device formats

2021-06-02T11:55:06Z

Device formats

Squ1dd13: Use tables instead of structure code

2021-06-02T11:50:02Z

Use tables instead of structure code

Squ1dd13: Names

2021-06-02T10:35:53Z

Names

Squ1dd13: Justified length of code

2021-06-02T10:33:08Z

Justified length of code

Squ1dd13: Removed because it's broken

2021-04-14T17:23:36Z

Removed <syntaxhighlight> because it's broken

Squ1dd13: /* .dat */

2021-04-14T17:22:51Z

‎.dat

Squ1dd13: Offsets are signed

2021-04-12T11:05:56Z

Offsets are signed

Squ1dd13: /* Format */ Details on storing data file size

2021-04-12T10:35:21Z

‎Format: Details on storing data file size

@@ Line 84: / Line 84: @@
 | <code>detailtile</code>
 | Integer
-| The number of times to tile the detail texture (unconfirmed).<br />
 |-
 | <code>alphamode</code>

@@ Line 8: / Line 8: @@
 A single texture database is made up of several files:
-* <code>name.txt</code> – text
+* <code>name.txt</code> – <b>t</b>e<b>xt</b>
 * <code>name.x.toc</code> – <b>t</b>able <b>o</b>f <b>c</b>ontents
-* <code>name.x.dat</code> – texture data
+* <code>name.x.dat</code> – texture <b>dat</b>a
 * <code>name.x.tmb</code> – <b>t</b>hu<b>mb</b>nails

@@ Line 1: / Line 1: @@
 The mobile ports of GTA: San Andreas and GTA: Vice City use texture systems different from their PC and console counterparts in order to reduce the
-amount of storage space required for storing textures.
+amount of storage space required for storing textures, and to improve performance by using texture formats which take advantage of the hardware of the target device.
 The information in this article is based on the formats found in the mobile version of GTA:SA, but most can also be applied to GTA VC.
@@ Line 166: / Line 166: @@
 The <code>.dat</code> file contains an array of texture listings along with the actual texture data.
-Textures are represented in different formats on different platforms. iOS devices use PVR for most textures, as
+Textures are represented in different formats on different platforms. PVR is used for a majority of textures
-every iPhone before the iPhone 8 used a PowerVR GPU which had special hardware support for the texture format.
+on iOS devices, because many iOS devices use PowerVR GPUs which have special hardware support for PVRTC.
 === Structure ===

@@ Line 172: / Line 172: @@
 The <code>.dat</code> file consists entirely of listing structures, with texture data following the metadata in each.
-Listing structures look like the following:
+The following table defines, in order, the fields of these listing structures.
-<pre>
-struct Listing {
+{| class="wikitable"
-    hash: u16,
+|-
-    encoding_type: u16,
+! Field
-    width: u16,
+! Type
-    height_mask: u16,
+! Use
-    compressed_size: u32,
+|-
-    rle_indicator: u32,
+| Hash
+| <code>u16</code>
-</pre>
+| Hash of the texture name. Used to ensure that <code>.dat</code> listings match up with <code>.txt</code> entries.
 ==== Hash ====
-The <code>hash</code> field is a 16-bit hash of the texture name. This allows the game to ensure that each listing matches
+Since the game expects the order of the <code>.txt</code> file to match up with the order of the <code>.dat</code> file, it is important
-up with the information from the <code>.txt</code> file rather than blindly trusting that the files match. This hash is calculated
+that the two never get out of sync. In order to prevent this, the <code>.dat</code> file stores a hash of the name given in the <code>.txt</code> file. This allows the game to ensure that each listing matches
 as follows:
@@ Line 210: / Line 232: @@
 ==== Width & Height ====
-The <code>height_mask</code> field dedicates 15 bits to storing the height of the texture, but the most significant bit
+The height mask dedicates 15 bits to storing the height of the texture, but the most significant bit
-encodes a Boolean denoting the use of mipmaps in the texture. If the bit is 1, the texture only contains a single image,
+determines the use of mipmaps in the texture. If the bit is 1, the texture only contains a single image,
 but if it is 0, there are multiple mips present. To find the number of mips, the width and height values (with the height
 being <code>height_mask & 0x7fff</code>) must be divided by two repeatedly until neither can be divided any further. The
@@ Line 238: / Line 260: @@
 ==== Run-Length Encoding ====
-The game uses additional run-length encoding as a form of lossless compression to reduce the size of many of the stored textures. This compression is only enabled on textures where <code>rle_indicator != 0</code>.
+The game uses additional run-length encoding as a form of lossless compression to reduce the size of many of the stored textures. This compression is only enabled on textures where the RLE indicator is not zero.
 Decompression is very simple, and requires a segment size and an indicator byte as well as the compressed bytes. The bytes are split into groups. Each group can be either <i>segment size</i> bytes, or <i>segment size + 2</i> bytes. If the group starts with the indicator byte, the next byte is the number of repetitions. The <i>segment size</i> bytes after that constitute the segment to be repeated. If the first byte in the group is not the indicator byte, a segment is read and appended to the output.

@@ Line 8: / Line 8: @@
 A single texture database is made up of several files:
-* <code>name.txt</code>
+* <code>name.txt</code> – text
-* <code>name.x.toc</code>
+* <code>name.x.toc</code> – <b>t</b>able <b>o</b>f <b>c</b>ontents
-* <code>name.x.dat</code>
+* <code>name.x.dat</code> – texture data
-* <code>name.x.tmb</code>
+* <code>name.x.tmb</code> – <b>t</b>hu<b>mb</b>nails
 The four files listed above are named according to the archive they come from: <code>name</code> is <code>gta3</code> for textures from <code>gta3.img</code>, <code>gta_int</code> for textures
@@ Line 173: / Line 173: @@
 Listing structures look like the following:
-<!-- Sorry Wesser, my code was in Rust – Squ1dd13 --><pre>
+<pre>
 struct Listing {
      hash: u16,

@@ Line 135: / Line 135: @@
 If the offsets are not regenerated, the mismatch of size values allows the game to detect the invalid offsets and ignore them.
-Below is example code for reading an offset file.
+Below is example code for reading an offset file, which also demonstrates some of the checks the game will perform.
 <pre>
 FILE *offsetFile = std::fopen(pathToOffsetFile, "rb");

@@ Line 136: / Line 136: @@
 Below is example code for reading an offset file.
-<syntaxhighlight lang="c++">
+<pre>
 FILE *offsetFile = std::fopen(pathToOffsetFile, "rb");
@@ Line 159: / Line 159: @@
 // offsetArray now contains the offsets from the file.
-</syntaxhighlight>
+</pre>
 In both error cases in the above code, the game will resort to reading the offsets from the data file.

@@ Line 164: / Line 164: @@
 == .dat ==
-The <code>.dat</code> file is a raw binary file that stores all of the texture data for the database.
+The <code>.dat</code> file contains an array of texture listings along with the actual texture data.
-Different image formats are used on different platforms – iOS devices store most textures in PVR format, but Android devices
+Textures are represented in different formats on different platforms. iOS devices use PVR for most textures, as
-use different formats.
+every iPhone before the iPhone 8 used a PowerVR GPU which had special hardware support for the texture format.
-There is extra run-length encoding used to compress the textures. A decompression algorithm is below.
+=== Structure ===
-<syntaxhighlight lang="c++">
+Listing structures look like the following:
-void ExpandRLE(uint8 *out, uint32 outSize, uint8 *in, uint32 segmentSize, uint32 rleIndicator) {
+<!-- Sorry Wesser, my code was in Rust – Squ1dd13 --><pre>
-     if (outSize == 0) {
+struct Listing {
-        return;
+    hash: u16,
+    encoding_type: u16,
-    // Loop until our pointer to the output buffer reaches the end.
+==== Hash ====
-    for (uint8 *endByte = out + outSize; out < endByte;) {
+The <code>hash</code> field is a 16-bit hash of the texture name. This allows the game to ensure that each listing matches
-        // If `indicator == rleIndicator`, the next segment is repeated.
+up with the information from the <code>.txt</code> file rather than blindly trusting that the files match. This hash is calculated
-        uint8 indicator = in[0];
+as follows:
-                    // Advance the pointer so we're appending instead of just copying.
+# Start with an unsigned 32-bit hash value of 0.
-                    out += segmentSize;
+# For every character byte in the string, shift the hash bits left by 5 and add the byte.
+# After the loop, shift the hash bits right by 5.
-                // Each compressed section is `segmentSize + 2` bytes long. `segmentSize` is
+Sample algorithm (Rust):
-                //  the size of the repeated data (which is present only once in the encoded
+<pre>
-                //  data), and the two bytes are `indicator` and `repetitions`.
+fn hash_name(name: &str) -> u16 {
-                in += segmentSize + 2;
+    let mut hash: u32 = 0;
-            // We read one segment and wrote one segment, so advance both pointers to reflect this.
+    for byte in name.as_bytes() {
-             out += segmentSize;
+        hash = hash
-             in += segmentSize;
+             .overflowing_add((hash << 5).overflowing_add(*byte as u32).0)
+             .0;
+     }
+}
-</syntaxhighlight>
+</pre>
 [[Category:File Formats]]

@@ Line 109: / Line 109: @@
 == .toc ==
 === Overview ===
-The <code>.toc</code> file is a table of offsets for the <code>.dat</code> file (which contains all of the texture data). This file is not
+The <code>.toc</code> (<b>t</b>able <b>o</b>f <b>c</b>ontents) file is a table of offsets for the <code>.dat</code> file (which contains all of the texture data). This file is not
 required by the game: a new offset table can be created by reading the <code>.dat</code> file. The purpose of the offset table is to accelerate
 texture loading. A game running without prebuilt offset tables will likely suffer from small stutters, generally lower FPS, and textures loading
@@ Line 118: / Line 118: @@
 read directly from the data file instead.
-The game is capable of generating offset table files for future use, but this code never runs in production builds.
+The game is capable of generating offset table files for future use, but this code never runs in production builds. It is likely that to generate the
 === Format ===
 Offset files have a very basic format: one 32-bit unsigned integer (<code>uint32</code>) for the size of the <i>data</i> file,
-and then the rest of the file is consecutive <code>uint32</code> offsets in the order of the entries in the <code>.txt</code>
+and then the rest of the file is consecutive 32-bit signed integer (<code>int32</code>) offsets in the order of the entries in
-file.
+the <code>.txt</code> file. These offsets are absolute offsets from the beginning of the data file to the start of the texture's data.
 The stored size of the data file is used to verify that the two files match. If this step were left out, the game would
@@ Line 130: / Line 135: @@
 If the offsets are not regenerated, the mismatch of size values allows the game to detect the invalid offsets and ignore them.
-To read an offset file, code similar to the following pseudo-C++ may be used:
+Below is example code for reading an offset file.
 <syntaxhighlight lang="c++">
 FILE *offsetFile = std::fopen(pathToOffsetFile, "rb");
@@ Line 148: / Line 153: @@
+}
-uint32 *offsetArray = new uint32[database.entryCount];
+int32 *offsetArray = new int32[database.entryCount];
 std::fread(offsetArray, 4, database.entryCount, offsetFile);
@@ Line 159: / Line 164: @@
 == .dat ==
-The <code>.dat</code> ("data") file is a raw binary file that stores all of the texture data for the database.
+The <code>.dat</code> file is a raw binary file that stores all of the texture data for the database.
 Different image formats are used on different platforms – iOS devices store most textures in PVR format, but Android devices