Difference between revisions of "RenderWare binary stream file"

From GTAMods Wiki
Jump to navigation Jump to search
m
 
(24 intermediate revisions by 7 users not shown)
Line 1: Line 1:
 
{{File-stub}}
 
{{File-stub}}
 
{{TocRight}}
 
{{TocRight}}
'''RenderWare binary stream files''' are hierarchically structured binary data files used by the [[Wikipedia:RenderWare|RenderWare]] 3.x graphics engine utilized by [[Wikipedia:Rockstar North|Rockstar North]] for their GTA III trilogy (only on PC, [[Wikipedia:PS2|PS2]] and [[Wikipedia:XBOX|XBOX]], not [[Wikipedia:PSP|PSP]]!) and related games (Manhunt, Bully).
+
'''RenderWare binary stream files''' are hierarchically structured binary data files used by the [[RenderWare]] 3.x graphics engine utilized by [[Wikipedia:Rockstar North|Rockstar North]] for their GTA III trilogy (only on PC, [[Wikipedia:PS2|PS2]] and [[Wikipedia:XBOX|XBOX]], not [[Wikipedia:PSP|PSP]]!) and related games (Manhunt, Bully).
  
GTA's [[model file]]s use the [[Wikipedia:file extension|file extension]] <code>.dff</code> and [[texture archive]]s use <code>.txd</code>. Other possible extensions, not used by the GTA games, are <code>.rws</code>, <code>.bsp</code> and <code>.dma</code>. GTA3 does use <code>.anm</code> for some of its cutscene animations, but it is the only GTA to do so.
+
GTA's [[model file]]s use the [[Wikipedia:file extension|file extension]] <code>.dff</code> ('''D'''ive '''F'''ile '''F'''ormat) and [[texture archive]]s use <code>.txd</code>. Other possible extensions, not used by the GTA games, are <code>.rws</code>, <code>.bsp</code> and <code>.dma</code>. GTA3 does use <code>.anm</code> for some of its cutscene animations, but it is the only GTA to do so.
  
 
==File Format==
 
==File Format==
  
RW streams are split up into sections. Each section has a 12 byte header and can either be empty, contain data or more child sections. Section content, especially of data sections, is determined by an ID in the header.
 
  
There is only one data structure common to all RW streams - the '''section header''':
+
RW streams are split up into chunks. Each section has a 12 byte header and can either be empty, contain data or more child chunks.
 +
The content depends on the type and parent chunks.
  
  4 byte - DWORD - section identifier
+
  uint32 - type
  4 byte - DWORD - section size, including child sections and/or data
+
  uint32 - size, including child chunks and/or data
  4 byte - DWORD - RW version ID
+
  uint32 - library ID stamp
  
The RW version ID varies depending on the RW engine version used for exporting the files. It must be the same for all headers within one file. Backwards compatibility is very limited.
+
===Versioning===
 +
 
 +
The library ID stamp contains the version and build number of the RW library that wrote the file.
 +
 
 +
The library version is a hexadecimal number and has the form <code>0xVJNBB</code> where V (3 bits) is the Renderware version, J (4 bits) is the major revision, N (4 bits) is the minor revision and B (6 bits) is the binary revision. Version 3.6.0.3 for instance would be encoded as <code>0x36003</code>.
 +
 
 +
The library build is 0xFFFF in all versions used by GTA.
 +
 
 +
To make the library ID stamp, <code>0x30000</code> is subtracted from the version first and then packed as follows (where D is the 16 bit build number):
 +
VVJJ JJNN NNBB BBBB DDDD DDDD DDDD DDDD
 +
Version <code>0x36003</code> build <code>0xFFFF</code> for instance is encoded as <code>0x1803FFFF</code>.
 +
 
 +
For more information on how the version number can be calculated, see [[RenderWare]].
  
 
===Valid RW Section IDs===
 
===Valid RW Section IDs===
Line 27: Line 39:
 
* [[Pipeline Set (RW Section)|Pipeline Set]] ''(0x0253F2F3)''
 
* [[Pipeline Set (RW Section)|Pipeline Set]] ''(0x0253F2F3)''
 
* [[Specular Material (RW Section)|Specular Material]] ''(0x0253F2F6)''
 
* [[Specular Material (RW Section)|Specular Material]] ''(0x0253F2F6)''
* [[2dfx (RW Section)|2dfx]] ''(0x0253F2F8)''
+
* [[2d Effect (RW Section)|2d Effect]] ''(0x0253F2F8)''
* [[Night Vertex Colors (RW Section)|Night Vertex Colors]] ''(0x0253F2F9)''
+
* [[Extra Vert Colour (RW Section)|Extra Vert Colour]] ''(0x0253F2F9)''
 
* [[Collision Model (RW Section)|Collision Model]] ''(0x0253F2FA)''
 
* [[Collision Model (RW Section)|Collision Model]] ''(0x0253F2FA)''
 
* [[Reflection Material (RW Section)|Reflection Material]] ''(0x0253F2FC)''
 
* [[Reflection Material (RW Section)|Reflection Material]] ''(0x0253F2FC)''
* [[Mesh Extension (RW Section)|Mesh Extension]] ''(0x0253F2FD)''
+
* [[Breakable (RW Section)|Breakable]] ''(0x0253F2FD)''
 
* [[Frame (RW Section)|Frame]] ''(0x0253F2FE)''
 
* [[Frame (RW Section)|Frame]] ''(0x0253F2FE)''
 
===Common RW Version IDs===
 
 
The following is an (incomplete) list of plain text RW versions commonly used in GTA and their encoded equivalent:
 
 
3.0.0.3 - 0x0003FFFF
 
3.?.?.? - 0x0800FFFF (GTA III)
 
3.1.0.0 - 0x00000310 (GTA III)
 
3.3.0.2 - 0x0C02FFFF (GTA VC)
 
3.4.0.3 - 0x1003FFFF (GTA VC)
 
3.6.0.3 - 0x1803FFFF ([[GTA SA]])
 
  
 
==Common File Structures==
 
==Common File Structures==
  
===Model Files (DFF)===
+
===Basic structure===
 +
The basic structure of DFF and TXD files is given below.
 +
The Extensions are given as empty, their contents depend on the plugins and plugin data of the object they belong to.
 +
These are described elsewhere.
  
 +
====DFF====
 
* [[Clump (RW Section)|Clump]]
 
* [[Clump (RW Section)|Clump]]
 
** [[Struct (RW Section)#Clump|Struct]]
 
** [[Struct (RW Section)#Clump|Struct]]
Line 54: Line 59:
 
*** [[Struct (RW Section)#Frame_List|Struct]]
 
*** [[Struct (RW Section)#Frame_List|Struct]]
 
*** [[Extension (RW Section)|Extension]] *
 
*** [[Extension (RW Section)|Extension]] *
**** [[Frame (RW Section)|Frame]]
 
**** [[HAnim PLG (RW Section)|HAnim PLG]]
 
 
** [[Geometry List (RW Section)|Geometry List]]
 
** [[Geometry List (RW Section)|Geometry List]]
 
*** [[Struct (RW Section)#Geometry_List|Struct]]
 
*** [[Struct (RW Section)#Geometry_List|Struct]]
Line 66: Line 69:
 
****** [[Texture (RW Section)|Texture]] - optional
 
****** [[Texture (RW Section)|Texture]] - optional
 
******* [[Struct (RW Section)#Texture|Struct]]
 
******* [[Struct (RW Section)#Texture|Struct]]
******* [[String (RW Section)|String]] - diffuse texture name
+
******* [[String (RW Section)|String]] - texture name
******* [[String (RW Section)|String]] - alpha texture name
+
******* [[String (RW Section)|String]] - mask name
 
******* [[Extension (RW Section)|Extension]]
 
******* [[Extension (RW Section)|Extension]]
******** [[Sky Mipmap Val (RW Section)|Sky Mipmap Val]] **
 
 
****** [[Extension (RW Section)|Extension]]
 
****** [[Extension (RW Section)|Extension]]
******* [[Specular Material (RW Section)|Specular Material]] - optional, vehicles only
 
******* [[Reflection Material (RW Section)|Reflection Material]] - optional, vehicles only
 
******* [[Material Effects PLG (RW Section)|Material Effects PLG]] - optional, vehicles only
 
******* [[Texture (RW Section)|Texture]] - optional<br>''only occurs with the section above, possibly even a child section''
 
******** [[Struct (RW Section)#Texture|Struct]]
 
******** [[String (RW Section)|String]] - environment map name
 
******** [[String (RW Section)|String]] - alpha texture name
 
******** [[Extension (RW Section)|Extension]]
 
********* [[Sky Mipmap Val (RW Section)|Sky Mipmap Val]] **
 
 
**** [[Extension (RW Section)|Extension]]
 
**** [[Extension (RW Section)|Extension]]
***** [[Bin Mesh PLG (RW Section)|Bin Mesh PLG]]
 
***** [[Native Data PLG (RW Section)|Native Data PLG]] (on Xbox and PS2)
 
***** [[2dfx (RW Section)|2dfx]]
 
***** [[Night Vertex Colors (RW Section)|Night Vertex Colors]] - static objects only
 
***** [[Mesh Extension (RW Section)|Mesh Extension]]
 
 
** [[Atomic (RW Section)|Atomic]] *
 
** [[Atomic (RW Section)|Atomic]] *
 
*** [[Struct (RW Section)#Atomic|Struct]]
 
*** [[Struct (RW Section)#Atomic|Struct]]
 
*** [[Extension (RW Section)|Extension]]
 
*** [[Extension (RW Section)|Extension]]
**** [[Right To Render (RW Section)|Right To Render]] - vehicles only
+
** [[Struct (RW Section)|Struct]] *
**** [[Material Effects PLG (RW Section)|Material Effects PLG]] - vehicles only
+
** [[Light (RW Section)|Light]] *
 +
*** [[Struct (RW Section)#Light|Struct]]
 +
*** [[Extension (RW Section)|Extension]]
 +
** [[Struct (RW Section)|Struct]] *
 +
** [[Camera (RW Section)|Camera]] *
 +
*** [[Struct (RW Section)#Camera|Struct]]
 +
*** [[Extension (RW Section)|Extension]]
 
** [[Extension (RW Section)|Extension]]
 
** [[Extension (RW Section)|Extension]]
*** [[Collision Model (RW Section)|Collision Model]] - vehicles only
 
 
===Texture Archives (TXD)===
 
  
 +
====TXD====
 
* [[Texture Dictionary (RW Section)|Texture Dictionary]]
 
* [[Texture Dictionary (RW Section)|Texture Dictionary]]
** [[Struct (RW Section)#Texture_Dictionary|Struct]] - 4 byte, 2x word, first one is the texture count
+
** [[Struct (RW Section)#Texture_Dictionary|Struct]]
** [[Texture Native (RW Section)|Texture Native]] *
+
** [[Raster (RW Section)|Raster]] *
*** [[Struct (RW Section)#Texture_Native|Struct]] - header and actual image data
+
*** [[Struct (RW Section)|Struct]]
*** [[Extension (RW Section)|Extension]] - empty
+
*** [[Extension (RW Section)|Extension]]
** [[Extension (RW Section)|Extension]] - empty
+
** [[Extension (RW Section)|Extension]]
  
===Texture Archives PS2 (TXD)===
+
====PS2 TXD====
 
* [[Texture Dictionary (RW Section)|Texture Dictionary]]
 
* [[Texture Dictionary (RW Section)|Texture Dictionary]]
** [[Struct (RW Section)#Texture_Dictionary|Struct]] - 4b - Texture count
+
** [[Struct (RW Section)#Texture_Dictionary|Struct]]
** [[Texture Native (RW Section)|Texture Native]] *
+
** [[Raster (RW Section)|Raster]] *
*** [[Struct (RW Section)#Texture_Native|Struct]] - 4b - Platform, 4b- Filter Flags
 
*** [[String (RW Section)|String]] - Texture name
 
*** [[String (RW Section)|String]] - Alpha name
 
 
*** [[Struct (RW Section)|Struct]]
 
*** [[Struct (RW Section)|Struct]]
**** [[Struct (RW Section)|Struct]] - 4b- Width, 4b - Height, 4b - Bits per pixel, unknown
+
*** [[String (RW Section)|String]] - texture name
**** [[Struct (RW Section)|Struct]] - 80 bytes unknown, pixels/indices (80 bytes unknown, palette)
+
*** [[String (RW Section)|String]] - mask name
 +
*** [[Struct (RW Section)|Struct]]
 +
**** [[Struct (RW Section)|Struct]]
 +
**** [[Struct (RW Section)|Struct]]
 
*** [[Extension (RW Section)|Extension]]
 
*** [[Extension (RW Section)|Extension]]
**** [[Sky Mipmap Val (RW Section)]] - 4 bytes unknown
+
** [[Extension (RW Section)|Extension]]
** [[Extension (RW Section)|Extension]] - empty
 
 
 
''* These sections (and their child sections) can appear multiple times.
 
 
 
''** Not always present, effect also unknown.
 
 
 
A byte may contain two indices, if bpp == 4.
 
 
 
16 bit images are in 1555 ARGB format or 565 (used by VC only on ancient hardware such as Riva TNT2 when you choose 16 bit color mode and game builds non-dxt version of txd files on disk).
 
  
The indices are swizzled and saved in a PS2 friendly format (CLUT).
+
''* These chunks can appear multiple times. The count is usually given in the parent's struct.
 
 
Note: There are slight differences between GTA: III and VC formats concerning the raster/image format.
 
  
 
==Tools & Scripts==
 
==Tools & Scripts==
Line 133: Line 114:
 
* [http://www.steve-m.com/downloads/tools/rwanalyze/ RW Analyze] &ndash; by {{U|steve-m}}
 
* [http://www.steve-m.com/downloads/tools/rwanalyze/ RW Analyze] &ndash; by {{U|steve-m}}
 
* various [[model file]] and [[texture archive]] related tools
 
* various [[model file]] and [[texture archive]] related tools
 +
 +
==Official Sources==
 +
 +
* [https://web.archive.org/web/20070127023600/http://www.renderware.com/ Official RenderWare Site (archived)]
 +
* {{Github|electronicarts/RenderWare3Docs|Official documentation repository}}
 +
* {{Github|GTAmodding/rw37|RenderWare 3.7 source code (for demo-purposes only)}}
  
 
==See also==
 
==See also==
 +
* [http://www.chronetal.co.uk/gta/index.php?page=dff Original DFF format description] by KCow
 
* {{GTAF|128451|RW Analyze discussion thread}}
 
* {{GTAF|128451|RW Analyze discussion thread}}
* [http://www.chronetal.co.uk/gta/index.php?page=dff Original DFF format description] by KCow
+
* [https://github.com/aap/librw librw] - a reimplementation of the original RenderWare graphics API by {{U|The hero}}
* [http://www.renderware.com Official RenderWare Site]
 
 
* [http://wiki.multimedia.cx/index.php?title=TXD MultimediaWiki TXD]
 
* [http://wiki.multimedia.cx/index.php?title=TXD MultimediaWiki TXD]
  
{{SA-navi}}
+
{{N|SA|VC|3}}
 
[[Category:3D File Formats]]
 
[[Category:3D File Formats]]

Latest revision as of 10:54, 14 September 2020

RenderWare binary stream files are hierarchically structured binary data files used by the RenderWare 3.x graphics engine utilized by Rockstar North for their GTA III trilogy (only on PC, PS2 and XBOX, not PSP!) and related games (Manhunt, Bully).

GTA's model files use the file extension .dff (Dive File Format) and texture archives use .txd. Other possible extensions, not used by the GTA games, are .rws, .bsp and .dma. GTA3 does use .anm for some of its cutscene animations, but it is the only GTA to do so.

File Format

RW streams are split up into chunks. Each section has a 12 byte header and can either be empty, contain data or more child chunks. The content depends on the type and parent chunks.

uint32 - type
uint32 - size, including child chunks and/or data
uint32 - library ID stamp

Versioning

The library ID stamp contains the version and build number of the RW library that wrote the file.

The library version is a hexadecimal number and has the form 0xVJNBB where V (3 bits) is the Renderware version, J (4 bits) is the major revision, N (4 bits) is the minor revision and B (6 bits) is the binary revision. Version 3.6.0.3 for instance would be encoded as 0x36003.

The library build is 0xFFFF in all versions used by GTA.

To make the library ID stamp, 0x30000 is subtracted from the version first and then packed as follows (where D is the 16 bit build number):

VVJJ JJNN NNBB BBBB DDDD DDDD DDDD DDDD

Version 0x36003 build 0xFFFF for instance is encoded as 0x1803FFFF.

For more information on how the version number can be calculated, see RenderWare.

Valid RW Section IDs

See the list of RW section IDs.

Rockstar's Custom Section IDs

Rockstar added several vendor specific RW plugins, which have their own ID range (0x0253F2F*).

Common File Structures

Basic structure

The basic structure of DFF and TXD files is given below. The Extensions are given as empty, their contents depend on the plugins and plugin data of the object they belong to. These are described elsewhere.

DFF

TXD

PS2 TXD

* These chunks can appear multiple times. The count is usually given in the parent's struct.

Tools & Scripts

Official Sources

See also