NWN Binary Model Files Basics

The binary format of the model files contains 3 main sections, the header, the model data and the vertex or raw data.

The header is a only twelve bytes long.  It provides us with the offset and size of the raw data, and a 32 bit value of 0.  The value of 0 is important since it can be used to tell the difference between an ASCII model file and a binary one.  The chances of an ASCII model file starting with 4 bytes of 0 is somewhere between zero and none.

The model data follows the header in the file.  Immediately following the model data is the raw data.  Thus, the second value in the header can be either the model size or the offset from the start of the model data to the raw data.  Either interpretation works since they are the exact same value.

Arrays and Pointers in a Model File

The structure of the model is reasonably complex.  Not only does it contain structures of data, but those structures reference other structures.  This is accomplished using arrays and pointers.  

There are two types of pointers inside of a model file, model data pointer and raw data pointer.  Each of these pointers are stored as a 32 bit value.  In the case of model data pointers, the pointer will contain an offset from the start of the model data to the data in question.  A value of zero represents a "NULL" pointer or a pointer that doesn't reference anything.  For raw data pointers, the pointer will contain an offset from the start of the raw data to the data in question.  A value of 0xFFFFFFFF (unsigned) or -1 (signed) represents a "NULL" pointer or a pointer that doesn't reference anything.  The reason the raw data uses a value of -1 is that an offset of zero is a valid pointer into the raw data.  Excluding one special case, this isn't true for model data pointers.  (Only one element in a binary model file points to the data at offset zero in the model data.  However, this value is transient and isn't actually stored in the model file.)

Some of you might be wondering why I am referring to these offsets as pointers.  After all, in the binary model format, they are always offsets and never actually a real pointer.  That is very true for the disk image of a model.  However, on 32 bit address processors (or processors such as the Alpha that can run in 32 bit address mode with sign extend), after the model is loaded from disk, all the offsets can be converted to pointers.  This improves greatly the run time performance.

Arrays in models are a slightly more complicated beast.  They consist of the following three elements.

For binary model files, the number of used entries and number of allocated entries will always be the same.  During run time or complication time, these value will usually differ since these arrays by nature will grow as more elements are added.

Arrays can contain most anything as long as the elements are all the same type.  Examples of common arrays in model files would be arrays of vertices, faces, and even pointers.  In the case of pointers, these arrays are commonly used to represent a model hierarchy.

Controllers

Each node (to be defined later), in a model can contain zero or more controllers.  Controllers can be considered as attributes of a node.  They defined certain aspects of the node.  One of the most common controller is the position controller.  This controller dictates the position of the node relative to the parent node.

One of the first obvious questions about controllers is why isn't this information included as part of the standard node structures.  There are two reasons for this.  The first reason is that controllers are optional.  In the case of an emitter node, there are around 50 different controllers that can be specified.  Having to dedicate storage space to all of these controllers, used or not, would be a waste of space.  The second and more important reason is that controllers can be animated.

To animate a model, the model must change over time.  The way to do this is with time keyed controllers.  All animations take a given amount of time from start to finish.  For example, the swing of a sword might take 1 second of total animation time.  In order to draw the model properly, controllers have different that are time keyed.  For example, normally, the position controller would be specified as follows.

position 0.2 1.0 0.124

However, in the case of a time keyed position, it would be specified as follows.

positionkey 3
  0.0 0.2 1.0 0.124
  0.4 0.4 1.0 0.510
  0.8 0.6 1.0 0.516

When specifying a time keyed controller, the first value is the start time for that value.  Following the start time is the actual value.  In this case, during animation time 0.0 through 0.4, the first position will be used.  Between 0.4 and 0.8, the second position will be used.  Between 0.8 and the animation time for the given animation, the 3rd value will be used.

To provide smoother animations, the values will be interpolated.  For example, given our previous data, if the current animation time was 0.2, then that would be half way between the first and second positions.  Thus each position will contribute equally to the actual position used.  If the animation time was earlier, then the first position would be given more weight.  The opposite is true if the animation time was later. 

To provide an even smoother animation, a second style of interpolation is available.  It is know as "bezier" interpolation.  Bezier interpolation is used when you specify "bezierkey" as the suffix to the controller name.  (Note: At this time, I have not verified that any Bioware model uses bezier interpolation.  I don't even know if the rendering engine supports this option.)

Inside a binary model file, controllers are stored as two arrays in the model data.  The first array is an array of a controller structure.  The second array is a simple array of floating point numbers.  The second array contains the actual controller data while the first tells us about all the controllers in the array.  The controller structure is as follows.

There are a few important notes about the controller structure.  The first thing of note is that for controllers that aren't time keyed, they are still stored as if they are time keyed but with a single row and a time key value of zero.  Thus, it is impossible to tell the difference between a controller that isn't time keyed, and a time keyed controller with a single row and a time key value of zero.  The second important note is that if the controller is actually bezier keyed, then the value of 0x10 is ORed in with the number of columns.  This is how you can tell the difference between a normal keyed controller and a bezier keyed controller.  Finally, all key values are stored continuously and all the data values are stored contiguously.  Thus, if a keyed controller had 3 rows with the time keys starting at floating point value 5, then the time keys for the other two rows would be value 6 and value 7.  Also, it appears that for some models, the controller "detonate" when used as a key controller doesn't even list a time key.  Thus, the number of columns listed is -1.

Following is a list of all controllers:

Model Routines and the Node types

All nodes in a model begin with six values which have also been called "tokens".  During the early process of decoding the binary models, many people including myself made the mistake that these tokens were used to identify the different types of nodes.  As silly as it sounds, this was the best we could do at the time.  These six tokens did uniquely identify each of the nodes.  However, I and I would imagine most others knew there had to be a deeper meaning to these tokens.  Nobody uses 6 4-byte values to uniquely identify a handful of different nodes.  All of these tokens were values in the range of 0x00400000 and 0x00500000.  There was no apparent bit mask to the values.  However, the difference between the token values was interesting since it was usually either 0x10 or 0x20.  Now, if someone was really had everything on the ball and had a reasonable knowledge of the Win32 image loader, they would have realized something that eluded everyone, including myself for the longest time.  These tokens were not tokens, they were routine addresses.  You see, the Win32/NT image loader loads images at 0x0041000.  Thus, the funny start to all the numbers.  

So, as of now, these "tokens" should be consider unreliable as a method of identifying model node types.  What if Bioware releases new models using a new build of their model compiler and these routine addresses change.  All the existing software that utilizes these "tokens" would fail.

Luckily, the proper way to identify the node type has been located.  Every node contains a 32 bit bit mask that identifies which structures make up the node.  Following is a list of all the flags.

Every node contains a node header.  In the case of a dummy node, only a node header is required. All mesh nodes also contain a mesh structure.  In the case of a trimesh node, only the node header and the mesh header is required.  By looking at a combination of flags, not only do we know what structures make up the node, but we know what the node type is.

As you can see, each of the different node types contains one or more flags.

Part Numbers

Part numbers are values assigned to nodes as the model compiler creates them.  However, after the complete geometry has been compiled, these values are adjusted.

If a model has a super model, then the geometry for the model is compared against the geometry for the super model.  Any node that matches the name of a node in the super model will be given the part number assigned to the node in the super model.  If a node in the model isn't found in the super model, then it receives a part number of -1.  If a model doesn't have a super model, then the part numbers are left as is.

After the geometry for an animation has been compiled, the same process is used to match the nodes in the animation with the nodes in the main model geometry.  It is important to note that the animation geometry is compare against the model's geometry and not the model's super model geometry.

Layout of the Binary Model File

The basic layout of the binary model file is as follows:

Not all models contain animation headers.  Also, in many models, there might be no raw data.  It is also important to remember that this diagram is a simplification of the real model layout.  For example, to fully defined a node, it might take 100 bytes of data or 10,000 bytes of data.  This information might be scattered in sections throughout the model data.

The Geometry Header

All models contain at least one geometry header.  This header is part of the larger model geometry header.  If the model contains animations, then there will be a geometry header in each of the animation geometry headers. 

The format of the geometry header is as follows:

As noted, the two arrays at 0x50 and 0x5C, and the data at 0x68 is unknown at this time.  

The Model Header

There is only one model header per model file.  This header always starts at offset 0 in the model data section or offset 12 from the start of the file.

The Animation Header

There are zero or more animation headers per model, one header for each animation.  All animations contain their own geometry information.  However, this usually consists of dummy nodes containing controller information.

Each animation can have zero or more events.  Unlike other arrays that contain pointers to the structures, the animation event array is an array of the actual structure.  The structure is as follows:

Nodes

Currently, there are nine different nodes types that make up a model's geometry.  These nodes specify such elements as lighting, animated graphics emitters, and different types of meshes.  Each node type share a common header that supplies us with enough information to tell what type of node it is and information about controllers and children.

Mesh Nodes

All nodes that contain mesh information share a common header.  The header is as follows:

The two arrays at 0x0210 and 0x0204 are interrelated. The array at 0x0210 isn't an array of UINT16s, it is an array of pointers to lists of UINT16.  The number of UINT16s in each of the lists is specified by the elements of the array at 0x0204.  For the second entry in both 0x0210 and 0x0204, the array 0x0204 might tell us the there are 14 vertices in the list while the 0x0210 array gives us the pointer to the list.

All the pointers 0x022C through 0x0248 point to lists of data.  The number of elements in these list is specified by the UINT16 at 0x0230 which is the vertex count.

The face structure is as follows:

The Dummy Node 

The dummy node is a default node in a geometry that only contains children nodes and controller information.  It has no other data associated with it.

The Light Node

The light node specifies light sources in the geometry of the model.

The Emitter Node

The emitter node specifies dynamic graphical elements that are emitted from the model such as smoke or sparkles.

The Reference Node

The reference node... TBD

The TriMesh Node

The trimesh node provides the basic drawing mesh used to render elements of the game.

The Skin mesh Node

The skin mesh node provides a specialized mesh where the texture is stretched and contorted as the model moves to provide a more realistic look to skin.  It is mostly used just for things such as dragon wings.  

Most all skin data is computed from the weights array.

The Animmesh node

The animmesh nodes are used for the game's GUI

The stored arrays are much like the other stored vertex arrays.  There size is the number of sets times the number of mesh vertices.  There is also a difference between how this information is parsed from the ASCII version and stored in the binary.  In the ASCII version, each vertex is listed sequentially for each vertex or texture vertex  sets.  However, in the binary version, each vertex is store with the different sets store sequentially.

The Danglymesh node

The danglymesh node provides a model with the look of movement by allowing faces to move due to momentum even after the whole model has stopped.  Thus is much like how a car passenger jerks forward in a car when it stops suddenly.

Much like the vertex information for the mesh, the constrains are expanded to match the vertex list stored in the model.

The AABB node

The aabb node provides the game with the ability to test for collisions.

The following is the layout of the AABB entry structure.

Update - 09/29/2002

1. Cleaned up the definition of the AABB entry slighly.
2. Updated the skin node.

Update - 08/22/2002

1. Corrected assorted typos and minor errors
2. "Triangle fan" changed to "Triangle strip".
3. Corrected the type on the offset in the mesh header at 0x0210.  The array exists in model data, but the actual data that the array points to exists in the raw data.
4. Added a missing unknown value in the mesh header at 0x00E4.  Corrected the offset of texture0/bitmap.
5. Updated information on the part numbers and the node counts.
6. The value at 0x0068 in the geometry header is now known.
7. Another flag in the geometry header discovered.

Credits

I wanted to thank Zaddix and Revinor.  Their information about the binary model formats provided an excellent starting point for my work.