-
Notifications
You must be signed in to change notification settings - Fork 22
ss specs.txt
0.1 Table of contents (INCOMPLETE)
0 DOCUMENT META-DATA
0.1 Table of contents
1 GENERAL FILE INFORMATION
1.1 Resource file format
1.1.1 Resource file header
1.1.2 Chunk directory
1.2 Resource file compression algorithm
1.2.1 Decoder approach
1.2.2 Encoder behaviour
1.3 Summary of data files
1.4 Mac resource files
2 GENERIC CHUNK TYPES
2.1 Textures and sprites
2.1.1 Bitmap header
2.1.2 Bitmap compression
2.1.3 Pixel 'Animation'
2.1.4 private palettes
2.2 Sounds
2.3 Fonts
3 THE MAP ARCHIVE 3.0.1 Level chunk list
3.1 Level map
3.1.1 Chunk xx02
3.1.2 Chunk xx03
3.1.3 The level information chunk
3.1.4 The tile map
3.1.6 The texture list
3.2 Objects
3.2.1 The master object table
3.2.2 The object cross-reference table
3.2.3 The weapons table, class 0
3.2.4 The ammo table, class 1
3.2.5 The projectile table, class 2
3.2.6 The grenades / explosives table, class 3
3.2.7 The patches table, class 4
3.2.8 The hardware table, class 5
3.2.9 The software / logs table, class 6
3.2.10 The scenery / decorations table, class 7
3.2.11 The items table, class 8
3.2.12 The switches / panels table, class 9
3.2.13 The doors / gratings table, class 10
3.2.14 The animations table, class 11
3.2.15 The traps and triggers table, class 12
3.2.16 The containers table, class 13
3.2.17 The critters table, class 14
4 OBJECT PROPERTIES
4.0 WEAPONS TABLE, class 0
4.0.0 SEMI-AUTO WEAPON TABLE, class 0/0
4.0.1 AUTOMATIC WEAPON TABLE, class 0/1
4.0.2 PROJECTILE WEAPON TABLE, class 0/2
4.0.3 MELEE WEAPON TABLE, class 0/3
4.0.4 ENERGY BEAM WEAPON TABLE, class 0/4
4.0.5 ENERGY PROJECTILE WEAPON TABLE, class 0/5
4.1 AMMO CLIP TABLE, class 1
4.2 PROJECTILE TABLE, class 2
4.2.0 TRACER TABLE, class 2/0
4.2.1 PROJECTILE TABLE, class 2/1
4.2.2 class 2/2
4.3 GRENADES / EXPLOSIVES TABLE, class 3
4.3.0 GRENADES TABLE, class 3/0
4.3.1 EXPLOSIVES TABLE, class 3/1
4.4 PATCHES TABLE, class 4
4.5 HARDWARE TABLE, class 5
4.6 SOFTS TABLE, class 6
4.7 FIXTURES TABLE, class 7
4.8 ITEMS TABLE, class 8
4.8.0 Junk
4.8.1 Debris
4.8.2 Corpses
4.8.3 Items
4.8.4 Access cards
4.8.5 Cyber items
4.8.6 Stains
4.8.7 Quest items
4.9 SWITCHES TABLE, class 9
4.9.0 Switches
4.9.1 Receptacles
4.9.2 Terminals
4.9.3 Panels
4.9.4 Vending
4.9.5 Cybertoggles
4.10 PORTALS (DOORS, GRATINGS) TABLE, class 10
4.11 ANIMATED TABLE, class 11
4.12 MARKER TABLE, class 12
4.13 CONTAINER TABLE, class 13
4.14 CRITTER TABLE, class 14
4.15 COMMON OBJECT PROPERTIES
5 MUSIC
Most of the resource files used by System Shock share the same basic format. These usually have the file extension `.res'.
Looking Glass resource files are chunkfiles as is common in games: each file is made up of sub-files or chunks (my name for them) which may in turn contain sub-chunks (my really bad name for them). System Shock has several resource files each containing a set of related chunks. Game maps, graphics, text and sounds all reside in resource files. Follows the basic format of a resfile.
1.1.1 Resource file header
The resource file header consists of the first 128 bytes of the file.
0000 string "LG Res File v2\r\n\x1A" 0011 blank not used 007C int32 file offset to chunk directory
Although the byte 0x1A is found in all res files immediately after the magic string, it is not part of the string. It serves as a terminator for an optional comment that may follow the string header. For a true res file implementation this should be respected - but for SystemShock only, treating the 0x1A byte as fixed is ok.
1.1.2 Chunk directory
The chunk directory is a 6 byte header followed directly by directory entries:
0000 int16 no. chunks in file 0002 int32 file offset to beginning of first chunk
Chunk directory entry : 10 bytes
0000 int16 chunk ID (globally unique modulo language versions) 0002 int24 chunk length (unpacked) 0005 int8 chunk type : 00 flat uncompressed 01 flat compressed 02 subdir uncompressed 03 subdir compressed 0006 int24 chunk length (packed in file) 0009 int8 content type: 00 palette (?data) 01 text 02 bitmap 03 font 04 video clip 07 sound effect (Glen Sawyer) 0F 3D model 11 audio log / cutscene (Glen Sawyer) 30 map
Note: all chunks begin on a 4-byte boundary, so does the chunk directory.
==============================
Chunk subdir header : 2 bytes + 4 bytes per subblock + 4 bytes
0000 int16 no. subblocks 0002 int32 chunk offset to first subblock ... nnnn int32 chunk offset to last subblock nnnn+4 int32 total length of chunk
Note: for type 3 (subdir, compressed) chunks the subdir is not compressed; offsets are offsets in the unpacked data
Some subdirs have their data immediately after the directory, others have two bytes space between them.
The compression algorithm used for resfiles is a simple dictionary-based coding scheme. The data stored in the compressed chunk consists of a continuous stream of 14-bit words stored big-endian i.e. if the first 7 bytes in the chunk are
aaaaaaaa bbbbbbbb cccccccc dddddddd eeeeeeee ffffffff gggggggg
the corresponding words are
aaaaaaaabbbbbb bbccccccccdddd ddddeeeeeeeeff ffffffgggggggg
with the high bit to the left in each case.
To unpack a word from the compressed data stream, say it has value x. If x == 0x3fff (16383) it is the end-of-stream marker. Stop. If x == 0x3ffe (16382) reinitialise the dictionary. Forget all stored positions and lengths of compressed words and start as if from scratch. If x is less than 256, write the literal byte x to the uncompressed data. Otherwise take n = x - 256; re-unpack the n'th word from the compressed data followed by the next uncompressed byte; if this would take us beyond the end of the so-far uncompressed data, write a literal zero byte instead.
1.2.1 Decoder approach
I handle decompression by keeping a dictionary of all 16127 (16384-256-1) possible reference words (as opposed to literal bytes or the end-of-stream marker) consisting of position in the uncompressed stream, unpacked length and original reference from the compressed data. Initialise all the lengths to
- Each time we take a word from the compressed stream, make a note of its position, and if it is a reference word (255 < x < 16383) its value. For a literal byte we then just write the byte to the output stream. Otherwise look up the reference in the dictionary; if its length is 1 we have not encountered it before, set the length to 1 + (length of original reference). Then unpack it by repeating (length) bytes from (position).
(I'm not 100% sure about this but it gets all the lengths right and comes up with reasonable looking unpacked data)
As promised, here is a sample unpacking routine in C.
void unpack_data (unsigned char *pack, unsigned char *unpack, unsigned long packsize, unsigned long unpacksize) {
unsigned char *byteptr;
unsigned char *exptr;
unsigned long word;
int nbits;
int val;
int ntokens = 0;
static int offs_token [16384];
static int len_token [16384];
static int org_token [16384];
int i;
for (i = 0; i < 16384; ++i)
{
len_token [i] = 1;
org_token [i] = -1;
}
memset (unpack, 0, unpacksize);
byteptr = pack;
exptr = unpack;
nbits = 0;
while (exptr - unpack < unpacksize)
{
while (nbits < 14)
{
word = (word << 8) + *byteptr++;
nbits += 8;
}
nbits -= 14;
val = (word >> nbits) & 0x3FFF;
if (val == 0x3FFF)
{
break;
}
if (val == 0x3FFE)
{
for (i = 0; i < 16384; ++i)
{
len_token [i] = 1;
org_token [i] = -1;
}
ntokens = 0;
continue;
}
if (ntokens < 16384)
{
offs_token [ntokens] = exptr - unpack;
if (val >= 0x100)
{
org_token [ntokens] = val - 0x100;
}
++ntokens;
}
if (val < 0x100)
{
*exptr++ = val;
}
else
{
val -= 0x100;
if (len_token [val] == 1)
{
if (org_token [val] != -1)
{
len_token [val] += len_token [org_token [val]];
}
else
{
len_token [val] += 1;
}
}
for (i = 0; i < len_token [val]; ++i)
{
*exptr++ = unpack [i + offs_token [val]];
}
}
}
}
1.2.2 Encoder behaviour
The encoder follows the standard LZW algorithm.
Futhermore, it has the following features:
- The End-Of-Stream marker is always written at the end, even if the decoder would know to stop because of reaching the end of the decompressed size.
- Regardless of at which bit-position the encoder stopped, always one extra 0x00 byte is added to the compressed byte stream.
- When the encoder has its dictionary exhausted it continues to work with it until it tried to create a new key for the 1000th time. Then, a dictionary reset is performed (and marked in the stream). Note that the decoder has to follow accordingly (Hint: it will anyway - even IF the decoder would create extra keys, it never would read their values from the stream since that is impossible)
System Shock data files fall into two categories: cached and rarely-accessed files which are left on the CD, and frequently-accessed files which are stored on the hard disc.
1.3.1 CD files
These are always found in directory cdrom/data on the CD.
archive.dat Level map archive bwtabl.dat citalog.res Audio logs (English) citbark.res Audio trap messages (English) cutspal.res Palettes for cutscenes death.res frnalog.res Audio logs (French) frnbark.res Audio trap messages (French) gamepal.res In-game palette data geralog.res Audio logs (German) gerbark.res Audio trap messages (German) gryntabl.dat intro.res MainMenu screens ipal.dat Colour cube lofrintr.res logeintr.res lowdeth.res lowend.res lowintr.res mongtabl.dat objart.res Sprites for objects objprop.dat Object properties (generic and class-specific) shadtabl.dat splash.res Splash screens splshpal.res Palette for splash screen (Origin) start1.res svfrintr.res svgadeth.res svgaend.res svgaintr.res svgeintr.res textprop.dat Texture properties vidmail.res Video mail whyttabl.dat win1.res
1.3.2 Hard-disc files
These start off in directory hd/data of the CDROM, and are copied to the data subdirectory of your System Shock install on the hard disc
citmat.res Textures for 3D object models cybstrng.res Game strings (English) digifx.res digiparm.bin frnstrng.res Game strings (French) gamescr.res HUD borders, fonts, buttons gerstrng.res Game strings (German) handart.res Graphics for wielded weapons intro.res MainMenu screens mfdart.res MFD icons (target/email/item display) (English) mfdfrn.res MFD icons (target/email/item display) (French) mfdger.res MFD icons (target/email/item display) (German) obj3d.res 3D object models objart2.res Graphics for critters objart3.res Graphics for critters, decals and doors objprop.dat Object properties sideart.res Sidebar icons texture.res Map textures
1.4 Mac resource files
The Mac (PPC) version of System Shock 1 has its resources stored in a different file format. Note: Unless otherwise stated, within this document everything refers to the Intel version -- Mac resource files have been deciphered for not a so long time. Also: All integer fields in the PPC files are stored in big endian.
Videos and audios are stored in dedicated files, videos have a readable file name and audios a number (probably the chunk id). Audio files are raw PCM (8bit, 22kHz) files and have an 8 byte header: 0000 uint32 Null 0004 uint32 ASCII 'mdat'
Resources are stored in files with the extension .rsrc (With the exception of Archive.data) and differ to the intel variant by:
- They have not the LG resource file format, but similar
- Chunks are not classified (type) on chunk level nor seem they to be compressed at all
Resource file header:
Not much has been deciphered as of yet (also contains some strings)
0000 uint32 Unknown (typically 0x00000100) 0004 uint32 file offset to chunk directory 0008 uint32 Previous value minus 0x00000100 ?? ... 007A uint32 total file length ...
Since the first three uint32 fields are similar (identical?) to the first of the chunk directory header, I believe that this is some sort of master- container file format with its entries linked like a double linked list; But since yet only files with two entries have been discovered, a sequence can not be seen... -- ff1
Chunk directory header: (38 bytes)
0000 uint32 File offset to start of chunks? (typically 0x00000100) 0004 uint32 File offset to chunk directory (pointing to itself?) 0008 uint32 Previous value minus 0x00000100 ?? 000C uint32 Length of Chunk Directory - up to file end *) 0010 6xb Unknown 0016 uint32 Unknown (typically 0x0000001C) 001A uint32 Unknown 001E uint32 Type? Resolves to ASCII: 'sIMG' objart 'sA01' archive 's???' gamepal 0022 uint16 Number of Chunk Directory Entries minus One 0024 uint16 Unknown (typically 0x000A)
*) Note that there are dummy bytes at the end (filler values) if the chunk entries don't fill up the directory size.
Chunk Directory Entry: (12 bytes)
0000 uint16 ChunkId 0002 uint16 Starting with 0x0000, a counter with +2 increment? 0004 uint32 Start Offset of Chunk, relative to base chunk start 0008 uint32 Unknown -- always 0x00000000 ?
Chunk Entry: 0000 uint32 Data Length 0004 Nxb Data
objart.rsrc: This file has only one chunk that contains its own header.
Header: 0000 uint16 Amount of entries minus One (= 0x06AA) 0002 n*4b Offset of image start from chunk start
A referenced entry then has a standard bitmap header, although the type field seems only to be a byte (the second), with the first being unknown. No RLE compressed bitmaps currently found.
Textures and sprites have content type 2 (bitmap) and use the following general format:
2.1.1 Bitmap header
The bitmap header is 28 bytes long, as follows.
0000 int32 ??? always 0
0004 int16 type (0x00: uncompressed, 0x02: appears like 0x00, 0x04: compressed)
0006 int16 ???
0008 int16 width
000A int16 height
000C int16 width in bytes
000E int8 ??? log2 width
000F int8 ??? log2 height
0010 int16
0012 int16 \ These seem to be used for animation frames to keep the
0014 int16 / sprite centred.
0016 int16 /
0018 int32 ??? (not) always 0
The hotspot rect (0x0010-0x0017) is interesting. It bounds a single pixel at the centre bottom of the sprite (it's only valid for sprites, I think).
Note on the int32 field at 0x0018: At chunks with more than one bitmap in it (nsubchunks > 1) this value was set as: bitmapN.value + (size of subchunkN) == bitmapN+1.value . I can't read anything out of this system - but perhaps I don't see the wood from the trees...
2.1.2 Bitmap compression
A compressed (type 4) bitmap can be unpacked as follows: 00 nn xx write nn bytes of colour xx nn .. .. 0<nn<0x80 copy nn bytes direct 80 00 00 skip rest of file (end of compressed data) 80 mm nn 0<nn<0x80 skip (nn*256+mm) bytes (write transparencies) 80 nn 80 .. .. copy nn bytes direct 80 mm nn 0x80<nn<0xC0 copy ((nn&0x3f)*256+mm) bytes 80 mm nn xx 0xC0<nn write ((nn&0x3f)*256+mm) bytes of colour xx nn 0x80<nn skip (nn&0x7f) bytes
Thanks to Joerg Fischer (jofis@cs.uni-sb.de) for kindly sending me the source code to his texture extractor (you can get it from the hackers' page at TTLG) which cleared up some questions I had about the bitmap format. Vasily Volkov (no known e-presence) also had a hand in the decompression. Joerg has asked that I not distribute the sources myself; email him direct if you want them.
Note that all bitmaps are subchunks, even when there is only one bitmap stored in a chunk. This is presumably to simplify the loading logic.
Textures are uncompressed square bitmaps stored at 4 resolutions each: 16x16, 32x32, 64x64 and 128x128. There are 273 textures stored, but some (a few) do not contain useful graphics. Chunks containing textures are:
76 16x16 textures (sub-chunks 0-272)
77 32x32 textures (sub-chunks 0-272)
707-979 64x64 textures (one chunk each)
1000-1272 128x128 textures (one chunk each)
Note on the 80 command: the following two bytes appear to be an uint16 value, encoded in LittleEndian. It could be that they are swapped in the MAC edition.
Furthermore, the 80 mm C0 case is not described above, but this one has not been found in the resource files (yet).
Notes on the encoder: If large areas are to be compressed, the lengths are first expressed with the 80 commands until the length can be encoded with one of the smaller codes.
But it appears that the original coder had a maximum length limit of 0x7FFF since there are rare occasions of large empty areas at the end encoded with 80 FF 7F 80 00 00 -- we would get the same result without the first 3 bytes.
It has also been found that the original encoder had some quirks, resulting in a little less optimal compression; For example, the sequence 44 02 00 1E had been found where 45 01 1E would have been technically 'more' correct and with better compression. It could be that these are the results of extra edits...
2.1.3 Pixel 'Animation'
The 'animations' that appear with some textures (SHODAN's mail images, hardware buttons, ...) are done by palette looping. It seems that there are generally four steps. Those I have yet found out:
?? 0x04 to 0x07 ?? 0x08 to 0x0B Sensaround: 0x0C to 0x0F Motion Booster: 0x10 to 0x13 SHODAN: seems to use 0x14 to 0x17 Jump Boots: 0x18 to 0x1B ?? 0x1C to 0x1F
ff1: Perhaps energy weapons are done the same way... 12052002, ff1: must be; furthermore the ones marked with ?? also ought to be animated (guessed). 0x00 to 0x03 I hardly think is one loop as it includes 'special' index 0x00
2.1.4 private palettes
Some bitmaps (yet only found at uncompressed ones) have their own special palette stored past the bitmap bits + 4 bytes. ie:
HEADER (size 0x001C) BITMAP BITS (size width * height) unknown (size 0x0004, int32 - found to be 1, could be flag for pal) PRIV PALETTE (size 0x0300)
When in search for some example bitmaps with private palettes, look for the chunk 0x073A (System Shock), which contains 3 images - the three intro screens.
All sounds are stored in 8bit, mono, linear signed format, either at 11 or 22kHz.
Digitised sound effects reside in the file digifx.res and have chunk type 07. These are simply Creative Labs .voc files embedded in the resfile, one chunk each. Check Wotsit (www.wotsit.org) for the format.
Audio logs reside in the files citalog.res (English), geralog.res (German) and frnalog.res (French) and have chunk type 0x11 (17). Chunk IDs are shared between the languages (i.e. the same log will have the same ID in each language file). If the text of a log has chunk ID n, the audio sample will have chunk ID 300+n.
Audio logs seem to be embedded movie files of some description but I don't know which format they are in yet.
Fonts have content type 3. They are stored in gamescr.res chunks 602 - 613 and chunks 1840 - 1845
The font header looks like the following: 0000 int16 color flag. Is 0 for 1 bit fonts and 0xCCCC for colored 8 bit fonts 0002 int8 34 * unknown (always zero) 0018 int16 first ASCII character in the font 001A int16 last ASCII character in the font 001C int8 32 * unknown (always zero) 003C int32 offset to position table 0040 int32 absolute offset to bitmap data from start of chunk 0044 int16 bitmap width (in bytes) 0046 int16 bitmap height (in pixels)
The width of a glyph in pixels calculates as offset(i+1) - offset(i), that is the offset of the next glyph substracted with the offset of the current glyph.
The size of the positon table is last character - first character + 2 int16s. last - first +1 characters are stored in the font. The last offset in the position table is only used to calculate the width of the last glyph.
The values in the position table are always offsets in pixels from the beginning of the current bitmap line. This is important for 1 bit fonts, where the actual byte containing the pixel is offset/8. For 8 bit colored fonts pixels and bytes are the same.
The glyphs itself is stored in one big bitmap with width * height bytes. All glyphs lie in one "row" in that bitmap.
MOVI Format:
MOVI Chunks are itself chunk directories. Length and type are implicitly taken from the index table from the first MOVI (master) chunk.
Master chunk 'MOVI' HEADER size 256 bytes (including 'MOVI') 0000 4xint8 'MOVI' 0004 int32 no. directory entries 0008 int32 size of index table 000C int32 size of the contents (excluding HEADER, PAL, INDEX) 0010 int32 length of movie (in 1/65536 seconds) 0014 int32 No idea 0018 int16 width of video 001A int16 height of video
0026 int16 sample rate of audio (?)
PALETTE size 3*256 bytes
the initial palette to be used for video.
INDEX TABLE list of Index Entries, each of the form:
0000 3xint8 Timestamp. This is an 8.16 fixed point number
of seconds.
0003 int8 type of entry:
b0-2 Chunk type
b3-6 Additional info.
Chunk types are:
0 End of movie
1 Video frame
2 Audio (PCM, 8bit, 1Channel)
3 Subtitle or control (text)
4 Palette
5 Dictionary (used in hi-res codec)
0004 int32 offset of the subchunk (starting from beginning)
0 End of movie This marks the end of the chunk directory. It only really exists so that the length of the last chunk may be calculated (there is no "length" field, so length is taken to be [offset next chunk] - [offset this chunk] ).
1 Video subchunk The 4 bits "additional info" in the type field gives the compression format. 2 are used: low-resolution cutscenes are format 4 (type=0x21) and use the "type 4" bitmap format described in section 2.1.2. Such a frame is preceded by an 8-byte bounding box definition (System Shock ignores this; it is always set to the entire frame size). 0000 int16 start_x 0002 int16 start_y 0004 int16 width 0006 int16 height 0008 compressed bitmap data
High-resolution cutscenes use format 0xf (type=0x79). This compression format is rather more sophisticated and a lot more complicated. It uses two auxiliary chunks, which are defined at the start of each scene (alongside the palette) and which remain constant throughout the scene. These are the auxpal and dictionary chunks of type 5, described below. The video chunks themselves are divided into 2 sections. The basic format of a video frame is 0000 int16 Offset (in bytes from start of frame) to section 2 (xxxx) 0002 ... Section 1: main packed data stream xxxx ... Section 2: pixel data
A high-resolution video frame is divided into 4x4 pixel tiles (so a 600x300 frame is 150x75 tiles in size). Each tile is unpacked independently of all the others. The real key to the frame is (unsurprisingly) the frame chunk section 1. This is interpreted as a (big-endian) bitstream with variable word length. To unpack a frame given its packed chunk and the (previously read) dictionary and auxpal chunks, we proceed as follows: 0. Read 12 bits from the packed stream. This forms an offset d (0-0xfff) into the dictionary chunk. (Not all 12 bits necessarily belong exclusively to this word, see below). 1. Take the d'th (counting from 0) 24-bit word from the dictionary chunk. This is the control word c. A control word is made up as follows: bits 0-16 (0x01ffff) Parameter field bits 17-19 (0x0e0000) Type field bits 20-23 (0xf00000) Count field 2. A count field of zero is a long offset. The base offset consists of the type and parameter field taken together (i.e. is a 20-bit offset 0-0xfffff). The next 4 bits from the packed data stream (after the offset d) are added to this, and the result forms a new offset into dictionary chunk. We then return to step 1 to collect a new control word. This enables much larger dictionary chunks, however the later parts of the chunk can only be reached at the cost of reduced compression. 3. The (nonzero) count field of the current control word is the number of bits by which to advance the pointer into the pack stream. If this is less than 12, the low bits of the current offset will of course form the high bits of the next. Since the dictionary chunk is run-length encoded (see below), a word may be repeated without taking up more space on disc, allowing the low bits of the offset to vary in order to accommodate the next. This is a cunning way of squeezing some bits (on average) out of the packed tile. 4. The action that is now taken depends on the type field. Different types may require parameters to be taken from the frame chunk section 2 and/ or the auxpal chunk: Type 0: The parameter field is interpreted as 2 literal pixel values (8-bit palette indices). These are duplicated twice horizontally (low high low high, from L-R) and 4x vertically to make a 4x4 tile. Note that a zero index here is NOT counted as transparent. Type 1: The parameter field is interpreted as 2 palette indices (zero is transparent): the high byte corresponds to a 1 bit in the pixmap and the low to a 0. The next 16 bits from frame section 2 are treated as 16 1-bit pixels: bit 0 (0,0), bit 1 (1,0), bit 4 (0,1) and so on to make a 4x4 tile. Type 2: The parameter field is interpreted as an offset into the auxpal chunk giving 4 palette indices. The next 32 bits from section 2 are treated are 16 2-bit pixels. Type 3: As type 2, but with 8 palette indices and a 48-bit word of 3-bit pixels from section 2. Type 4: As type 3, but 4 bits per pixel, 16 indices, 64 bits. Type 5: Skip. The parameter field is ignored[1]. The next 5 bits from the packed stream (section 1) gives the number of tiles to skip: a value of 0x1f here means skip the rest of the row. Type 6: Repeat previous control word[2]. Type 7: As type 6 (this value is not used). 5. If the frame has not yet been fully unpacked, return to step 0.
Notes on the high-resolution frame format: [1] The real decompression algorithm proceeds row-by-row, making an intermediate list of all the control words for that row in a row buffer. The parameter field of a type 5 control word is replaced by the parameter from the packed stream at this stage, and forms the offset when the row buffer is unpacked into the image. [2] This can improve compression ratios if two consecutive tiles or skips can be represented by the same control word (this doesn't necessarily mean that tiles are identical, since they may take bitmap or offset data from the frame). A copy word may well take fewer bits to reach than the previous full word.
2 Audio subchunk This is just raw 8-bit audio data.
3 Subtitle subchunks 0000 4xint8 subtitle control 0004 uint16 sizeof of header (?) 0004 8xint8 unknown 0010 null terminated string
Subtitle control 'AREA' Presumably it defines the area for the subtitles, it appears only once in the movie (preceeding all subtitles). example: "10 165 310 195 CLR"
Subtitle controls 'STD ', 'FRN ', 'GER ' which language the subtitle (textpart) is in (given that the game was written by Americans, std = english, of course). The null terminated string is the text to display
4 Palette subchunks Both types 0x04 and 0x4C always come up in pairs and point to the same offset. (Could be because older and newer type version of the file format). As the sizes of those subchunks always is 0x0300 bytes I assume they contain palette information - which is confirmed.
5 Dictionary There are 2 different types of chunk here, which always appear in pairs at the start of a scene. The auxpal chunk (type=0x05) is very simple: it consists of an (unpacked) list of palette indices making up the auxpals for the image tiles (see above). The dictionary chunk (type=0x0d) contains all the control words for the video codec. It uses a simple run-length coding scheme: the top 8 bits of each 32- bit word in the packed chunk gives the number of occurrences, the low 24 bits are the control word. The basic format of the dictionary chunk is: 0000 uint32 Size of unpacked chunk (in bytes, 3 to a word) 0004 ... Packed dictionary chunk.
The game maps are stored in the file archive.dat which contains 834 chunks starting at ID 4000. The sharp-eyed amongst us will note that 834 == 52 * 16 + 2. I would have thought 64 * 13 would be more logical as there are 10 levels (1-9 + R) and 3 groves, but the chunk sizes clearly repeat in blocks of 52. I haven't yet examined the maps thoroughly enough to figure out the extra 3: perhaps they are to do with cyberspace?
3.0.1 Level list
Reactor Map 0 (chunk 40xx)
Levels 1-9 Map L (chunk 4Lxx)
SHODAN c/space Map 10 (chunk 50xx)
Delta grove Map 11 (chunk 51xx)
Alpha grove Map 12 (chunk 52xx)
Beta grove Map 13 (chunk 53xx)
C/space L1-2 Map 14 (chunk 54xx)
C/space other Map 15 (chunk 55xx)
(Thanks to Glen Sawyer glen_s@enol.com for compiling the list)
Each map uses 52 blocks but has IDs allocated for 100. Chunks 4000 and 4001 are not specific to any individual map, so level R uses chunks 4002-4053, level 1 4102-4153 and so on.
3.0.2 Archive name
This resides in chunk 4000 and consists of a C-style string containing the archive name. In the original archive.dat this is "Start game" with a great deal of trailing garbage to a length of 128 bytes. When the map is first archived to change levels this becomes "Starting Game" and keeps the trailing junk. In a save game proper (savgamXX.dat) the save game name makes up the whole chunk.
3.0.3 Player information
This resides in chunk 4001.
The first 6 chunks (xx02-xx07) in each map (seem to) contain information about the level geometry.
3.1.1 Chunk xx02 3.1.2 Chunk xx03
These chunks are only 4 bytes long, containing a single 32-bit word. I don't know what they mean yet tho.
3.1.3 The level information chunk
This resides in blocks 4004, 4104 etc. and contains miscellaneous information about the level.
0000 int32 \ Map size in tiles 0004 int32 / 0008 int32 ?? always 6 could be log2 map size 000C int32 ?? always 6 0010 int32 log2 (no. height units per tile width). If this value is x, a (non-sloping) tile with height 2**x will be a perfect cube. 0014 int32 This is a placeholder for the tile map pointer when the chunk is loaded into memory. It is meaningless on disc 0018 int32 Cyberspace flag. 1 if level is c/space, 0 otherwise
3.1.4 The tile map
This resides in blocks 4005, 4105 etc. and is always compressed. The unpacked chunk has size 0x10000 (aha! 166464, you say). This is a 64x64 grid of tiles, starting at the bottom left corner of the level, where each tile does indeed (thanks Jim!) have format
0000 int8 Type. I have so far identified 00 solid 01 open 02 diagonal open s/e 03 diagonal open s/w 04 diagonal open n/w 05 diagonal open n/e 06 slope s->n (all slopes expressed as low->high) 07 slope w->e 08 slope n->s 09 slope e->w though there are certainly more types defined. Perhaps there are diagonal slopes as well. 10-Jun-2000 Glen has now identified the diagonals: 0A slope se->nw valley 0B slope sw->ne valley 0C slope nw->se valley 0D slope ne->sw valley 0E slope nw->se ridge 0F slope ne->sw ridge 10 slope se->nw ridge 11 slope sw->ne ridge A "valley" tile has 3 vertices level and one lower: the floor is split into 2 triangular sloping sections along the diagonal from the lower to the opposite vertex. A "ridge" tile has 3 vertices level and one higher, and is likewise split along the diagonal. (This means that ridged tiles are no longer completely convex and need careful handling in 3D). If that makes sense I'll be impressed 8-) 0001 int8 Floor. bit 0-4 floor height bits 5-6 texture orientation bit 7 also hazard flag? 0002 int8 Ceiling. These are bit 0-4 height (ceiling is in units DOWN from top) bit 5-6 texture orientation bit 7 hazard flag (floor contains biohazard, ceiling radiation hazard flag) 0003 int8 Steepness of slope (only bit 0-4 used) 0004 int16 Index into object xref table of first object in tile 0006 int16 Texture info 0008 int32 Flags 000C 4xint8 State (?) These always seem to contain FF 00 00 00 in the archive.dat file; presumably they contain game flags when levels are archived in saved games.
It appears that slopes can refer to floor, ceiling or both, presumably controlled by bits in the flag word. This is controlled by bits 10-11 in the flag word as follows: xxxxx0xx Floor & ceiling, same direction xxxxx4xx Floor & ceiling, ceiling opposite dir to tile type xxxxx8xx Floor only xxxxxCxx Ceiling only Only 30 flag bits still to go 8-)
Some further info about valleys, ridges and inverted ceilings: An uninverted ceiling of a valley floor looks like a ridge (and vice versa), so that in each case the ceiling and the floor could fit into the other. At an inverted ceiling, the 'mirrored' type of the other group is used; As an example: The inverted ceiling of a SE->NW valley takes the ceiling info of a NW->SE ridge tile. To verify these cases look into levels 7 and 9: 7 (antenna rooms): Here valley (floor) tiles have inverted ceilings, also valleys 9 (CPU room): Here valley (floor) tiles have non-inverted ceilings, thus ridges
Some info about map heights/slope: It appears that Shock uses 8 bit range for the map heights (object heights can go over 127) however only 5 bits are stored for floor/ceil height. Also the slope is stored as a 8 bit value, it however also uses the 5 bit range, so the first 3 bits appear to be unused.
Texture info: It appears that each level may use a maximum of 64 textures out of the 273 available. This strongly suggests that 6 bits are used per texture. My current best guess at bits in the texture info word is: 0-5 Wall texture (index into texture list) 6-10 Ceiling texture 11-15 Floor texture It appears that there are only 32 floor/ceiling textures available. The height bytes don't supply the missing bit; heights are definitely signed. 13-Jun-2000 That last bit was a lie. Only the bottom 5 bits of the height bytes are used for tile heights. The others seem to be environmental flags. Each tile may only store ONE wall texture (unless there are others stored elsewhere that I haven't yet found out about). They may clearly pick and choose between their own and adjacent textures; current best guess is that this is controlled by bit 8 of the flags word.
Flags: 80000000 tile visited (automapper) 0F0F0000 shade control 0000F000 music nibble 00000C00 slope control 00000200 Spooky music flag? This appears to be set for areas which have been largely remodelled by SHODAN's forces. 00000100 use adjacent rather than local textures for walls 0000001F vertical texture offset adjust
3.1.6 The texture list
This resides in blocks 4007, 4107 etc. and contains a 16-bit word for each texture used by the level, up to a maximum of 64. A texture ID as stored in the tile map is an index into this list.
An "object" in System Shock can be anything that isn't part of the basic level geometry itself i.e. not a wall or floor texture. This includes all items, sprites, 3D models, decals, doors and gratings, and invisible stuff such as traps and triggers.
An object is generally identified by class, subclass and type. This forms a hierarchy of object classification from coarsest (class) to finest (type). This is denoted in this document and elsewhere as class/subclass/type, e.g. the Cyberjack is object 12/0/4 .
Object classes are 0 Weapons 1 Ammo 2 Projectiles 3 Grenades and explosives 4 Patches 5 Hardware 6 Software & logs 7 Scenery and fixtures 8 Gettable and other objects 9 Switches and panels 10 Doors and gratings 11 Animated objects (?) 12 Traps and markers 13 Containers (includes corpses) 14 Critters
The object information is stored from chunk xx08 to xx24 inclusive. The first two tables give general information about the objects and their positioning in the level map. The remaining 15 are each specific to a given object class, and contain extra information about the objects in that class.
3.2.1 The master object table
This resides in chunks 4008, 4108 etc. and contains an entry for everything in the level that is not part of a tile (i.e. a wall, floor or ceiling). Each entry is 27 bytes long as follows:
0000 int8 in-use flag. 0 slot is free, 1 in use.
0001 int8 object class
0002 int8 object subclass
0003 int16 class index. This is an index into the class specific table in
one of the following chunks.
0005 int16 index into object cross-reference table (next chunk)
0007 int16 prev link
0009 int16 next link
000B int16 x coord (high byte is tile x)
000D int16 y coord (high byte is tile y)
000F int8 z coord (?)
0010 int8
0011 int8 } These seem to be the 3 angles for 3d positioning
0012 int8 /
0013 int8 ?? AI index - is 0xFF for all but damageable things (critters
and crates)
0014 int8 object type
0015 int16 Hitpoints? Initial values tend to be round decimal numbers
0017 int8 State (sprite frame)
0018 3*int8 unknwon
3.2.2 The object cross-reference table
This resides in chunks 4009, 4109 etc. and is used to link map tiles with the objects that they contain. The "index" field in the tile map is an index into this table. Entries themselves contain an index field which is used to chain objects together when there is more than one object in a map tile.
Objects which extend over more than one tile get an entry in this table for each tile which partially contains them. Entries for a single object and multiple tiles are linked by the 5th field (0008) while entries for a single tile and multiple objects are linked by the 4th (0006).
An object cross-ref entry consists of 10 bytes as follows:
0000 int16 Tile x position 0002 int16 Tile y position 0004 int16 Index into master object table 0006 int16 Cross-ref index for next object in tile 0008 int16 Cross-ref index for next tile object extends into
3.2.3 The weapons table, class 0
This resides in chunks 4010, 4110 etc. and contains special info on weapons. Each entry consists of 8 bytes as follows:
0000 int16 Weapon index in master object table 0002 int16 "Prev" link for slot list 0004 int16 "Next" link for slot list 0006 int8 Ammo type (projectile) or charge (energy) 0007 int8 Ammo count (projectile) or ?temperature (energy)
3.2.4 The ammo table, class 1
This resides in chunks 4011, 4111 etc. and contains special info on ammo clips. An ammo clip is an ammo clip is an ammo clip, really, so this chunk isn't very interesting; it has 6 bytes in it:
0000 int16 Ammo clip index in master object table 0002 int16 "Prev" link for slot list 0004 int16 "Next" link for slot list
3.2.5 The projectile table, class 2
This resides in chunks 4012, 4112 etc. and is not used in the map archive for obvious reasons. It might be used in saved games.
3.2.6 The grenades / explosives table, class 3
3.2.7 The patches table, class 4
This resides in chunk 4014, 4114 etc. and contains information about the dermal patches. There is no special information on these, so this table just contains the master object cross-ref and the links for the slot list.
3.2.8 The hardware table, class 5
This resides in chunks 4015, 4115 etc. and contains information on hardware. Each entry is 7 bytes long:
0000 int16 Hardware index in master object table 0002 int16 "Prev" link for slot list 0004 int16 "Next" link for slot list 0006 int8 Version
3.2.9 The software / logs table, class 6
This resides in chunks 4016, 4116 etc. and contains information on software and logs. Each entry is 9 bytes long:
0000 int16 Software index in master object table 0002 int16 "Prev" link for slot list 0004 int16 "Next" link for slot list 0006 int8 (Softs) Version no. of software 0007 int8 (Log) Log chunk number (offset from 0x09B8 2488) 0008 int8 (Log) Level no. log refers to
3.2.10 The scenery / decorations table, class 7
This resides in chunks 4017, 4117 etc. and contains information on permanent fixtures of the station which aren't parts of walls. Each entry is 16 bytes long; the first 6 bytes are the index and slot-list links as follows, and the rest depend on the object type.
For WORDS 07:02:03 0006 int16 text (subchunk to chunk 0868 (2152)) 0008 int16 font and size 000A int16 colour (0 seems to default to red)
For animated screens (Glen figured this one out): 0006 int16 Number of frames 0008 int16 Loop repeats backwards flag 000C int16 Start frame (offset from chunk 321)
Some values of "start frame" are special: 246 Static fading into SHODAN's face 247 248-255 Surveillance ID, see "surveillance control chunk" below
For values of "start frame" greater than 255 the low 7 bits give a text message (subchunk of text chunk 0877 2167) to be rendered onto the screen. Here 127 0x7F is the special value; it is used for the random numbers in the CPU rooms on levels 1-6 before the nodes have been destroyed. If bit 7 is set for a text message the text scrolls vertically. Each frame consists of several strings, starting at (start frame & 0x7f) + (current frame). The number of strings per frame is simply the number that will fit on the screen; partial lines are not drawn.
For bridges subchunk 7 (Glen again): 0008 int8 bits 0-3 X size (4 is tile width) bits 4-7 Y size (4 is tile width) - 0 is bridge's normal size in its 3D model 0009 int8 bridge height (0 is default) 32 units per texture height 000A int8 bits 0-6 top/bottom texture bit 7 set if texture comes from the main textures referred to in chunk xx07, otherwise it is taken from the 3D model texture maps in citmat.res . 000B int8 side textures (similarly)
Note that in CYBERSPACE levels (10, 14, 15) fixtures are not used as such, but are co-opted as extra softs/logs in case that table becomes full, and act as objects of class 6. From a cursory investigation the fixture data in this case seems to be: 0006 int16 version no. (softs) 0008 int16 softs/logs subclass 000C int16 softs/logs type
3.2.11 The items table, class 8
3.2.12 The switches / panels table, class 9
This resides in chunks 4019, 4119 etc. and contains information on switches and panels. Each entry is 30 bytes long, having the first 12 Bytes in common; the second half is specific to switch type. As it seems, the State of the switch (mostly Puzzles) isn't stored within this table.
0000 int16 Panel index in master object table 0002 int16 "Prev" link for slot list 0004 int16 "Next" link for slot list 0006 int16 unknown?? 0008 int16 Condition: Variable Index 0010 int16 Condition: Message on fail
Number Pads (9 3 7): 000C int16 Combination in BCD 000E int16 Map Object to trigger 0018 int16 Map Object to Extra Trigger (?)
Puzzles (9/3/0 to 9/3/3)
These are either wire or block (power) puzzles. The dword at offset 0x10 seems to be the determining factor: if bit 28 is set (0x10000000) it is a block puzzle, else it is a wire puzzle.
For both types the word at offset 0x0C is a reference to a map object to frob when the puzzle is completed.
For a wire puzzle: 0010 int8 Size (nibble0: Wires (default: 4 if 0), nibble1: Connectors per side (default: 6 if 0)) 0011 int8 Power level to be reached (out of 0xFF) 0012 int16 unknown 0014 int32 Target State of Wires 0018 int32 Current State of Wires The States are stored in 3bit pairs from right to left (first pair: first wire, second pair second, ...) the first triple states the left connector, the second the right one. (so a maximum of 8 connectors is possible and maximum of 5 Wires (32 / 6 = 5)
For a block puzzle: 0010 int32 "Helper" trigger object for state (is an Action 0x00 Trigger) Bit 28 of this field is set to indicate that it is a block puzzle. 0016 int32 Puzzle information: b4-6 Y coord of power source connector b7-8 Source direction (10=left) b12-14 Y coord of power destination connector b15-16 Destination direction (11=right, 00=up) b20-23 Width b24-27 Height b28-31 Side effect type
The actual state of the puzzle is stored in the "helper" object's trigger info, from offset 0x10 on. Each block has 3 bits describing what is in it. Blocks are stored from top left to bottom right in the usual order, but the way in which they are encoded is slightly complicated. Puzzle state is read in 32-bit words starting at the LAST dword in the trigger info, and the block descriptors are rotated out at the bottom. When the word has been fully examined, any bits left over are kept and combined with enough bits from the bottom of the previous word to make up a 3-bit block descriptor. Thus the top left block is described by the bottom 3 bits of the last trigger word (the bottom 3 bits of the byte at offset 0x1C), the next block to the right by bits 3-5 of the same word, and so on until the 11th block, if the puzzle is that large. This is made up of the top 2 bits of the last word (bits 6-7 of byte 0x1F) as its low 2 bits and the bottom bit of the penultimate word (bit 0 of byte 0x18) as the high bit. The 12th block is taken from bits 1-3 of the penultimate word, and so on. It might be simpler just to look at Trig_get_block_puzzle() in src/trigger.c for clarification of the above.
Block types are: 00 Empty 01 Inactive connector (x) 02 Active connector (+) 04 Solid block 06 Switching node (hollow square)
Panels: yet unknown
Buttons (9 0 2): yet unknown
Cyberjacks: 000C int16 X of target Cyberspace 0010 int16 Y of target Cyberspace 0014 int16 Z of target Cyberspace 0018 int16 Level (Cyberspace)
Elevators (9 3 5): 000C int16 Map index of Panel of target Level1 000E int16 Map index of Panel of target Level2 0012 int16 Map index of Panel of target Level3 0018 int16 Bitfield of accessible Levels (Actual) 001A int16 Bitfield of accessible Levels (Shaft) Levels with a 1 in the "shaft" field but not in the "Actual" field give a "Shaft damage: Unable to go there" message.
3.2.13 The doors / gratings table, class 10
This resides in chunks 4020, 4120 etc. and contains information on doors and gratings. Each entry is 14 bytes long:
0000 int16 Door index in master object table 0002 int16 "Prev" link for slot list 0004 int16 "Next" link for slot list 0006 int16 ?? trigger cross-ref 0008 int16 Message 000A int8 Access required 0-31
3.2.14 The animations table, class 11
3.2.15 The traps and triggers table, class 12
This resides in chunks 4022, 4122 etc. and contains information on traps and triggers.
A trigger has a type and an action. The type is stored with the generic object definition in the master object table and determines how the trigger is set off. The action is stored with the trigger definition in this table and determines what happens. Types of trigger are
Entry 0C 00 00 Player enters trigger's tile
Null 0C 00 01 Not set off automatically, must be
explicitly activated by a switch or
another trigger
Floor 0C 00 02
Player death 0C 00 03 Player dies. These are used to
resurrect the player if the
resurrection machine has been reset
Deathwatch 0C 00 04 Object is destroyed / dies
AOE entry 0C 00 05
AOE continuous 0C 00 06
AI hint 0C 00 07
Level 0C 00 08 Player enters level
Continuous 0C 00 09
Repulsor 0C 00 0A Repulsor lift floor
Ecology 0C 00 0B
SHODAN 0C 00 0C
Tripbeam 0C 01 00
Biohazard 0C 02 00
Rad hazard 0C 02 01
Chem hazard 0C 02 02
Map note 0C 02 03 Map note placed by player (presumably)
Music mark 0C 02 04
Trigger data is 28 bytes long. The first 12 bytes have the same format for all triggers; the remaining 16 depend for their interpretation on the action.
0000 int16 Trigger index in master object list 0002 int16 "Prev" link for slot list 0004 int16 "Next" link for slot list 0006 int8 Action 0007 int8 Once-only flag? 0 or 1 0008 4xint8 Condition
The condition is usually a game variable and value, but depends on the trigger type; for deathwatch triggers it is the class and type of the object(s) being watched.
Trigger actions are
00 Do nothing / default action (switch)
01 Transport (elevator panel / cyber term)
02 Resurrection?
03 Clone object
000C int16 Object to transport.
000E int16 Delete flag?
0010 int16 Tile destination X
0014 int16 Tile destination Y
0018 int16 Destination height?
04 Set variable
000C int16 variable to set
0010 int16 value
0012 int16 ?? action 00 set 01 add
0014 int16 Optional message to receive
06 Activate / Open. Set off triggers, open doors.
000C int16 1st object to activate.
000E int16 Delay before activating object 1.
0010 ... Up to 4 objects and delays stored here.
07 Change lighting
000C int16 Control point 1
000E int16 Control point 2
... ?
08 View "Static" effect
09 Moving platform
000C int16 Tile x coord of platform
0010 int16 Tile y coord of platform
0014 int16 Target floor height
0016 int16 Target ceiling height
0018 int16 Speed
0C Choice. Set off trigger depending on [what?]
000C int16 Trigger 1
0010 int16 Trigger 2
0F Player receives email
000C int16 Chunk no. of email (offset from 2441 0x0989)
10 This is used in the radiation treatment area on level R.
13 Change object state.
16 Trap message
000C int16 "Success" message
0010 int16 "Fail" message
17 Spawn
000C int32 Class/subclass/type of object to spawn
0010 int16 Control point 1 (object)
0012 int16 Control point 2 (object)
0014 ??
0018 ??
18 Change type. This is used for force bridges / doors
000C int16 Object ID to change.
0010 int8 New type.
0012 ??
3.2.16 The containers table, class 13
This resides in chunks 4023, 4123 etc. and contains information on containers. As the name suggests, a container is an object that may contain other objects; this includes corpses and dead monsters as well as crates etc. Each entry is 21 bytes long:
0000 int16 Container index in master object table 0002 int16 "Prev" link for slot list 0004 int16 "Next" link for slot list 0006 4xint16 Up to 4 objects contained 000E int8 Width (for crates) 0 means use default 000F int8 Height (for crates) 0 means use default 0010 int8 Depth (for crates) 0 means use default 0011 int8 Top texture (for crates) 0 means use default 0012 int8 Side texture (for crates) 0 means use default 0013 int16 ??
Crates, like bridges, may specify their dimensions and texture mapping information independently of the actual 3D model they are associated with (which is just a placeholder and is ignored). Default dimensions are 80x80x80 for a "small crate" (13/0/0), 160x160x160 for a "large crate" (13/0/1) and 240x240x240 for a "secure crate" (13/0/2). Textures are taken from the special model texture block from chunk 2180.
3.2.17 The critters table, class 14
==============================
The surveillance control chunk
This resides in chunks 4043, 4143 etc. and controls surveillance screens i.e. those displaying live scenes from within the 3D world.
It contains a maximum of 8 16-bit words giving the object IDs of up to 8 "null" trigger objects; these are dummy objects which exist only to provide a position and orientation for the camera transform associated with that screen.
Objects referred to in this chunk are linked by special values in the "start frame" field of their respective screens. Special start frames 248-255 refer to words 0-7 in this chunk. Thus if a screen has start frame 248, the first word in the surveillance control chunk is used as an object ID to look up an object whose position and orientation are then used to render a scene into a bitmap, which in turn is projected onto the screen.
=============================== Logs, eMails, vMails, Data Fragments (From Rebecca)
Those texts are stored in blocks of Strings (String arrays). Information about them is also stored in this block: Line Content 0 Info 1 'Title' (that appears in lists) 2 Sender 3 Subject 4 to n-1 Verbose Text n Empty Line ("") n+1 to m-1 Terse Text m Empty Line ("")
Info Line: has the following format: [event ][colour ]LeftId[,[ ]RightId] event: 'iEE' or 't' EE = Hex Number of Log/eMail to follow immediately 't' is set for Texts following a 'iEE' Text colour: 'cCC' CC = Hex Number of Colour Index in Palette; only Sender and Subject are drawn in this colour LeftId, RightId: decimal subchunk number of left (and right) bitmaps to show; (based from main chunk ID 0x28) Note that the blank between ',' and RightId is omitted sometimes...
vMails only have a number between 256 and 261 in this line - but don't match any bitmaps -- orig SS even skips Sender and Subject Lines (since there is no bitmap where they could be shown)
Title, Sender, Subject: always one line
Verbose and Terse Text: although the texts are torn apart, those breaks do not mark Newlines. Instead, character 0x0A does this. Character 0x02 marks possible soft hyphens (but as it turned out, not all texts are formatted this way). The string '$N' is a placeholder for the hackers name.
=============================== Notes (Sheets lying on the ground on Citadel)
Same as above, those Texts are stored in string arrays. They don't have any special formatting or different versions, just one block of Text form the first line on and end with one empty line.
This file is not a resource file, it is a flat file containing tables of miscellaneous object information. For each object class, there is a general table, followed by a table for each subclass. Each object (with very few exceptions) therefore has 2 table entries. If there is nothing of interest to be defined, the entry may be a single zero byte.
The object properties file has a "header" consisting of a single 4-byte integer, 0x0000002d, purpose unknown to date.
The generic weapon info begins at file offset 0x0004, and has 2 bytes per weapon to a total of 32:
0000 int8 If I were to hazard a guess, I might surmise that this was involved with firing rate 0001 int8 This controls what types of clip the weapon takes. b0-3 clip types. There is a bit for each of a possible 4 types within the subclass, if set the weapon accepts that clip (0-3) b4-7 clip subclass This byte is zero for energy weapons, of course.
In the following, a "common weapon information" structure refers to an 8-byte table as follows
0000 int16 Damage 0002 int8 "Offence" value 0003 int8 Damage type. This seems to be organised as a bitfield: 01 impact 02 energy 04 EMP 08 ion (the ion rifle has this bit set) 10 gas 20 tranq 40 needle (SV needle darts and full-auto rounds) 0004 int8 Seems to have to do with special effects. EMP weapons have 0x33 here 0005 int16 Not used in the main weapons table. Seems to be used for critter attack descriptions 0007 int8 Armour penetration
4.0.0 SEMI-AUTO WEAPON TABLE, class 0/0
There is no extra information on these, and the table consists of 5 zero bytes.
4.0.1 SEMI-AUTO WEAPON TABLE, class 0/1
There is no extra information on these, and the table consists of 2 zero bytes.
4.0.2 PROJECTILE WEAPON TABLE, class 0/2
This table contains 16 bytes per weapon in this subclass, 32 in total:
0000 8byte common weapon info 0008 int8 0009 int32 Projectile class/subclass/type
4.0.3 MELEE WEAPON TABLE, class 0/3
This table contains 13 bytes per melee weapon, 26 in total:
0000 8byte common weapon info 0008 int8 Energy usage 0009 int8 ?? kickback ?? 000a int8 ?? Range ??
4.0.4 ENERGY BEAM WEAPON TABLE, class 0/4
This table seems to have the same values as the melee weapons table above.
4.0.5 ENERGY PROJECTILE WEAPON TABLE, class 0/5
This table contains 18 bytes per projectile weapon, 36 in total:
0000 8byte common weapon info 0008 int8 Energy usage 000d int32 Projectile class/subclass/type
The generic ammo clip info begins at file offset 0x00B0, and has 14 bytes per ammo clip to a total of 210:
0000 8byte common weapon info 0008 int8 No. rounds per clip 0009 int8 ?? kickback ?? 000a int16 000c int8 ?? Range ?? 000d int8
The "specific" info for ammo clips just consists of a single zero byte each.
The generic projectile info begins at file offset 0x0191, and has 1 byte per projectile.
4.2.0 TRACER TABLE, class 2/0
This table has 20 bytes reserved for each projectile in this class, to a total of 120. In the file objprop.dat they are all zero.
4.2.1 PROJECTILE TABLE, class 2/1
This table contains 6 bytes per projectile in this class, to a total of 96. These control the cyberspace model colour scheme. Naturally they are only used for the c/space projectiles.
4.2.2 class 2/2
This table has a single zero byte for each object in this class, 2 in all.
The generic explosives info begins at file offset 0x0283, and has 15 bytes per explosives type, to a total of 120.
0000 8byte common weapon info ...
4.3.0 GRENADES TABLE, class 3/0
This table has a single zero byte for each object in this class, 5 in all.
4.3.1 EXPLOSIVES TABLE, class 3/1
This table has 3 bytes for each object in this class, to a total of 9.
The patch info begins at file offset 0x0309. Generic: 22 bytes, all zeros. Specific: 1 byte, all zeros.
Beginning at file offset 0x03AA. Generic: 9 bytes, all zeros. Specific: 1 byte, all zeros.
Beginning at file offset 0x0440. Generic: 5 bytes, all zeros. Specific: 1 byte, all zeros.
Beginning at file offset 0x04C4. Generic: 2 bytes, all zeros. Specific: 1 byte, all zeros.
The common items info begins at file offset 0x05ab and has 2 bytes per item in class 8, to a total of 160 - all zeros.
4.8.0 JUNK class 8/0 0x064b 8x1 bytes - zero 4.8.1 DEBRIS class 8/1 0x0653 10x1 bytes - zero 4.8.2 CORPSES class 8/2 0x065d 15x1 bytes - zero 4.8.3 ITEMS class 8/3 0x066c 6x1 bytes - zero 4.8.4 ACCESS CARDS class 8/4 0x0672 12x1 bytes - zero
4.8.5 CYBER ITEMS TABLE, class 8/5
This table begins at objprop.dat offset 0x067e and contains 6 bytes per cyber item, 72 in all, containing the colour scheme for each.
4.8.6 STAINS class 8/6 0x06c6 9x1 bytes - zero 4.8.7 QUEST ITEMS class 8/7 0x06cf 8x2 bytes - zero
The common switch table begins at objprop.dat offset 0x06df and has but a single zero byte per switch object, to a total of 35.
There is NO table for vending machines class 9/4, and no space allotted in objprop.dat . These don't appear in the game and were obviously an intended element that didn't make it.
4.9.0 SWITCHES class 9/0 0x0702 9x1 bytes - zero 4.9.1 RECEPTACLES class 9/1 0x070b 7x1 bytes - zero 4.9.2 TERMINALS class 9/2 0x0712 3x1 bytes - zero 4.9.3 PANELS class 9/3 0x0715 11x1 bytes - zero 4.9.4 VENDING class 9/4 ------ --- 4.9.5 CYBERTOGGLES class 9/5 0x0720 3x1 bytes - zero
Beginning at file offset 0x0723. Generic: 1 byte, all zeros. Specific: 1 byte, all zeros.
Beginning at file offset 0x0775. Generic: 2 bytes
0000 uint16 unknown
4.11.0 ???? class 11/0 9x1 bytes - zero 4.11.1 ???? class 11/1 11x1 bytes - zero
4.11.2 ????, class 11/2
This table contains 1 byte per animated in this subclass:
0000 byte unknown
Beginning at file offset 0x07DB. Generic: 1 byte, all zeros. Specific: 1 byte, all zeros.
Beginning at file offset 0x0801. Generic: 3 byte, all zeros. Specific: 1 byte, all zeros.
Beginning at file offset 0x08B9. Generic: 75 bytes per critter (!):
0000 75byte unknown
4.14.0 ???? class 14/0 9x1 bytes - zero 4.14.1 ???? class 14/1 12x2 bytes - zero
4.14.2 ????, class 14/2
0000 uint16 unknown
4.14.3 CYBER, class 14/3
0000 6byte colour scheme
4.14.4 ???? class 14/4 2x1 bytes - zero
The very last table in the file is the common object properties; every object in the game has an entry here, 27 bytes per object (476 in total):
0000 int32 ??? mass (in units of 100g) 0004 int16 hitpoints 0006 int8 armour 0007 int8 render type 01 3D object 02 sprite 03 screen 04 critter 06 fragments (e.g. the Cyberdog) 07 not drawn 08 oriented surface (door, wall decoration) 0B special case handling required 0C force door 000e int8 vulnerabilities. This has the same bit values as the weapon "type" field 000f int8 Special vulnerabilities. This relates to the "special effects" field of the weapon descriptions. Some objects are particularly vulnerable to certain types of weapon, e.g. magpulse+robots. 0012 int8 "defence" value 0014 int16 flags 0001 inventory object (main or access card) 0002 touchable (something interesting happens when touched; projectile / pushable / melee 0010 consumable; inv. item is consumed when used 0020 blocks 3d (door) when shut i.e. is opaque, don't bother drawing behind it 0100 solid but openable i.e door 0200 solid, can't be walked or fallen through 0400 ?? set for some explosions 0800 explodes on hit; missile or live grenade 0016 int16 3D model: index, in obj3d.res 0019 int8 b4-7 no. extra frames
Some notes on render type: 3D objects use the model information from chunk (2300 + "3D model" field). Critters are drawn as sprites, but with the appropriate frame based on orientation and state. "Fragments" objects have 2 bitmaps. The first contains the colour information for the fragments. The second gives the z position of each fragment: it is a grey-scale bitmap with the shade of grey (offset from colour 0xd0) giving the z value.
The in-game music in System Shock is context specific, which means the music adapts to where you are and what you do (or happens to you). To achieve this, the music is split up into sequences that can be joined dynamically. System Shock not only does this, but also dynamically lays several sequences (of different types) over each other, adding variety.
The music files (.xmi) are XMIDI files for the Miles sound system - pretty much common in games of the 90s. Some files contain only one sequence - these are for movies and main menu. The in-game files are named "thm??.xmi" - The ?? do not match to the corresponding level number, but, as the name suggests, are theme indices (See Tile info above).
0: Only technical chirps; flight deck hangars come to mind 1: Medical level 2: Level 6 3: Level 2 4: Level 5 & 7 5: Nature (Groves) 6: Bridge? 7: Elevator. Without doubt. 10: Cyberspace
For each of the theme files, corresponding .dat and .bin files exist, describing the sequences.
ff1: I don't exactly know why there are two descriptor files (with different endings) - my best guess is that one is read by the Miles sound engine, and the other by the game.
A flat binary file, with sequence specific descriptors:
0000 int8 Amount of sequences 0001 int8 Unknown -- always 0x01 0002 n16byte Sequence descriptor. Note: Not all files contain the proper number of descriptors. Best guess is that the missing are not used.
Sequence descriptor: 0000 int8 ???? often equal to byte at offset 0001 0001 int8 ???? often equal to byte at offset 0000 0002 byte always 0x00 0003 int16 ???? 0005 int8 always 0x0A -- could be length identifier of following array Note: thm10.dat, last one has 0x00 0006 10*int8 ???? contents in the range of [0x00..0x04]
Also a flat binary file, always a size of 405 bytes, that groups the sequences to theme groups. In-game there is one 'main' music track and a number of background voices.
The main track has three groups:
- Idle, quiet sequences; Played when strolling around empty areas
- Tension sequences; Played with enemies nearby
- Action sequences; During fights
The number and identity of background voices has not been fixed as of yet, but the following factors may be of relevance and need to be determined:
- Remodelled areas (tiles) -- see remodelled flag at Tile structure
- Machinery/Electrics nearby
The file seems to be split into three parts:
- Main track infos
- background voices
- unknown data?
At least the bytes of the first two parts specify the indices of a music sequence. If set to 0xFF the slot is not used.
File data, first part: 0000 - 000F up to 16 entries for quiet sequences 0010 - 0017 up to 8 entries for tension 0018 - 001F up to 8 entries for action 0020 - 0020 one (optional) entry for startup sequence (new game, load game) 0021 - 0022 unused, always 0xFF 0023 - 0023 Death sequence 0024 - 0024 Revival sequence (?) - not always set (makes sense) 0025 - 0028 unused, always 0xFF
File data, second part; 32 entries of 10 bytes length: 0029 - 0032 background voice; basic theme; note that some have the health alarm there? 0033 - 003C mainly one-note piano beats; space? some with alarm? 003D - 0046 sounds a bit eery; another background voice? 0047 - 0050 one-note piano beats -- slightly different than the other group 0051 - 0096 7x10 unused (0xFF) 0097 - 00A0 always 5 entries, always the same: the health alarm 00A1 - 00AA faster health alarm 00AB - 00BE 2x10 unused (0xFF) 00BF - 00C8 mechanical beats (some with alarms?) 00C9 - 00D2 unused (0xFF) 00D3 - 00DC always 5 entries, background for tension change (raising tension?) 00DD - 00E6 always 5 entries, background for tension change (falling tension?) 00E7 - 0122 6x10 unused (0xFF) 0123 - 012C only set for thm0 -- one entry with mechanical beat 012D - 0136 thm0 0137 - 0140 thm0 0141 - 014A thm0 014B - 0154 thm0 0155 - 0168 2x10 unused (0xFF)
File data, third part; Unknown: 0169 - 0194 ???? -- bytes are set [0x01..0x05 or ..0x07], 0x00 only at the end
Common sequences (health alarm, death, ...) are often around the same index, so they are easy to spot in the .bin files.
It is interesting that not all sequences (with meaningful data) are used according to .bin files.
The elevator theme has only the idle group set; No tension, fight or background voices set (only exception is the death sequence of course). So, regardless what happens to you in the elevator, you get that soothing melody ;)
Background voices often come in groups of 5 or 7 sequences, yet sometimes there doesn't appear to be meaningful data in them (some contain the health alarm -- why?)
Are sequences played in the same order they are specified in their group or does something affect the order?
The contents of the 10 bytes in the .dat files seem a bit gaussian distributed; Also, their sum is close to the first two bytes of the sequence descriptor...
This is not a resource file, but a flat file containing an 11-byte record for each texture in the game, structured as follows:
The file is larger then the maximum number of textures in the game (273) It seems like the data after those 273 entries is just crap?
It appears that the file content starts at byte 4 Each block has the following layout:
0004 int8 \ These are usually the same as the low byte of the texture no. 0005 int8 / 0002 int32 unknown (flags?) (always 10???) 0006 int8 climbable flag 0007 int8 unknown 0008 int8 Starfield control (for station windows) 0009 int8 Animation group 000A int8 Animation index (within group)
About the texture animations: Each "Animation group" represents a animated texture. The "animation index" is the frame number inside that animated texture. The texture with animation index = 0 is probably always the start frame of the animation. It also seems that textures with an index != are NEVER directly referenced in the tilemap and are used exclusively as frames of the animation and NOT as standelone textures. It also seems that animated textures are always played in reverse after reaching the max. frame number.
This file contains the model definitions for all 3D objects (not sprites). Each model lives in its own chunk, of type 0x0F, of which it is subchunk 0.
Coordinates are (apparently) stored as 24.8 fixed-point numbers.
The model header consists of 8 bytes, followed by the instructions on how to draw the object:
0006 int16 no. faces
Models appear to be based around drawing commands:
0000 end of sub-hull 0000 int16 command = 0x0000
0001 define face: 0000 int16 command = 0x0001 0002 int16 face length 0004 3fix normal vector 0010 3fix point on face 001C ... face drawing commands
0003 define multiple vertices 0000 int16 command = 0x0003 0002 int16 no. vertices 0006 3*fix first vertex 0012 ... more vertices
0004 draw flat-shaded polygon: 0000 int16 command = 0x0004 0002 int16 no. vertices 0004 n*int16 vertices (defined previously in file)
0005 set colour for flat shading 0000 int16 command = 0x0005 0002 int16 colour
0006 split plane??? this defines a plane and references 2 faces, but I don't know what it's actually for 0000 int16 command = 0x0006 0002 3fix normal vector 000E 3fix point on face 001A int16 left child offset (from start of this command) 001C int16 right child offset (from start of this command)
000A define vertex: 0000 int16 command = 0x000A 0002 int16 vertex no. to define 0004 int16 reference vertex 0006 fix offset from reference in X direction
000B define vertex: as 0x000A except offset is in Y direction 000C define vertex: as 0x000A except offset is in Z direction
000D define vertex: as 0x000A except 2 offsets X, Y 000E X, Z 000F Y, Z
0015 ??? define initial vertex 0000 int16 command = 0x0015 0004 3*fix vertex coords
001C define colour and shade 0000 int16 command = 0x001C 0002 int16 colour 0004 int16 shade
0025 define texture mapping: 0000 int16 command = 0x0025 0002 int16 no. vertices 0004 int16 vertex no. of first vertex fix texture u coord (fix16.16) fix texture v coord (fix16.16) 000E int16 vertex no. of second vertex ...
0026 plot texture-mapped face: 0000 int16 command = 0x0026 0002 int16 texture no. (stored in citmat.res 475-525) 0004 int16 no. vertices 0006 n*int16 vertex numbers
Video mails. This file has 24 chunks:
- the first 12 (id 0A40 to 0A4B) contain the frames in subchunks
- the second 12 (id 0A4C to 0A57) are video information chunks (type 04) stored in one subchunk each
This I have yet found out about the videos:
- a framerate of about 10 per second
- some videos are split up into several parts (chunks)
- the TriOp init jingle is id 0A4A (the first part always played)
- the frames (bitmaps) contain huge areas of value 00 which means the previous pixel at this position has to be preserved. (ff1: But this is not always true as I found out...)
- the video info structure should store the information about keyframes (if there are such)
The video information structure (type 04) I yet don't know what everything means; it's of variable size since it contains a sub-table:
0000 int16 width of video (always 00C8) 0002 int16 height of video (always 0064) 0004 int16 corresponding chunk id of frames 0006 6xint8 ?? always 00 000C int16 ?? (TriOp jingle: 0001 all other: 0000) 000E nx5xint8 sub-table of n entries mmmm int16 'end tag' always 010C
The video info sub-table This table seems to determine how frames should be rendered. The from_ and to_ fields are inclusive; the first entry has from_frame = 0 and the last has to_frame = last frame
0000 int8 'video command' (my name for it, always 04) 0001 int8 from_frame 0002 int8 to_frame 0003 int8 ?? render operation? () 0004 int8 ?? flags? (contains 0x00 to 0x04) ()
*) those last two bytes could be the frame time as Jim suggested - but if that is true, where is the information how frames are drawn?