Calligraphy.md

Calligraphy

Calligraphy is a custom game data management system developed by Gazillion. Its main purpose was most likely providing game designers convenient tools for editing game data. Data exported from Calligraphy is stored in the Calligraphy.sip archive.

Calligraphy uses five file formats: directory, curve, asset type, blueprint, and prototype. All Calligraphy files start with the same four-byte header:

struct CalligraphyHeader
{
    char[3] Magic;
    byte Version;
}

The magic string defines what format is used in the file. The version depends on the game version: game versions 1.9-1.17 used Calligraphy version 10, and all later game versions starting with 1.18 released on January 24th 2014 use Calligraphy version 11.

All strings in Calligraphy files are fixed-length UTF-8 strings with the length encoded in a 16-bit value preceding the text:

struct FixedString16
{
    ushort StringLength;
    byte[StringLength] String; // UTF-8
}

Directory

Directory (.directory) files contain information required for the initialization of the DataDirectory class. There's a total of five directory files, each containing a number of records with slightly different structures.

All directories start with a Calligraphy header and the number of records contained in the directory.

struct DirectoryFile
{
    CalligraphyHeader Header;
    uint NumRecords;
    Record[NumRecords] Records;
}

Curve.directory (signature CDR), Type.directory (signature CDR), and Blueprint.directory (signature BDR) have the same standard record structure:

struct GenericRecord
{
    ulong Id;
    ulong Guid;
    GenericRecordFlags Flags;
    FixedString16 FilePath;
}

[Flags]
enum GenericRecordFlags : byte
{
    None        = 0,
    Protected   = 1 << 0
}

Note: although curve records do contain a field for flags, it's always 0 (none).

Prototype.directory (signature PDR) has a modified structure:

struct PrototypeRecord
{
    ulong PrototypeId;
    ulong PrototypeGuid;
    ulong BlueprintId;    // Also refers to the default prototype
    PrototypeRecordFlags Flags;
    FixedString16 FilePath;
}

[Flags]
enum PrototypeRecordFlags : byte
{
    None        = 0,
    Abstract    = 1 << 0,
    Protected   = 1 << 1
}

Replacement.directory (signature RDR) is a special directory used for handling deprecated GUIDs. Replacement records are managed by the ReplacementDirectory class. This file has a different record structure:

ulong OldGuid;
ulong NewGuid;
FixedString16 Name;    // Formatted Calligraphy path

Please note that file paths contained in these directory files use the \ symbol as the path delimiter, while pak files use /. To use these paths for reading files from the pak file system you need to replace \ with / while reading them.

Curve

Curve (.curve, signature CRV) files contain collections of 64-bit floating point values. They are used for various purposes as values for prototype fields. Loaded curves are managed by the CurveDirectory class.

Curve files have the following structure:

struct CurveFile
{
    CalligraphyHeader Header;
    int StartPosition;
    int EndPosition;
    double[EndPosition - StartPosition + 1] Values; 
}

Asset Type

Asset type (.type, signature TYP) files contain collections of asset references of specific types. They function as essentially enumerators for instances of various types of external data, and some of them are bound to enums in code. Loaded asset types are managed by the AssetDirectory class.

Asset type files have the following structure:

struct AssetTypeFile
{
    CalligraphyHeader Header;
    ushort NumAssets;
    AssetValue[NumAssets]; Assets;
}

Each asset in an asset type has the following structure:

struct AssetValue
{
    ulong AssetId;    // Processed by the client as the StringId for the name
    ulong AssetGuid;
    AssetValueFlags Flags;
    FixedString16 Name; 
}

[Flags]
enum AssetValueFlags : byte
{
    None        = 0,
    Protected   = 1 << 0
}

Blueprint

Blueprint (.blueprint, signature BPT) files contain definitions for Calligraphy prototype fields and field groups. Each blueprint is paired with a default prototype (.defaults) that contains default values for all fields defined in the blueprint. Default prototypes share ids with blueprints, so the same ulong value can refer to both a blueprint and it's corresponding default prototype.

Blueprint files have the following structure:

struct BlueprintFile
{
    CalligraphyHeader Header;
    FixedString16 RuntimeBinding;    // Name of the class that handles prototypes that use this blueprint
    ulong DefaultPrototypeId;        // Same as this blueprint's id

    ushort NumParents;
    BlueprintReference[NumParents] Parents;

    ushort NumContributingBlueprints;
    BlueprintReference[NumContributingBlueprints] ContributingBlueprints;

    ushort NumMembers;
    BlueprintMember[NumMembers] Members;
}

Blueprint references define what additional field groups a prototype has. These references have the following structure:

struct BlueprintReference
{
    ulong BlueprintId;
    byte NumOfCopies;
}

Blueprint members are definitions for individual prototype fields that have the following structure:

struct BlueprintMember
{
    ulong FieldId;        // Processed by the client as a StringId
    FixedString16 FieldName;
    CalligraphyBaseType BaseType;
    CalligraphyStructureType StructureType;

    if (BaseType == CalligraphyBaseType.Asset || BaseType == CalligraphyBaseType.Curve
    || BaseType == CalligraphyBaseType.Prototype || BaseType == CalligraphyBaseType.RHStruct)
        ulong Subtype;
}

BaseType defines the type of data stored in a field. Calligraphy supports nine base types:

enum CalligraphyBaseType : byte
{
    Asset = 0x41,       // A (Id reference to an asset)
    Boolean = 0x42,     // B (Stored as a UInt64)
    Curve = 0x43,       // C (Id reference to a curve)
    Double = 0x44,      // D (For all floating point values)
    Long = 0x4c,        // L (For all integer values)
    Prototype = 0x50,   // P (Id reference to another prototype)
    RHStruct = 0x52,    // R (Embedded prototype without an id)
    String = 0x53,      // S (Id reference to a localized string)
    Type = 0x54         // T (Id reference to an AssetType)
}

StructureType defines whether a field contains a single value or a list of multiple values:

enum CalligraphyStructureType : byte
{
    Simple = 0x53,      // Simple
    List = 0x4c         // List (only for assets, prototypes, rhstructs, and types)
}

Subtype specifies the id of the parent value that the value in this field has to inherit from. For example, for prototypes it is the id of the blueprint / default prototype. Only assets, curves, and prototypes have subtypes.

Prototype

Prototype (.prototype or .defaults, signature PTP) files contain values for fields defined in blueprints. Each .defaults prototype file is paired with a .blueprint file that defines its fields, and .prototype files inherit from these default prototypes. .prototype files have their own ids generated by hashing their paths, while .defaults files share ids with their paired blueprints.

Prototype files have the following structure:

struct PrototypeFile
{
    CalligraphyHeader Header;
    Prototype Prototype;
}

Prototypes themselves have the following structure:

struct Prototype
{
    PrototypeDataHeader Header;

    if (Header.DataExists)
    {
        ushort NumFieldGroups;
        PrototypeFieldGroup[NumFieldGroups] FieldGroups;
    } 
}

Prototype data header has the following structure:

struct PrototypeDataHeader
{
    PrototypeDataDesc Flags;
    bool ReferenceExists = Flags.HasFlag(PrototypeDataDesc.ReferenceExists);
    bool DataExists = Flags.HasFlag(PrototypeDataDesc.DataExists);
    bool PolymorphicData = Flags.HasFlag(PrototypeDataDesc.PolymorphicData);

    if (ReferenceExists)
        ulong ReferenceType;    // Parent prototype id, invalid (0) for .defaults 
}

[Flags]
enum PrototypeDataDesc : byte
{
    None            = 0,
    ReferenceExists = 1 << 0,
    DataExists      = 1 << 1,
    PolymorphicData = 1 << 2
}

Each field group is a collection of fields belonging to blueprints that contribute to a prototype. They have the following structure:

struct PrototypeFieldGroup
{
    ulong DeclaringBlueprintId;
    byte BlueprintCopyNumber;

    ushort NumSimpleFields;
    PrototypeSimpleField[NumSimpleFields] SimpleFields;

    ushort NumListFields;
    PrototypeListField[NumListFields] ListFields;
}

Simple fields contain a single value and have the following structure:

struct PrototypeSimpleField
{
    ulong FieldId;
    BaseType Type;    // See Blueprint section above for more info
    object Value; 
}

List fields contain a list of values and have a similar structure:

struct PrototypeListField
{
    ulong FieldId;
    BaseType Type;
    ushort NumValues;
    object[ValuesLength] Values;  
}

The actual value type depends on the base type in Type:

• Boolean: a boolean stored as a 64-bit unsigned integer.

• Double: a double precision 64-bit floating point value.

• Long: a 64-bit signed integer value.

• RHStruct: a new prototype definition starting with PrototypeDataHeader.

• Asset, Curve, Prototype, String, Type: a 64-bit data id.

RHStructs are fully-featured prototypes without an id that can have other RHStructs as their field values. Because of that, some prototypes have a heavily nested recursive structure.