Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1.0.1 #1

Merged
merged 11 commits into from
Sep 21, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions .github/workflows/md_lint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
name: Lint markdown files

# Build on every branch push, tag push, and pull request change:
on: [push, pull_request]

jobs:
checks:
runs-on: ubuntu-latest
name: Lint md files
steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Lint markdown files
uses: articulate/actions-markdownlint@v1.1.0
with:
config: .markdownlint.jsonc
3 changes: 2 additions & 1 deletion .github/workflows/mypy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@ jobs:
runs-on: ubuntu-latest
name: mypy
steps:
- uses: actions/checkout@v4
- name: Checkout repository
uses: actions/checkout@v4

- name: Set up Python 3.7
uses: actions/setup-python@v4
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@ jobs:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4
- name: Checkout repository
uses: actions/checkout@v4

- name: Install local ipl3checksum
run: pip install .
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/tests_other_repo.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ on: [push, pull_request_target]

jobs:
build_repo:
name: Build repo
name: Test other repo
runs-on: ubuntu-latest

strategy:
Expand Down
35 changes: 18 additions & 17 deletions .github/workflows/upload_pypi.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,29 +9,30 @@ jobs:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4
- name: Checkout repository
uses: actions/checkout@v4

- name: Install build module
run: pip install build
- name: Install build module
run: pip install build

- name: Build wheel and source
run: python -m build --sdist --wheel --outdir dist/ .
- name: Build wheel and source
run: python -m build --sdist --wheel --outdir dist/ .

- uses: actions/upload-artifact@v3
with:
path: dist/*
- uses: actions/upload-artifact@v3
with:
path: dist/*

upload_pypi:
needs: [build_wheel]
runs-on: ubuntu-latest
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/')
steps:
- uses: actions/download-artifact@v3
with:
name: artifact
path: dist

- uses: pypa/gh-action-pypi-publish@v1.8.10
with:
user: __token__
password: ${{ secrets.pypi_password }}
- uses: actions/download-artifact@v3
with:
name: artifact
path: dist

- uses: pypa/gh-action-pypi-publish@v1.8.10
with:
user: __token__
password: ${{ secrets.pypi_password }}
2 changes: 0 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -162,8 +162,6 @@ cython_debug/
#.idea/


#
.vscode/

asm/
*.z64
Expand Down
14 changes: 14 additions & 0 deletions .markdownlint.jsonc
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{
// https://github.com/DavidAnson/markdownlint/blob/main/doc/md024.md
// MD024 - Multiple headings with the same content
"MD024": {
"siblings_only": true
},

// https://github.com/DavidAnson/markdownlint/blob/main/doc/md013.md
// MD013 - Line length
"MD013": {
"code_block_line_length": 120,
"headings": false
}
}
7 changes: 7 additions & 0 deletions .vscode/extensions.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
// See https://go.microsoft.com/fwlink/?LinkId=827846
// for the documentation about the extensions.json format
"recommendations": [
"davidanson.vscode-markdownlint"
]
}
33 changes: 33 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

## [1.0.1] - 2023-09-21

### Added

- Allow invoking `ipl3checksum` as a CLI program.
- Currently it only allows the `-V`/`--version` argument, which prints the
version of the library.
- A `CHANGELOG.md`
- Cleanup the `README.md`
- Reorder sections.
- Add more notes about installing and the develop version.
- Reference the changelog.
- List features.
- Add a `py.typed` file.

## [1.0.0] - 2023-09-20

### Added

- Initial relase

[unreleased]: https://github.com/olivierlacan/keep-a-changelog/compare/1.1.0...HEAD
[1.0.1]: https://github.com/olivierlacan/keep-a-changelog/compare/1.0.0...1.1.1
[1.0.0]: https://github.com/Decompollaborate/ipl3checksum/releases/tag/1.0.0
112 changes: 89 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,7 @@ A Python library to calculate the IPL3 checksum for N64 ROMs.

## How to use it?

First you need to install the library, one way of doing it is via `pip`.

```bash
python3 -m pip install -U ipl3checksum
```

Now you can invoke the library from your script.
To calculate the checksum of a ROM:

```py
romBytes = # A big endian bytes-like object
Expand All @@ -36,32 +30,104 @@ cickind = ipl3checksum.detectCIC(romBytes)
print(cickind) # Either a `ipl3checksum.CICKind` or None if was not able to detect the CIC
```

## Features

- Supports all 6 retail CIC variants.
- Can calculate the checksum of a ROM using the algorithm of any of the
supported CIC variants.
- Can detect any of the supported CIC variants.

### Restrictions/requirements

- The library assumes the passed ROM contains a ROM header at offset range
`[0x0, 0x40]` and a correct IPL3 is at `[0x40, 0x1000]`
- Since the checksum algorithm is calculated on the first MiB after IPL3 (from
`0x1000` to `0x101000`), then the library expects the passed ROM to be at least
`0x101000` bytes long, otherwise the library will reject the ROM.
- If it is not the case, then pad your ROM with zeroes to that size.

## Installing

First you need to install the library, one way of doing it is via `pip`.

```bash
python3 -m pip install -U ipl3checksum
```

If you use a `requirements.txt` file in your repository, then you can add
this library with the following line:

```txt
ipl3checksum>=1.0.0,<2.0.0
``````

Now you can invoke the library from your script.

### Development version

The unstable development version is located at the [develop](https://github.com/Decompollaborate/ipl3checksum/tree/develop)
branch. PRs should be made into that branch instead of the main one.

The recommended way to install a locally cloned repo is by passing the `-e`
(editable) flag to `pip`.

```bash
python3 -m pip install -e .
```

In case you want to mess with the latest development version without wanting to
clone the repository, then you could use the following commands:

```bash
python3 -m pip uninstall ipl3checksum
python3 -m pip install git+https://github.com/Decompollaborate/ipl3checksum.git@develop
```

NOTE: Installing the development version is not recommended unless you know what
you are doing. Proceed at your own risk.

## Versioning and changelog

This library follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
We try to always keep backwards compatibility, so no breaking changes should
happen until a major release (i.e. jumping from 1.X.X to 2.0.0).

To see what changed on each release check either the [CHANGELOG.md](CHANGELOG.md)
file or check the [releases page on Github](https://github.com/Decompollaborate/ipl3checksum/releases).
You can also use [this link](https://github.com/Decompollaborate/ipl3checksum/releases/latest)
to check the latest release.

## Where does this come from?

This algorithm comes directly from the IPL3, which each variant is part of the first 0x1000 bytes of the rom of every retail N64 ROM.
This algorithm comes directly from the IPL3, which each variant is part of the
first 0x1000 bytes of the rom of every retail N64 ROM.

There are various implementations floating around on the internet, but for this specific one was reverse-engineered by myself.
I made this because I couldn't find a library to calculate this checksum, so I decided to reverse-engineer it myself instead of
taking somebody else's work. It also was an interesting learning experience.
There are various implementations floating around on the internet, but for this
specific one was reverse-engineered by myself. I made this because I couldn't
find a library to calculate this checksum, so I decided to reverse-engineer it
myself instead of taking somebody else's work. It also was an interesting
learning experience.

## Note about licensing

Most of the repository is licensed under the [MIT license](LICENSE), but I also made a
[reference implementation](docs/reference_implementation.md) that is part of the public domain (licensed under CC0-1.0), feel free to
use it however you prefer (acknowledgment is always appreciated, but not required).
Most of the repository is licensed under the [MIT license](LICENSE), but I also
made a [reference implementation](docs/reference_implementation.md) that is part
of the public domain (licensed under CC0-1.0), feel free to use it however you
prefer (acknowledgment is always appreciated, but not required).

## I want to learn more! What is an IPL3? What is CIC?

I'm not really the guy that can answer all your hunger for knowledge, but here are a few links that may be helpful:
I'm not really the guy that can answer all your hunger for knowledge, but here
are a few links that may be helpful:

* CIC-NUS: <https://n64brew.dev/wiki/CIC-NUS>
* Initial Program Load 3 (IPL3) <https://n64brew.dev/wiki/Initial_Program_Load#IPL3>
* List of retail games, containing which CIC they use: <https://docs.google.com/spreadsheets/d/1WgZ7DZSzWwYIxwg03yoN9NK_0okuSx9dVL2u5MWPQ60/edit#gid=1247952340>
* Research about the CIC 6105: <https://github.com/Dragorn421/n64checksum>
* Disassembly of all the retail IPL3 binaries: <https://github.com/decompals/N64-IPL/blob/main/src/ipl3.s>
- CIC-NUS: <https://n64brew.dev/wiki/CIC-NUS>
- Initial Program Load 3 (IPL3) <https://n64brew.dev/wiki/Initial_Program_Load#IPL3>
- List of retail games, containing which CIC they use: <https://docs.google.com/spreadsheets/d/1WgZ7DZSzWwYIxwg03yoN9NK_0okuSx9dVL2u5MWPQ60/edit#gid=1247952340>
- Research about the CIC 6105: <https://github.com/Dragorn421/n64checksum>
- Disassembly of all the retail IPL3 binaries: <https://github.com/decompals/N64-IPL/blob/main/src/ipl3.s>

## References

* "IPL3 checksum algorithm" section of the "PIF-NUS" article on n64brew.dev: <https://n64brew.dev/wiki/PIF-NUS#IPL3_checksum_algorithm>
* Used for getting the "8-bit IPL3" seed value.
* List of retail games, containing which CIC they use: <https://docs.google.com/spreadsheets/d/1WgZ7DZSzWwYIxwg03yoN9NK_0okuSx9dVL2u5MWPQ60/edit#gid=1247952340>
- "IPL3 checksum algorithm" section of the "PIF-NUS" article on n64brew.dev: <https://n64brew.dev/wiki/PIF-NUS#IPL3_checksum_algorithm>
- Used for getting the "8-bit IPL3" seed value.
- List of retail games, containing which CIC they use: <https://docs.google.com/spreadsheets/d/1WgZ7DZSzWwYIxwg03yoN9NK_0okuSx9dVL2u5MWPQ60/edit#gid=1247952340>
9 changes: 6 additions & 3 deletions docs/reference_implementation.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,11 @@

This whole file is licensed under CC0-1.0. See [docs/LICENSE](./LICENSE).

This file contains a reference implementation of the IPL3 checksum algorithm, adapted to work with every known retail CIC/IPL3 variant.
This file contains a reference implementation of the IPL3 checksum algorithm,
adapted to work with every known retail CIC/IPL3 variant.

This implementation is a very crude direct translation from the original assembly and it could be greatly simplified in a reimplementation.
This implementation is a very crude direct translation from the original
assembly and it could be greatly simplified in a reimplementation.

```c
/* SPDX-License-Identifier: CC0-1.0 */
Expand Down Expand Up @@ -158,7 +160,8 @@ void calculateChecksum(const uint8_t *rom, uint32_t cic, uint32_t *dst1, uint32_
switch (cic) {
case 6105:
case 7105:
/* ipl3 6105 copies 0x330 bytes from the ROM's offset 0x000554 (or offset 0x000514 into IPL3) to vram 0xA0000004 */
/* ipl3 6105 copies 0x330 bytes from the ROM's offset 0x000554 (or offset 0x000514 into IPL3) */
/* to vram 0xA0000004 */
/* lw $t7, 0x0($s6) */
t7 = readWord(rom, s6 - 0xA0000004 + 0x000554);

Expand Down
6 changes: 4 additions & 2 deletions notes/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Notes

This folder contains the scripts I made while I was trying to understand the checksum algorithm of every IPL3 variant.
This folder contains the scripts I made while I was trying to understand the
checksum algorithm of every IPL3 variant.

All of the implementations here are direct and crude translations from the original asm.
All of the implementations here are direct and crude translations from the
original asm.
8 changes: 7 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

[project]
name = "ipl3checksum"
version = "1.0.0"
version = "1.0.1"
description = "Library to calculate the IPL3 checksum for N64 ROMs"
readme = "README.md"
requires-python = ">=3.7"
Expand All @@ -17,5 +17,11 @@ dynamic = ["dependencies"]
requires = ["hatchling", "hatch-requirements-txt"]
build-backend = "hatchling.build"

[project.scripts]
ipl3checksum = "ipl3checksum.frontends.climain:ipl3checksumMain"

[tool.cibuildwheel]
skip = ["cp36-*"]

[tool.setuptools.package-data]
ipl3checksum = ["py.typed"]
2 changes: 1 addition & 1 deletion src/ipl3checksum/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

from __future__ import annotations

__version_info__: tuple[int, int, int] = (1, 0, 0)
__version_info__: tuple[int, int, int] = (1, 0, 1)
__version__ = ".".join(map(str, __version_info__))
__author__ = "Decompollaborate"

Expand Down
14 changes: 14 additions & 0 deletions src/ipl3checksum/__main__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
#!/usr/bin/env python3

# SPDX-FileCopyrightText: © 2023 Decompollaborate
# SPDX-License-Identifier: MIT

from __future__ import annotations

import argparse

from .frontends import climain


if __name__ == "__main__":
climain.ipl3checksumMain()
9 changes: 9 additions & 0 deletions src/ipl3checksum/frontends/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
#!/usr/bin/env python3

# SPDX-FileCopyrightText: © 2023 Decompollaborate
# SPDX-License-Identifier: MIT

from __future__ import annotations


from . import climain as climain
Loading