Merge pull request #8 from quant-aq/dmcc/getting-started

Revamp docs, add Quick Start doc

README.md

@@ -18,54 +18,32 @@ by providing both new infrastructure (a more comprehensive versioning scheme
 including both system runtimes and external datasets) and a corresponding set of
 best practices to ensure experiments are maximally trackable.
 
-In its current form, Aeromancy requires a fairly specific software stack:
+In its current form, Aeromancy requires a fairly specific software stack: (hey,
+we said it was opinionated)
 
 - **Experiment tracker**: [Weights and Biases](https://wandb.ai)
 - **Object storage** (artifacts): S3-compatible, e.g.,
   [Ceph](https://github.com/ceph/ceph)
 - **Virtualization**: [Docker](https://www.docker.com/)
+- **Python Package Manager**: [pdm](https://pdm.fming.dev)
+- **Revision Control**: [Git](https://git-scm.com/)
 
 **Note:** As is likely obvious, Aeromancy documentation is in a very early
 state. As this is a pre-release support may be limited. For now, we include a
 couple pointers for how to setup your environment for Aeromancy.
 
-## Getting started
+## Documentation overview
 
-**Coming soon**: A proper Getting Started section.
+- If you're new to Aeromancy, [start here](docs/docs/quick_start.md)!
+- In the Developer Reference section of the documentation, we include some
+  design docs which provide an [architectural overview](docs/docs/scaffolding.md) and a
+  [glossary](docs/docs/tasks.md) of terms.
+- To see autogenerated docs for code from this repo, you'll need to start a
+  local doc server (`pdm doc`).
 
-To quickly set up an Aeromancy project, we've created a
-[Copier](https://copier.readthedocs.io/en/stable/) template. See instructions at
-the
-[quant-aq/aeromancy-project-template](https://github.com/quant-aq/aeromancy-project-template?tab=readme-ov-file#quick-start).
-
-## Requirements
-
-- Python 3.10.5 or higher
-- [`pdm`](https://pdm.fming.dev): Install via `pip install --user pdm` then
-  install Aeromancy packages with `pdm install`.
-- **Environment variables**:
-  - S3 backend location and credentials:
-    - `AEROMANCY_AWS_ACCESS_KEY_ID`
-    - `AEROMANCY_AWS_SECRET_ACCESS_KEY`
-    - `AEROMANCY_AWS_S3_ENDPOINT_URL`
-    - `AEROMANCY_AWS_REGION`
-  - `WANDB_API_KEY` (from [Weights and Biases](https://wandb.ai))
-- **SSH Authentication**: You'll want `ssh-agent` setup if you need to access
-  private GitHub repositories. Check out these
-  [instructions](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent).
-
-### Mac OS
-
-- Use [Homebrew](https://brew.sh/) to install the following:
-  - `brew install apache-arrow@13.0.0_5 bat@0.23.0 graphviz@8.1.0
-    openblas@0.3.24 pre-commit@3.3.3`
-- Install Docker Desktop from [docker.com](https://www.docker.com/) (not Brew
-  since it has a trickier upgrade story)
-
-## Common commands
+## Common development commands
 
 - `pdm lint`: Run pre-commit linters
 - `pdm test`: Run test suite
 - `pdm doc`: Start doc server (see also the [public
-  version](https://quant-aq.github.io/aeromancy/) for the latest checked in
-  version)
+  version](https://quant-aq.github.io/aeromancy/) for the latest release)

docs/docs/customizing.md

@@ -0,0 +1,46 @@
+
+# Customizing Aeromancy projects
+
+To quickly set up an Aeromancy project, we've created a
+[Copier](https://copier.readthedocs.io/en/stable/) template. See instructions at
+the
+[quant-aq/aeromancy-project-template](https://github.com/quant-aq/aeromancy-project-template?tab=readme-ov-file#quick-start).
+
+In the generated Python project setup (`pyproject.toml`), you may also want to
+adjust:
+
+- **Extra Python packages:** Add them with `pdm add <pkgname>`. See [PDM
+  docs](https://pdm.fming.dev/latest/usage/dependency/) for more information on
+  this.
+- **`pdm` [scripts](https://pdm.fming.dev/latest/usage/scripts/)**: Some of
+  these are necessary for running Aeromancy (like `pdm go`), but you can add
+  more if there are common tasks for your project.
+- **Extra `docker run` arguments**: (E.g., mounting
+  [volumes](https://docs.docker.com/engine/reference/commandline/run/#mount)).
+  These can be baked `pdm go` script with `--extra-docker-run-args='...'`. The
+  [template](https://github.com/quant-aq/aeromancy-project-template) includes a
+  standard volume mapping (`data/`) for ingesting datasets.
+- **Extra Debian packages:** (outside of those included by Aeromancy), you may
+  want to bake them into the `pdm go` script with `--extra-debian-package='...'`
+  (specify the flag once per package name).
+
+## Filesystem layout
+
+Ultimately, the structure of an Aeromancy project should look something like
+this:
+
+```text
+<projectroot>/
+  pyproject.toml
+  pdm.lock
+  main.py  # AeroMain
+  src/
+    <projectname>/
+      <youractions>.py
+      <youractionbuilder>.py
+```
+
+The structure of the classes containing your
+[`Action`][aeromancy.action.Action](s) and
+[`ActionBuilder`][aeromancy.action_builder.ActionBuilder] is flexible -- they
+just need to be importable in AeroMain.

docs/docs/index.md

@@ -1 +1,41 @@
---8<-- "README.md"
+# Aeromancy
+
+[![Tests](https://github.com/quant-aq/aeromancy/actions/workflows/ci.yml/badge.svg)](https://github.com/quant-aq/aeromancy/actions/workflows/ci.yml)
+[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
+[![pdm-managed](https://img.shields.io/badge/pdm-managed-blueviolet)](https://pdm.fming.dev)
+[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
+[![pre-commit enabled](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white)](https://pre-commit.com/)
+![Apache 2.0 licensed](https://img.shields.io/github/license/quant-aq/aeromancy)
+
+**Aeromancy** is an opinionated philosophy and open-sourced framework that
+closely tracks experimental runtime environments for more reproducible machine
+learning. In existing experiment trackers, it’s easy to miss important details
+about how an experiment was run, e.g., which version of a dataset was used as
+input or the exact versions of library dependencies. Missing these details can
+make replicability more difficult. Aeromancy aims to make this process smoother
+by providing both new infrastructure (a more comprehensive versioning scheme
+including both system runtimes and external datasets) and a corresponding set of
+best practices to ensure experiments are maximally trackable.
+
+In its current form, Aeromancy requires a fairly specific software stack: (hey,
+we said it was opinionated)
+
+- **Experiment tracker**: [Weights and Biases](https://wandb.ai)
+- **Object storage** (artifacts): S3-compatible, e.g.,
+  [Ceph](https://github.com/ceph/ceph)
+- **Virtualization**: [Docker](https://www.docker.com/)
+- **Python Package Manager**: [pdm](https://pdm.fming.dev)
+- **Revision Control**: [Git](https://git-scm.com/)
+
+!!! note
+    Aeromancy documentation is still in a very early state. As this is a
+    pre-release, support may be limited.
+
+## Documentation overview
+
+- If you're new to Aeromancy, [start here](quick_start.md)!
+- In the Developer Reference section of the documentation, we include some
+  design docs which provide an [architectural overview](scaffolding.md) and a
+  [glossary](tasks.md) of terms.
+- Lastly, we have autogenerated documentation in [Code
+  Reference](reference/aeromancy/index.md).