Port from Gitbook to sphinx

[kousu]

Fixes #20 (the way I want it to be fixed anyway; I'm open to suggestions and alternatives)

[kousu]

This is posted at https://wiki-demo.neuropoly.org and ready for some review!

Live at https://neuro.polymtl.ca

[joshuacwnewton]

What would the workflow be for making changes? Something like...

1. Browse the website and notice a change you would like to make.
2. Click on "Edit on GitHub" link.
3. Click on the pencil "edit" icon.
4. Make changes in GitHub's markdown editor.
   • How would someone visualize their pending changes before merging them?
5. Click "Create a new branch for this commit and start a pull request."
6. The PR workflow will automatically build the changes.
   • I see the workflow publishes the changes when master is updated. Can there be a workflow to render in-progress, pending changes without actually publishing them, as is done for PRs on RTD-based websites?
7. The PR is reviewed and approved.
   • Who will review and approve the changes?
   • How will new students know who to request a review from?
8. The changes are now live on the website.

The developer in me likes this workflow! But, the "new lab student" / "lab advisor who's rushed for time" in me thinks that this might be too many hoops to jump through in order to make a change.

[kousu]

@joshuacwnewton wrote:

What would the workflow be for making changes? Something like...

1. Browse the website and notice a change you would like to make.

2. Click on "Edit on GitHub" link.

3. Click on the pencil "edit" icon.

The edit link drops you directly into the editor ^_^

4. Make changes in GitHub's markdown editor.
   
   * How would someone visualize their pending changes before merging them?

With 'show diff'; I don't know if we can have it turned on by default? Maybe it's an account setting we can advise people to use?

The markdown diffs are pretty good!

5. Click "Create a new branch for this commit and start a pull request."

No, just save directly to master and be done with it:

I really think we should discourage PRs for the wiki. Not everything needs to be code reviewed, not really.

@joshuacwnewton wrote:

Can there be a workflow to render in-progress, pending changes without actually publishing them, as is done for PRs on RTD-based websites?

No. Github Pages is too simple to handle branches.

However someone could fork the repo, and so long as they're working on master their fork should automatically go live at https://username.github.io/neuropoly-docs.

I am also not against hosting on readthedocs especially if we do want PR previews. I have no horse in the RTD / Github race, they're both just companies to me; I demo'd this way because it meant one less credential to manage; and also PRs should be the exception not the rule.

---

I realize the Github preview won't render identically to sphinx. But I'm betting here that these situations (weird syntax, custom embeds, etc) won't come up that often so we can live with just re-edit until it works. Basically I'm repeating what Julien said

I think that clicking on the "Preview" button on github should provide >95% of use cases like fixing a typo (5% scenario not covered when embedding does not show up or other non-MD objects that require sphinx compilation).

Changes could also be previewed by building the docs locally if we really need to (say, the large refactor case). It's really not that inaccessible:

git clone -b your-branch https://github.com/neuropoly/neuropoly-docs/
cd neuropoly-docs
pip install .[sphinx]
make html
chrome _build/html/index.html

Waiting on the CI feedback cycle is a drag -- CI is best for final checks and publishing, not developing. We shouldn't be encouraging it for daily feedback.

@jcohenadad wrote:

(ie: no landing page for the parent link of the TOC, and a page only appears if user clicks on the child pages

Oh, maybe! This sounds like something sphinx can do. There's a lot of options for the TOC. https://www.sphinx-doc.org/en/master/usage/restructuredtext/roles.html#cross-referencing-syntax says "If you prefix the content with !, no reference/hyperlink will be created." so, worth experimenting.

@jcohenadad wrote:

We could start with slightly smaller fonts,

lol I intentionally increased the size c39bb0e. Gitbook has larger TOC fonts so I thought I'd copy that. I thought one of the big reasons we liked Gitbook was that the TOC was extremely prominent and easy to navigate.

We can undo it!

@jcohenadad wrote:

And there is only one page to maintain (ie: it gets duplicated during sphinx compilation, right?)

Not during compilation, no. But I guess I could make that happen.

I want sphinx to recognize README.md as index.html. Surely there must be a plugin that can do that.

@jcohenadad wrote:

The emojis are rendered using your system fonts, which is good for filesize, but maybe bad for compatibility. E.g. on my linux system
Interesting, i didn't know that. Is there a reasonable workaround?

Just ignore it. Most systems handle emojis fine these days. Certainly anyone with a Mac or Windows and Android or iOS computer will.

@jcohenadad wrote:

To encourage students to create their own page, we need to provide an "easy" workflow (eg template, not-heavy explanation on how to create the file and where to locate it)

Oh interesting. Github has templates for issues and PRs but not for files. I guess we can just put team/_TEMPLATE.md and tell people to copy from it.

@jcohenadad wrote:

Why not going with the very active Jupyter Book project. There is also the MyST feature that makes our MD compatible and extendable to including many cool objects.

This already uses MyST. That's how it works in .md at all instead of .rst. I tried using myst_nb at first but it was very much heavier than myst_parser. If we did that, it has to boot a Jupyter kernel for each change, on top of installing all of Jupyter/numpy/matplotlib, and then run all your notebooks. If we did that we wouldn't get 30s turnaround time, it would balloon and only get worse with time.

I think myst_nb is great for code docs, but it's not appropriate for our primary wiki.

About merging the internal lab and external lab website: i'm still not 100% sure if we should do it. What would be the pros/cons, etc. I see how it could be confusing for external people visiting our lab website (ie: many entries in the TOC, overwhelming, etc.)

We can limit the TOC depth with :maxdepth 3: so that it's not overwhelming, and I think, probably, we can prevent the main TOC from diving into the lab manual at all (if we can adjust :maxdepth: for a specific link). And then when in the lab manual section, adjust the sidebar so that root TOC starts at the lab manual, hiding the other sections.

If we can't get the TOC to hang together just right then I won't advocate it. But if we can I hope we'll consider it.

[joshuacwnewton]

I really think we should discourage PRs for the wiki. Not everything needs to be code reviewed, not really.

I'm happy to hear this. I wrongly assumed that you wanted a PR-based workflow (oops), but if we're committing directly to master, then that addresses a lot of my concerns. I'm on board with pretty much everything you've written. 😄

[dpapp86]

As someone unfamiliar with Markdown, after 5 minutes with Nick, I have to vote in support of the transition.

[naga-karthik]

First off, thanks @kousu for your effort in porting our webpage! Here are some of my comments:

1. The font seems to be too big for me. Especially the drop-down sections on the left of the page. Also, in general, I think the font size can be reduced.
2. There is no YouTube button here: https://wiki-demo.neuropoly.org/events-and-workshops.html (Follow us on ...?)
3. (Might not be related to the website) I noticed that in the Research Section, there is no subsection for Deep Learning, which I think must definitely be there.
4. When you click on sections having drop-down menus (e.g. Job Opportunities), one would expect to be shown the subsections in the main page also. Currently, what happens is, I click Job Opportunities, it loads a new but empty page and I have to look at the subsections on the left to access the page I am interested in. This is slightly psychological because "loading of a new page" gives me an impression that I am about to be "shown" something.
5. I think this has been mentioned before, some of the icons on the Teams page do not render.

[gaspardcereza]

Hey @kousu, both Gitbook and Sphinx look pretty similar to me. As for editing the pages, I don't have any strong preference between doing it from the Gitbook interface or in GitHub, so if you think it's easier for you to manage the Sphinx version, I think we should go with it.

[kousu]

Emailed dnsmaster@polymtl.ca last night per instructions at https://docs.github.com/en/pages/configuring-a-custom-domain-for-your-github-pages-site/managing-a-custom-domain-for-your-github-pages-site#configuring-an-apex-domain to switch our DNS to say

comms3$ dig neuro.polymtl.ca @ns1.polymtl.ca          

; <<>> dig 9.10.8-P1 <<>> neuro.polymtl.ca @ns1.polymtl.ca
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 49419
;; flags: qr aa rd; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;neuro.polymtl.ca.		IN	A

;; ANSWER SECTION:
neuro.polymtl.ca.	86400	IN	A	185.199.108.153
neuro.polymtl.ca.	86400	IN	A	185.199.109.153
neuro.polymtl.ca.	86400	IN	A	185.199.110.153
neuro.polymtl.ca.	86400	IN	A	185.199.111.153

I had to tell Github about the change: 7b44c3f, then I had to go back to https://github.com/neuropoly/neuropoly-docs/settings/pages just now and wait for Github to be satisfied, and then reenable Force HTTPS:

One weird thing: the cache timeout seems very long. Github only set an hour (3600s) timeout on their DNS records, but ours is 24 dayshours (86400s). This means that, in the rare case Github changes addresses, our site will be inaccessible for up to 3 weeks 1 day to different parts of the planet.

To work around this, I've put us to https://www.neuro.polymtl.ca. https://neuro.polymtl.ca is still, for some people, pointed at an internal webserver on campus, but it's got a redirect to www.neuro.polymtl.ca set up, so everything should just work for everyone. I'll undo this in a day or so so we can have the shorter URL again.

[joshuacwnewton]

All of my requested changes were addressed a long while ago, so approving. 🙂

[kousu]

I turned down everyone's permissions on https://app.gitbook.io, and de-synced their copy:

Once this is merged I'll delete the Gitbook project/group too.

[kousu]

I went to namecheap and wiped all the DNS records I used for the demo address: