Import repo #1
Merged
Import repo #1
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Lays out the design and sets up some of the golang / CI foundation. Use the `slog` for logging, it can do structured logging later if we want to feed these logs to grafana. This lib is scheduled to become part of the standard lib in future versions of golang. Sticking with the stdlib `flag` instead of the popular `cobra` lib for CLI parsing, I think the needs here are simple enough to get away with it.
Allow reading from a file or a URL, to shorten the dev feedback cycle. Adds more github/makefile/docker structure to handle automated tests and keep the local install requirements low. `go test` doesn't have any native junit support, so add a helper script and bring in [go-junit-report](https://github.com/jstemmer/go-junit-report). Adds the [testify](https://github.com/stretchr/testify) assertion lib, which I find worthwhile despite the [official recommendation](https://github.com/golang/go/wiki/TestComments#assert-libraries).
* AMPEX-71: parse XML into structs - uses the built-in XML unmarshalling to decode into structs, with a few custom `UnmarshalXML` methods to handle in-order processing. - silly workaround for some badly formatted XML in some of the files - beginning of markdown conversion, there's a lot more variety in the source data - add table-driven tests to cover different XML snippets without much fuss, will need to build out a bigger test suite
There's too much variety in these to encode the semantics in a sensible way. This basically ports the [xslt](http://hg.nginx.org/nginx.org/file/tip/xslt/content.xslt) to generate markdown instead of HTML. Used some scratch code in main.go to print out all the `<syntax>` variations in use, and copied from there to determine the test cases. Reorganized the code a little to account for similarities between `<para>` and `<syntax>` and try to group similar structs in the same file.
* add `golangci-lint` - `make lint` and vscode settings for local dev - fancy github action to get inline feedback on PRs, jumping through hoops to be sure it always uses the version from the `Makefile` - another vscode "feature" to install the binaries, which has to copy the version numbers. Might upgrade that to use a `devtools` container like other NGINX SaaS projects do to keep the versions in one place. * fix potential connection leak
Resolves problems with golangci-lint on arm64 platforms, and gets us a little more inline with other projects. Lots of TODOs, but this gets vscode devcontainer + golang + arm64 happy again.
created atom.go to parse the xml and pull the first link tag from the first entry tag created atom_test.go to test functionality of functions in atom.go created test.xml for testing purposes added commented code template to main.go for future use * removed commented lines of code * minor changes and test changes added necessary comments for code understanding changed previous tests to table-driven tests for redundancy * changed io/ioutil to io packaged * minor changes to table-driven tests changed struct to map removed the need to call test within another test changed slog in main.go to follow structure of other log messages * checking return value of w.Write for linter
* AMPEX-72: support <link> elements Converts `<link>`s to markdown. This took a lot of restructuring and a hack. The main problem is that `<link>` is context sensitive: - `<link doc="../page.xml">title</link>` links somewhere else based on what module file is currently being parsed - `<link doc="page.xml">` uses the `name` attr from page.xml as the link title, again relative to the path of the module currently being parsed - `<link id="anchor"/>` links to another directive on the current module So to calculate the href and title of a link, we need to know: 1. the paths, links, and names of all pages we might link to: `<module>`s or `<article>`s 2. the link and path of the module we're currently parsing We still want the markdown conversion to happen as part of `xml.Unmarshal`ing because that's when we have access to the order of XML elements, but the [`xml.Unmarshaler`](https://pkg.go.dev/encoding/xml#Unmarshaler) has no way to provide extra context like "other pages we might link to" or "the current module being parsed" The hack is to use a package variable (protected by a mutex) to give `xml.Unmarshaler`s more context. Using the mutexes to serialize concurrent usage seemed easier than manually tweaking the tests to selectively remove `t.Parallel`. Considered adding a post-processing step on the markdown using [`text/Template`](https://pkg.go.dev/text/template), but that seemed way more complicated than the package vars. * optimize what files we look at
These all map to code blocks. Listed out the rest of the elements we know we need so the log warning output could get less noisy.
Lists have `type` that controls rendering, respect that and generate matching markdown. `<list type="tag">` has no direct markdown equivalent, so we approximate it using unordered lists and indentation. Needed some fiddly accounting to handle nested lists, nested `<para>`, and irrelevant XML whitespace.
* WIP: sketch up JSON writing * write the json formatted version with tests impelemented write function to write json to io implemented getVersion to return the reference object version finished implementation for toModule directive wrote test case for New, Write, and GetVersion function in output_test.go * Update tools/reference-converter/main.go Co-authored-by: Ryan Davis <ry.davis@f5.com> --------- Co-authored-by: Ryan Davis <ry.davis@f5.com>
- support a handful of straighforward elements: `<http-status>`, `<header>`, `<commercial_version>`, and `<emphasis>`. Only `<note>` remains as unsupported - refactor tests to de-dupe and consistently use the main entry point of the `parse` module - this gets a little better test coverage and easier to handle as more elements want to access `parse.current` - pulled test helpers out into their own file as they got a little more complex - some misc renames for consistency - begin whitespace fights - the source XML has some specific patterns for how it uses indentation and newlines, get our test cases and markdown conversion handling those better. This is mostly a game of whack-a-mole with `strings.Trim`
* use the devtools image for reference-converter - follow the `.run` patten from other repos, so make targets run from within the devcontainer don't use docker, but outside does - uses a different image from devcontainer to add a bunch of vscode specific stuff not needed at CI time - runs as the `node` user to resolve a bunch of permissions issues - use the devtools image in the pipeline * fix CI permissions issues CI is running as `runner` when it clones the repo, so when we volume mount there's not enough access for the node user. * more permissions / arch shenanigans * drop debugging noise
- make the test helpers create more realistic XML - blend the `<syntax>` tests in with the one big one - add some more `<syntax>` varieties with regards to whitespace. Gave up and used regex to fix some whitespace after the XML decoding instead of trying to work in into the tokenizing loop
* created reference.json fixed bug associated with os.open function removed unicode warnings with other languages * fix for build issue with lang attr * Delete Nginx.css * Delete Nginx.test.tsx * Delete Nginx.tsx * minor issues with building fixed * removing empty file
* Fix markdown format * Fix format
* Add support for multiple syntax in directives * Fix comments
* Fix list indentation in markdown * Remove unused code * Fix comments
* Reference converter daily workflow * fixes * use output param * Fix comments
* Add is_block to json * Fix comments
There's lots of stuff in this release, mostly upgrading to use slog.
* Add component to display tooltip for inline directives * Fix build * Remove unused field in prop * Fix comments
* Create reference lib package * directive index * Fix tsconfig * Fix comments * Add comments * Split into two packages * Update readme
* Worflow to automate npm publish * Add working dir * Fix comments
* Bundle reflib using rollup * Update workflow * Fix comments * fix workflow * Remove dist
The data files had weird encoding because `json.Marshal` escapes "problematic HTML characters" inside JSON strings. Disables that default so we get regular looking content.
The new repo will have a different layout, gets paths and magic strings happier.
Importing history from another repo
arpith-f5
reviewed
Aug 29, 2023
ryepup
commented
Aug 30, 2023
zsteinkamp
approved these changes
Aug 30, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Proposed changes
Importing relevant code from a private repo.
Checklist
Before creating a PR, run through this checklist and mark each as complete.
CONTRIBUTINGdocumentREADME.mdandCHANGELOG.md)