: Same as -a but only used in coverage mode; this is to stop your coverage runs from overwriting your normal runs.
--S: Provide a shell when done (useful inside a container (i.e. docker))
+-S: Provide a shell when done (useful inside a container (i.e. docker/podman))
"
exit 1
}
@@ -44,11 +62,13 @@ shell="no"
optlist=""
run_chapters=""
coverage=""
-builddir="build"
+update_jbovlaste=""
+builddir="build"
target="all"
diff='maybe'
+vimdiff=""
-while getopts "ndcC:a:A:sSthT:" opt
+while getopts "jndcC:a:A:sSthVT:" opt
do
case "$opt" in
c) coverage="true"
@@ -66,6 +86,8 @@ do
n) diff='no';;
d) diff='yes';;
S) shell='yes';;
+ V) vimdiff='-V';;
+ j) update_jbovlaste='yes';;
[?h]) usage;;
esac
done
@@ -174,19 +196,28 @@ then
rm -f $builddir/cll.xml
fi
+# If regeneration of xml/jbovlaste.xml was requestiod, remove
+# that file and it will get regenerated downstream.
+if [ "$update_jbovlaste" ]
+then
+ rm xml/jbovlaste.xml
+fi
+
make -f scripts/Makefile builddir="$builddir" test="$optlist" chapters="$chapters" covcopydir="$covarchivedir" copydir="$archivedir" $target
if [ $diff = 'yes' ]
then
- ./scripts/diff_official -c
+ ./scripts/diff_official -c "$vimdiff"
fi
if [ $diff = 'maybe' ]
then
+ echo
+ echo
echo "Would you like to diff the new output against the last official/ build? Ctrl-C or 'no' for no, anything else for yes."
read answer
if [ "$answer" != 'no' ]
then
- ./scripts/diff_official -c
+ ./scripts/diff_official -c "$vimdiff"
fi
fi
diff --git a/docker_init.sh b/container_init.sh
similarity index 100%
rename from docker_init.sh
rename to container_init.sh
diff --git a/official/README.md b/official/README.md
index cb6ec5a47..89550d5c1 100644
--- a/official/README.md
+++ b/official/README.md
@@ -30,7 +30,7 @@ confirm that nothing has changed except what you want to change.
(Note that when the build asks "Would you like to diff the new
output against the last official/ build?", it is running
scripts/diff_official; so if you've already done that, you've done
-this step.
+this step.)
scripts/diff_official makes copies of both the relevant build/ dir
and the relevant official/ dir, and massages them to make the output
@@ -47,36 +47,14 @@ diff.
As an example, this mini-script finds anything that looks like a
navgiation header and removes any newlines inside it; I used this
when I changed all the nav headers so that each file had ~5 lines of
-changes (easy to review) vs. ~30 lines (not so much).
+changes (easy to review) vs. ~3000 lines (not so much).
ruby -e 'puts ARGF.read.encode("UTF-8", "binary", invalid: :replace, undef: :replace, replace: "").gsub(%r{]*(navheader|navfooter|toc-link|back-to-info-link).*?
}m) { |x| x.gsub(%r{\s+}," ") }' "$@"
When you're satisfied that the changes you made are the changes you
-want, put the changes you want in official/ with something like
-this:
+want, run:
- $ cp -r build/xhtml_section_chunks official/cll_v1.1_xhtml-section-chunks_2016-05-25
- $ cd official
-
-Now you need to fix the links; you can get a list of all the links like so:
-
- $ find . -type l \! -name dtd
-
-Fixing looks like this:
-
- $ rm cll_v1.1_xhtml-section-chunks
- $ ln -s cll_v1.1_xhtml-section-chunks_2016-05-25 cll_v1.1_xhtml-section-chunks
-
-(Obviously, update that if we're not on version 1.1 anymore!)
-
-(Repeat for the other 2 chunk types.)
-
-The goal here is to make it so that every currently-relevant
-file/directory has a symlink to it, and that that symlink's name
-only changes when we change CLL versions.
-
-NOTE: The - and _ in the build/ dir do not match what's in the
-official/ dir. Sorry about that.
+ $ scripts/update_official -x -v 1.1 [assuming the version is still 1.1]
Updating EPUB & MOBI official/
------------------------------
@@ -87,10 +65,13 @@ less), if it passes all those tests it's probably fine.
However, it's now part of diff_official, too, since it's just a zip.
-When done, copy and update symlinks as with XHTML.
+The MOBI is generated directly from the EPUB via kindlegen, but you can
+check it if you want.
-Copy and update the MOBI as well; it's generated directly from the
-EPUB via kindlegen.
+When done, run:
+
+ $ scripts/update_official -e -v 1.1 [assuming the version is still 1.1]
+ $ scripts/update_official -m -v 1.1 [assuming the version is still 1.1]
Updating PDF official/
----------------------
@@ -98,7 +79,9 @@ Updating PDF official/
Get a PDF diff viewer for your OS and compare them visually. I'm
using https://github.com/vslavik/diff-pdf on Windows.
-When done, copy and update symlinks as with XHTML.
+When done, run:
+
+ $ scripts/update_official -p -v 1.1 [assuming the version is still 1.1]
Step 2: Updating github
=======================
@@ -107,6 +90,9 @@ The forever home of this data is the docbook-prince branch of https://github.com
Before you push, though, there are some things to do.
+Please keep your actual *changes* in a separate commit from your *official/*
+changes checkin, so people can see the real changes more easily.
+
Make *certain* that everything is done completely (all the work
above, the CHANGELOG, etc). Commit your changes (ideally in a very
few, very semantically distinct commits). Rebase from the
diff --git a/run_container.sh b/run_container.sh
new file mode 100755
index 000000000..5c767d259
--- /dev/null
+++ b/run_container.sh
@@ -0,0 +1,73 @@
+#!/bin/bash
+
+CONTAINER_BIN=${CONTAINER_BIN:-$(which podman 2>/dev/null)}
+CONTAINER_BIN=${CONTAINER_BIN:-$(which docker 2>/dev/null)}
+
+if $CONTAINER_BIN info >/dev/null 2>&1
+then
+ # Everything is working; no-op
+ CONTAINER_BIN="$CONTAINER_BIN"
+else
+ if sudo $CONTAINER_BIN info >/dev/null 2>&1
+ then
+ CONTAINER_BIN="sudo $CONTAINER_BIN"
+ else
+ echo "I can't get a working container system. You need to either have podman (preferred) or docker installed and running for this build system to work. I have tried both \"podman info\" and \"docker info\", with and without sudo."
+ exit 1
+ fi
+fi
+
+# Manually pull any -a or -A arguments so we can use them to build
+# volume-sharing options into our container run call. This assumes
+# arguments are well-formed; *shrug*
+args=( "$@" )
+extra_vols=()
+extra_dirs=()
+for index in ${!args[@]}
+do
+ arg=${args[$index]}
+ if [ "$arg" = '-a' -o "$arg" = '-A' ]
+ then
+ nextarg=${args[$index+1]}
+ nextargfile="/tmp/$(echo "${args[$index+1]}" | tr -c 'a-zA-Z0-9-' '_' | sed -e 's/^_*//' -e 's/_*$//' )"
+ extra_vols+=("-v" "$nextarg:$nextargfile")
+ extra_dirs+=("$nextarg")
+ args[$index+1]="$nextargfile"
+ fi
+done
+
+$CONTAINER_BIN kill cll_build >/dev/null 2>&1
+$CONTAINER_BIN rm cll_build >/dev/null 2>&1
+
+dir=$(readlink -f $(dirname $0))
+
+# If SELinux is on
+if getenforce | grep -q Enforcing
+then
+ # Make it accessible to both the user and the container
+ chcon -R -t container_home_t . "${extra_dirs[@]}"
+fi
+
+# FOR TESTING; forces complete container rebuild
+# $CONTAINER_BIN build --no-cache -t lojban/cll_build -f Dockerfile .
+# $CONTAINER_BIN rmi lojban/cll_build
+echo "Running container image build; this may take a while."
+echo
+$CONTAINER_BIN build -t lojban/cll_build -f Dockerfile . >/tmp/rc.$$ 2>&1 || {
+ echo "Container image build failed. Here's the output: "
+ echo
+ cat /tmp/rc.$$
+ echo
+ echo
+ echo "Container image build failed. Output is above."
+ echo
+ echo
+ rm -f /tmp/rc.$$
+ exit 1
+}
+
+rm -f /tmp/rc.$$
+
+$CONTAINER_BIN run --name cll_build --log-driver syslog --log-opt tag=cll_build \
+ -v $dir:/srv/cll "${extra_vols[@]}" -it lojban/cll_build \
+ /tmp/container_init.sh "$(id -u)" "$(id -g)" "${args[@]}"
diff --git a/run_docker.sh b/run_docker.sh
deleted file mode 100755
index b1883b205..000000000
--- a/run_docker.sh
+++ /dev/null
@@ -1,33 +0,0 @@
-#!/bin/bash
-
-echo -e "\n\nIf the first argument is '-v', then the following argument should be an extra directory to mount, for use with the ./cll_build -a or -A options, or 'none'."
-echo "Syntax is of docker's -v option, so to mount ~/www on /tmp/www, do: ~/www:/tmp/www"
-echo -e "\nAll other arguments are passed to ./cll_build\n\n"
-sleep 1
-
-extra_dir=""
-if [ "$1" = '-v' ]
-then
- shift
- extra_dir="-v $1"
- shift
-fi
-
-sudo docker kill cll_build
-sudo docker rm cll_build
-
-dir=$(readlink -f $(dirname $0))
-
-# Make it accessible to both the user and the container
-chcon -R -t container_home_t .
-
-# FOR TESTING; forces complete docker rebuild
-# sudo docker build --no-cache -t lojban/cll_build -f Dockerfile .
-# sudo docker rmi lojban/cll_build
-sudo docker build -t lojban/cll_build -f Dockerfile . || {
- echo "Docker build failed."
- exit 1
-}
-sudo /bin/docker run --name cll_build --log-driver syslog --log-opt tag=cll_build \
- -v $dir:/srv/cll $extra_dir -it lojban/cll_build \
- /tmp/docker_init.sh "$(id -u)" "$(id -g)" "$@"
diff --git a/scripts/Makefile b/scripts/Makefile
index 4456710d2..b1d3f7453 100644
--- a/scripts/Makefile
+++ b/scripts/Makefile
@@ -1,9 +1,5 @@
-test =
-# Grab environment variables; not sure why this is necessary.
-chapters = $(if $(CHAPTERS), $(CHAPTERS), chapters/01.xml chapters/02.xml chapters/03.xml chapters/04.xml chapters/05.xml chapters/06.xml chapters/07.xml chapters/08.xml chapters/09.xml chapters/10.xml chapters/11.xml chapters/12.xml chapters/13.xml chapters/14.xml chapters/15.xml chapters/16.xml chapters/17.xml chapters/18.xml chapters/19.xml chapters/20.xml chapters/21.xml)
-builddir = $(if $(BUILDDIR), $(BUILDDIR), $(PWD)/build)
-copydir = $(COPYDIR)
-covcopydir = $(COVCOPYDIR)
+chapters = $(if $(chapters), $(chapters), chapters/01.xml chapters/02.xml chapters/03.xml chapters/04.xml chapters/05.xml chapters/06.xml chapters/07.xml chapters/08.xml chapters/09.xml chapters/10.xml chapters/11.xml chapters/12.xml chapters/13.xml chapters/14.xml chapters/15.xml chapters/16.xml chapters/17.xml chapters/18.xml chapters/19.xml chapters/20.xml chapters/21.xml)
+builddir = $(if $(builddir), $(builddir), $(PWD)/build)
xsl_base = xml/docbook-xsl-1.78.1
SHELL=/bin/bash -o pipefail
@@ -25,9 +21,9 @@ realclean: clean
#*******
$(builddir)/cll.xml: $(chapters) xml/iso-pub.ent xml/identity.xsl xml/docbook2html_config_common.xsl scripts/update_jbovlaste_xml.sh scripts/generate_glossary.rb scripts/merge.sh
- rm -f $(builddir)/xml $(builddir)/dtd
- ln -nsf $(PWD)/xml $(builddir)/xml
- ln -nsf $(PWD)/dtd $(builddir)/dtd
+ rm -rf $(builddir)/xml $(builddir)/dtd
+ cp -pr $(PWD)/xml $(builddir)/
+ cp -pr $(PWD)/dtd $(builddir)/
scripts/merge.sh -b "$(builddir)" $(test) $(chapters)
$(builddir)/cll_processed_xhtml.xml: $(builddir)/cll.xml scripts/xml_preprocess.rb
@@ -52,11 +48,11 @@ xhtml_nochunks: $(builddir)/xhtml_no_chunks.done
$(builddir)/xhtml_%_chunks.done: xml/docbook2html_config_%_chunks.xsl $(builddir)/cll_processed_xhtml.xml xml/docbook2html_config_xhtml.xsl scripts/master.css scripts/xhtml_postprocess_appcache.sh scripts/xhtml_postprocess_cleanup.sh scripts/xhtml_postprocess_cleanup_file.rb
rm -rf $(builddir)/xhtml_$*_chunks
mkdir -p $(builddir)/xhtml_$*_chunks
- ln -nsf $(PWD)/dtd $(builddir)/xhtml_$*_chunks/dtd
+ cp -pr $(PWD)/dtd $(builddir)/xhtml_$*_chunks/
xmlto -m xml/docbook2html_config_$*_chunks.xsl -x $(xsl_file) -o $(builddir)/xhtml_$*_chunks/ xhtml$(nochunks) $(builddir)/cll_processed_xhtml.xml 2>&1 | grep -v 'No localization exists'
# We run tidy on the no-chunk version, because it's all in
# one place so it's easy
- test -f $(builddir)/xhtml_$*_chunks/index.html || sed 's/mml:[^>]*/div/g' $(builddir)/xhtml_$*_chunks/cll_processed_xhtml.html | scripts/tidy -e - 2>&1 | grep -v ' lacks "summary" attribute'
+ test -f $(builddir)/xhtml_$*_chunks/index.html || cat $(builddir)/xhtml_$*_chunks/cll_processed_xhtml.html | scripts/tidy -e - 2>&1 | grep -v ' lacks "summary" attribute'
# Handle the no-chunk case; can't figure out how to set the
# name of the output file, so move it to index.html
test -f $(builddir)/xhtml_$*_chunks/index.html || mv $(builddir)/xhtml_$*_chunks/cll_processed_xhtml.html $(builddir)/xhtml_$*_chunks/index.html
@@ -81,15 +77,15 @@ pdf: $(builddir)/pdf.done
$(builddir)/pdf.done: $(builddir)/cll_processed_xhtml.xml xml/docbook2html_config_prince.xsl scripts/master.css scripts/xhtml_postprocess_cleanup.sh scripts/xhtml_postprocess_cleanup_file.rb
rm -rf $(builddir)/pdf
mkdir -p $(builddir)/pdf
- ln -sf $(PWD)/scripts/master.css $(builddir)/pdf/final.css
- ln -nsf $(PWD)/media $(builddir)/pdf/media
- ln -nsf $(PWD)/dtd $(builddir)/pdf/dtd
+ cp -pr $(PWD)/scripts/master.css $(builddir)/pdf/final.css
+ cp -pr $(PWD)/media $(builddir)/pdf/
+ cp -pr $(PWD)/dtd $(builddir)/pdf/
xmlto -m xml/docbook2html_config_prince.xsl -o $(builddir)/pdf/ -x $(xsl_base)/xhtml/docbook.xsl xhtml-nochunks $(builddir)/cll_processed_xhtml.xml 2>&1 | grep -v 'No localization exists'
scripts/xhtml_postprocess_cleanup.sh $(builddir)/pdf/
ruby scripts/xml_prince_postprocess.rb $(builddir)/pdf/cll_processed_xhtml.html >$(builddir)/pdf/cll_processed_xhtml_pdf.html 2>$(builddir)/xml_prince_postprocess.out
- prince -vvv --script=scripts/prince_check_margins.js --script=scripts/prince_shave_index.js $(builddir)/pdf/cll_processed_xhtml_pdf.html $(builddir)/cll.pdf
+ prince -vvv --script=scripts/prince_check_margins.js --script=scripts/prince_shave_index.js $(builddir)/pdf/cll_processed_xhtml_pdf.html -o $(builddir)/cll.pdf
touch $(builddir)/pdf.done
ifdef copydir
mkdir -p $(copydir)
@@ -124,7 +120,7 @@ endif
$(builddir)/epub-xhtml/index.html: $(builddir)/cll_processed_xhtml.xml xml/docbook2html_config_common.xsl xml/docbook2html_config_epub.xsl $(PWD)/scripts/master.css $(PWD)/media/* scripts/xhtml_postprocess_cleanup.sh scripts/xhtml_postprocess_cleanup_file.rb
rm -rf $(builddir)/epub-xhtml/
mkdir -p $(builddir)/epub-xhtml/
- ln -nsf $(PWD)/dtd $(builddir)/epub-xhtml/dtd
+ cp -pr $(PWD)/dtd $(builddir)/epub-xhtml/
xmlto -m xml/docbook2html_config_epub.xsl -o $(builddir)/epub-xhtml/ -x $(xsl_base)/xhtml/chunk.xsl xhtml $(builddir)/cll_processed_xhtml.xml 2>&1 | grep -v 'No localization exists'
cp $(PWD)/scripts/master.css $(builddir)/epub-xhtml/final.css
cp -pr $(PWD)/media $(builddir)/epub-xhtml/
diff --git a/scripts/build_epub.sh b/scripts/build_epub.sh
index 2d3462fbc..4f61ccbf4 100755
--- a/scripts/build_epub.sh
+++ b/scripts/build_epub.sh
@@ -82,7 +82,7 @@ do
fi
properties=""
- if grep -q mml: $file
+ if grep -q '