README: Add more info on IBM Spectrum Protect (TSM) setup

Previously the only information available on the TSM setup needed was in the NeIC wiki, adapted the important parts of that
documentation for a broader audience.

Also move a misplaced note on ENDIT plugin configuration to the right place.

README.md

@@ -30,16 +30,57 @@ https://wiki.neic.no/wiki/DCache_TSM_interface
 ## TSM (IBM Spectrum Protect)
 
 Setup TSM so that the user running dCache can `dsmc archive` and `dsmc
-retrieve` files. If you want to have several pool-nodes talking to tape, you
-probably want to setup a TSM proxy node that you can share across machines
-using `dsmc -asnode=NODENAME`.
+retrieve` files. If you want to have several pool-nodes talking to tape, we
+recommend setting up a TSM proxy node that you can share across machines
+using `dsmc -asnode=NODENAME`. Due to recent changes in TSM client authentication
+we strongly recommend not using a machine-global TSM node, but instead creating
+a dedicated TSM node for each dCache runtime user. See
+the [IBM documentation re non-root usage](https://www.ibm.com/support/knowledgecenter/en/SSEQVQ_8.1.6/client/c_cfg_nonadmin.html)
+for the recommended setup.
 
 A dCache hsminstance typically maps into a dedicated TSM proxy node. With a
 proxy node you can have multiple read and write pool nodes to the same data in
 TSM. Different TSM nodes need to have different hsminstances.
 
-Note that you need to increase the node `MAXNUMMP` setting to the sum of
-concurrent/parallel `dsmc archive` and `dsmc retrieve` sessions.
+The common ENDIT-optimized TSM storage hierarchy setup is to have a dedicated domain for each proxy node
+and define a tape storage pool as the archive copygroup destination. Since `tsmarchiver.pl`
+batches archive operations into larger chunks there is limited benefit of
+spooling data to disk on the TSM server before moving it to tape.
+
+For each TSM node defined on your TSM server, ensure that the following options are
+correct for your environment:
+* `MAXNUMMP` - Increase to the sum of concurrent/parallel `dsmc archive` and
+`dsmc retrieve` sessions plus some margin to avoid errors when tapes are concurrently
+being mounted/dismounted.
+* `SPLITLARGEObjects` - set to No to optimize for tape.
+
+On your TSM client machine (ie. dCache pool machine), ensure that you have set the appropriate tuning options for
+optimizing performance in a tape environment, see the [IBM documentation on Using high performance tape drives](https://www.ibm.com/support/knowledgecenter/SSGSG7_7.1.6/perf/c_srv_tape_drives_highperf.html) for further details.
+It is also recommended to define the `out` directory as a separate file system in TSM using the `VirtualMountPoint`
+configuration option.
+
+Typical `dsm.sys` excerpt:
+```
+TXNBYTELIMIT      10G
+VIRTUALMountpoint /grid/pool/out
+```
+
+We also recommend disabling ACL support as this is file system specific
+(as in you can't restore files to a different file system type) thus having
+it enabled makes it hard to change the setup in the future.
+
+Typical `dsm.opt` excerpt:
+```
+SKIPACL YES
+```
+
+If the machine is running scheduled TSM backups you want to exclude the pool filesystem(s) from the backup.
+
+Typical system `include-exclude` file excerpt:
+```
+exclude.dir     /grid/pool*/.../*
+exclude.fs      /grid/pool*/.../*
+```
 
 ## dCache
 
@@ -52,6 +93,9 @@ flushing.
 To get any efficiency in retrieves, you need to allow a large number of
 concurrent restores and have a long timeout for them.
 
+The configuration of the ENDIT dCache plugin is done through the dCache
+admin interface.
+
 ## ENDIT daemons
 
 Download the ENDIT daemons to a directory of your choice, `/opt/endit` is our
@@ -155,9 +199,6 @@ To run multiple/concurrent ENDIT daemon instances, the `ENDIT_CONFIG` environmen
 to use a different configuration file. This is not to be confused with enabling parallel/multiple archive and
 retrieve operations which is done using options in the ENDIT daemon configuration file.
 
-The configuration of the ENDIT dCache plugin is done through the dCache
-admin interface.
-
 # Collaboration
 
 It's all healthy perl, no icky surprises, we hope. Patches, suggestions, etc are