aqp 2.0

This is an abbreviated version of the "What is new in aqp 2.0?" vignette. Many thanks to @brownag, @smroecker, @pierreroudier, @jskovlin, @jjmaynard, and all of the other contributors to this version of aqp.

This is a major update to aqp that may create some issues for code depending on specific inputs/outputs in aqp < 1.42, particularly those relying on slice(), slab(), and profile_compare(). slice() and profile_compare() are now deprecated, but will continue to work for the rest of calendar year 2023. There are no plans to maintain these functions beyond aqp 2.0. The new version of slab() is a drop-in replacement for the previous version of the function.

New Vignettes:

• Numerical Classification of Soil Profiles
• Overlapping Annotation

Notable changes include:

• deprecation of slice() in favor of the new, faster, more robust implementation in dice()
• complete overhaul of slab(), with new arguments, faster back-end, and weighted aggregation implemented (finally)
• deprecation of profile_compare() in favor of the NCSP()--a complete overhaul based on Maynard et al., 2020
  • site level attributes now handled by compareSites()
  • variable weights now possible via argument
• perturb() and estimatePSCS() are now vectorized, and optimized for larger collections
• mixMunsell() now uses mixingMethod = 'exact' by default for the simulation of subtractive mixtures
• gower package moved to SUGGESTS
• plotColorMixture() now using grid graphics functions to determine color swatch geometry and setting overlap detection threshold
• removal of PMS2Munsell() and support data
• deprecation of coordinates()<- and proj4string()<- in favor of initSpatial()<-
• removal of rruff.sample example XRD patterns
• get.ml.hz() no longer uses the name argument

Major changes to plotSPC():

• The maximum depth range of the figure is now based on max.depth or max(x). This means that sketches generated with aqp 2.x will generally have less white space at the bottom of the figure. Make more room for additional annotation or visual effect by setting the desired depth range with the max.depth argument.
• now uses electroStatics_1D() for fixing horizon depth label overlap, solutions are deterministic and almost always better
• better depth axis interval heuristics (if not specified), varying based on figure depth range
• depth axis adjustments via new argument depth.axis, logical or list
• deprecation of arguments:
  • plot.depth.axis: set via depth.axis = TRUE, depth.axis = FALSE, or customize depth.axis = list(...)
  • cex.depth.axis: set via depth.axis = list(cex = 1)
  • axis.line.offset: set via depth.axis = list(line = -2)

New features:

• example data, wilson2022
• fast prototyping of SPCs via quickSPC() and list / character templates
• re-use arguments to plotSPC() via options(.aqp.plotSPC.args = list(...))
• coarse fragment classification via fragmentSieve() and fragmentClasses()
• S4 as.data.frame(<SPC>) as shorthand for as(<SPC>, 'data.frame')
• plotSPC() now marks truncated profiles with a ragged bottom
• fixOverlap() now has two label-placement solvers, based on 1) electrostatics and 2) simulated annealing
• new depth axis styles in plotSPC()

Incremental changes, should have no effect on previous code:

• bug fix in plotSPC() when fixLabelCollisions = TRUE, adjustments suggested to fixOverlap() are now scaled correctly
• explainPlotSPC() reports label adjustment index when label collision repair is enabled
• aesthetic cleanup in explainPlotSPC()
• soilColorSignature() gains arguments and perceptual color distances (dE00) via farver package
• as(<SPC>, "data.frame"): Replace plyr::join() with merge()
• correctAWC(): NA handling - return NA when frags are NA
• mutate_profile(): Faster (data.table-based) evaluation of profile-level expressions (#255)
• profileApply: Add support for custom lapply()-like function (APPLY.FUN) for processing chunks (#256)
• Add .interpretHorizonColor() outputs to last_spc_plot in aqp.env for use in custom legend() (#254)
• Add simplify argument to SoilTextureLevels() and ssc_to_texcl() to optionally convert to an ordered factor with maximum of 12 levels (rather than 21). This smaller list of classes excludes sand grain size variants such as fine sand, loamy coarse sand, and very fine sandy loam.

aqp 1.31

Enhancements / Bug Fixes

• bug fix in checkHzDepthLogic() when byhz = TRUE
• aggregateColor() now uses mixMunsell for the estimation of soil color mixtures
• plotColorMixture() will respect "names" attribute of colors-to-mix, without erroneous alpha-sorting
• parseMunsell() now more robust and faster, c/o P. Roudier
• mixMunsell:
  • new method exact for direct conversion of mixture spectra to sRGB or closest Munsell chip (via spec2Munsell())
• glom() z1 and z2 arguments vectorized to allow for profile-specific intervals
  • z1 and z2 support non-standard evaluation based on column names in siteNames(p), and also can take character vector (length 1) with column names in siteNames(p)
• depthOf(), minDepthOf(), maxDepthOf(), getSurfaceHorizonDepth(), getMineralSoilSurfaceDepth(), getPlowLayerDepth() can now be applied to multiple profiles.
  • If the input SoilProfileCollection has more than one profile then result is a data.frame containing profile ID, top or bottom depths, horizon designation and pattern

New Functions / Data

• new function colorChart() for graphical depiction of Munsell chip frequency by group
• unique method for SoilProfileCollection objects now returns a SoilProfileCollection by default
  • this may break existing code! use the new argument SPC = FALSE for previous behavior (#159)
• new convenience function PMS2Munsell() for converting PMS codes -> closest Munsell chip (#124)

aqp 1.27

Enhancements / Bug Fixes

• mixMunsell now relies on suggested package {gower} for 5-10x speed bump
• {aqp} no longer imports from {reshape} (less one dependency), all transformations from wide<->long are done via {data.table}
• methods from {data.table} are now imported by {aqp} (new dependency)
• Major overhaul of plotColorQuantiles(), now using {lattice} graphics
• Argillic critical clay contents crit.clay.argillic rounded to whole numbers per NSSH Part 614, subpart B, sections 614.13 and 614.14
• mutate_profile uses data.table::rbindlist(fill=TRUE) to combine site- and horizon-level transformations
• updates to horizon boundary encoding functions (hzTopographyCodeToOffset, hzTopographyCodeToLineType, hzDistinctnessCodeToOffset)
• plotSPC updates:
  • argument named changes: hz.boundary.lty is a horizon-level attribute that contains line type codes
  • hz.topography.offset a horizon-level attribute that contains representative offsets that encode horizon boundary topography
  • plotSPC now encodes hz.topography.offset using a vertical "bump" (chevron)
• addBracket can now accept multiple bracket annotations per profile

New Functions / Data

• New dataset equivalent_munsell and method equivalentMunsellChips for "equivalent" Munsell chips lookup list based on all pairwise dE00 contrasts for integer "chips" in {aqp} munsell data set
• new function L1_profiles computes multivariate (L1) medians, compare to marginal medians via slab

aqp 1.25

This release of aqp marks a significant step towards major changes planed for version 2.0. Since 1.17, there have been major changes to the internals of the SoilProfileCollection object (thanks to @brownag) and associated methods. We expect some evolution (but less drastic as compared to the previous release) before version 2.0.

See NEWS.md for a complete listing of changes.

Important Changes

• estimateSoilDepth loses top and bottom arguments, these are automatically extracted
• combine replaces/expands aqp::union due to conflicts with base::union
• split receives some upgrades to the S4 definition to increase parity with split.default
• filter is now an alias for new method subset, which mirrors base::subset
• default horizon ID (hzID) is now a character data type

Enhancements

SoilProfileCollection Internals

• new function duplicate will makes copies of profiles within a SoilProfileCollection
• two new SoilProfileCollection wrapper methods: munsell2SPC, spc2mpspline
• improvements to glom(..., invert=TRUE), glomApply, and better tests
• new wrapper method around glomApply: trunc for cases when top and bottom depth interval is the same for all profiles in a SoilProfileCollection
• enhanced SoilProfileCollection object validity checks via S4; new method spc_in_sync (#152)
• optimization of [ subset method and optional use of data.table (#155)
• depths<- has been optimized and minimally validates input data
• combine uses depths<- internally; explicitly enforcing profile ID + top depth order in horizon data is safer but results in different ordering if union-ing IDs that "intermingle" (need to be re-sorted).
• new experimental method is permute_profile; similar to sim but for boundaries. The interface to this function is likely to change/be expanded.
• basic support for promotion of tbl_df and data.table to SoilProfileCollection
• new method aqp_df_class to determine class name in use in a SoilProfileCollection object
• optimization of [i,] [,j] subset methods for data.frame-based slots (#135)
• new verbs: mutate, mutate_profile (#118)
• define [[ subsetting method; an "ambivalent" accessor for site- or horizon-level properties
• new subset verbs grepSPC, filter, subApply for use in %>%-lines

Color / Visualization

• simulate subtractive mixtures of Munsell colors with mixMunsell
  • see companion function plotColorMixture for visualization of spectra / mixture
• complete overhaul of textureTriangleSummary:
  • uses soiltexture package for visualization (plotrix implementation dropped)
  • argument names changes (! may break old code, sorry)
  • dropped simulation via sim = TRUE argument, see bootstrapSoilTexture for a better approach
• new function bootstrapSoilTexture for simulating realistic sand/silt/clay compositions
• add returnData argument to contrastChart
• plotSPC upgrades (#146)

Everything Else

• new methods related to mollic epipedon: mollic.thickness.requirement, hasDarkColors
• new estimateSoilDepth-like methods for depth to multiple features via pattern matching: depthOf, minDepthOf, maxDepthOf
• soil texture helper functions: ssc_to_texcl, texcl_to_ssc, texmod_to_fragvoltot, texture_to_taxpartsize c/o @smroecker
• added segment c/o @smroecker

Additions

• new lookup table pms.munsell.lut for converting Pantone spot color codes to (closest) Munsell chip
• new example data us.state.soils: 50 state soils + PR and VI soils
• ROSETTA centroids and water retention by texture class (#131)

Bug Fixes

• fix for routing of NULL through $<- and horizons<- or site<- (#163)
• fix handling of missing metadata in (old) serialized SoilProfileCollection objects
• fix for promotion of data.table with character vector (not formula) interface
• fix for unit-length and zero-length legends in plotSPC
• fix for plot generic to show aqp::plot in ?plot
• fix for getSurfaceHorizonDepth with buried horizons / non-contiguous instances of matching horizons (#132)
• fix for default plotSPC with small number of profiles (#128)
• remove implicit conversion to SpatialPointsDataFrame with unit-length [ j-index subset ((#125)
• fix in slab when slab.structure[2] > max(x)

New Compatibility with Pipe-based Workflows!

The SoilProfileCollection object is now compatible with {magrittr} pipe %>% based workflows. Here is an example using select soil morphologic data from Jacobs (2002) "Redoximorphic Features as Indicators of Seasonal Saturation, Lowndes County, Georgia", a sample dataset available in {aqp} (jacobs2000)

# {aqp} + {magrittr} pipes
library(aqp)
library(magrittr)

# see ?jacobs2000 for details
data("jacobs2000", package = "aqp") 

# create a base plot to inspect full dataset
par(mar=c(0,0,0,2))
plot(jacobs2000, color = "matrix_color", 
     plot.order = order(jacobs2000$time_saturated))

# select 3 profiles along gradient, and truncate 50-150cm interval
jacobs.sub <- jacobs2000 %>% 
                subset(profile_id(.) %in% c("92-2","92-4",'92-7')) %>%
                trunc(50, 150)

# create a base plot
plot(jacobs.sub, color = "matrix_color")

# add redox concentrations
jacobs.sub %>% addVolumeFraction('concentration_pct', 
                     col = .$concentration_color, 
                     pch = 16, cex.max = 1)

# add redox depletions 
jacobs.sub %>% addVolumeFraction('depletion_pct', 
                     col = .$depletion_color, 
                     pch = 16, cex.max = 1)

aqp v1.17

This is the last major release of the 1.x series. Future development will focus on the 2.x series of releases.

Important Changes

SoilProfileCollection

Pre-v1.17 soilProfileCollection objects can be checked and rebuilt using ideas from #74. Further documentation pendning.

Structure before v1.17.

Formal class 'SoilProfileCollection' [package "aqp"] with 7 slots
  ..@ idcol     : chr "soil"
  ..@ depthcols : chr [1:2] "top" "bottom"
  ..@ metadata  :'data.frame':	1 obs. of  1 variable:
  ..@ horizons  :'data.frame':	1539 obs. of  18 variables:
  ..@ site      :'data.frame':	296 obs. of  1 variable:
  ..@ sp        :Formal class 'SpatialPoints' [package "sp"] with 3 slots
  ..@ diagnostic:'data.frame':	0 obs. of  0 variables

Structure as of v1.17, note the new hzidcol slot.

Formal class 'SoilProfileCollection' [package "aqp"] with 8 slots
  ..@ idcol     : chr "soil"
  ..@ hzidcol   : chr "hzID"
  ..@ depthcols : chr [1:2] "top" "bottom"
  ..@ metadata  :'data.frame':	1 obs. of  1 variable:
  ..@ horizons  :'data.frame':	1539 obs. of  18 variables:
  ..@ site      :'data.frame':	296 obs. of  1 variable:
  ..@ sp        :Formal class 'SpatialPoints' [package "sp"] with 3 slots
  ..@ diagnostic:'data.frame':	0 obs. of  0 variables

rbind.SoilProfileCollection

Has been deprecated in favor of aqp::union(), and gains new functionality.

Enhancements

Soil Color Related

• aggregateColor() gets a new feature, similar colors can be grouped via cluster::pam()
• new functions: previewColors(), colorQuantiles(), plotColorQuantiles()

SoilProfileCollection Management

• horizonDepths()<-, edit top/bottom names after SoilProfileCollection init
• profile_id()<-, edit profile IDs after init; be careful!
• hzID() and hzID()<-, get/set unique horizon IDs
• hzidname() and hzidname()<-, get/set column containing unique horizon IDs
• checkSPC and rebuildSPC

Soil Taxonomy Related

• argillic.clay.increase.depth
• crit.clay.argillic
• estimatePSCS
• get.increase.depths
• get.increase.matrix
• getArgillicBounds
• getSurfaceHorizonDepth

Horizon Intersection (glom) Related

• clod.hz.ids
• glom

New / Updated Documentation

• Investigating Soil Color

Bug Fixes

• sanity checks within horizonNames()<-

Misc. Changes

Note that some un-used sample data have been removed. The sp5 sample dataset has been re-made to include the new @hzidcol slot.