DKAN 1.13.5

This is a "patch" release of DKAN, containing bug fixes and minor
updates, but adding no new functionality.

Improvements in this release

• This is a small release adressing some minor issues in 1.13.4. The only significant change is the addition of the devel module to DKAN core. The devel module is used by almost all Drupal developers to debug, test expiremental code, generate dummy content, and perform other developer tasks. DKAN developers are constantly adding the devel module after building DKAN, and a new set of Behat scenarios added to the test suite need to make use of the dummy content generation functions to pass.

• The devel module and its submodules will be disabled by default in DKAN however, and should only be enabled in development environments. Enabling devel in production should be considered a security risk.

• All other improvements are minor issues related to installation and tests that arose during deployments of DKAN 1.13.4. Check the CHANGELOG and Pull Request #2003 where most of these changes are captured.

DKAN 1.13.4

This is a "patch" release of DKAN, containing bug fixes and minor updates, but adding no new functionality.

Improvements in this release

The 1.13.4 release of DKAN contains a very large number of minor fixes. For a full list check the CHANGELOG. Noteable improvements include:

• Some logic concerning when previews were rendered for resources has been simplified. DKAN will now attempt to render a preview for any resource that contains an uploaded or remote file. No API or Website URL links in resources will be previewed other than with an iframe to the URL. Note that in some cases, resources that previously did have Recline previews may now simply display an iframe to the remote resource.

• Relatedly, various issues with previews for resources hosted on external domains have been addressed.

• Publishing options ("draft," "published" etc) were not available when the Workflow module was enabled. This has been fixed.

• A bug where data previews broke when a resource contained only numeric data has been fixed.

• A number of Workflow permissions issues have been resolved, allowing users to access "stale drafts" and other data that had been hidden.

• The groups page now lists groups alphabetically rather than by create date.

• Fixed the output for "language" in Project Open Data and DCAT to be standards-compliant.

• Fixed issues with how timestamps are applied to harvested datasets. See below for details.

• An existing hook_update function that upgraded from a previous version of the Open Data Federal Extras module has been fixed to not fail when the fields are not present.

• The drush dsu command, to update a datastore from a local file, was broken and has been restored.

• Windows line endings are now supported on Datastore "fast imports"

• Several issues causing the Harvester to break on large harvests have been fixed. Remote files on servers (like Socrata) that will not supply a HEAD request are briefly stored on disk instead of in memory to obtain metadata such as filesize. Also, resource previews are no longer rendered for purposes of indexing for Search API, which was causing indexing at the end of a large harvest to fail.

• Drupal core was updated to 7.56 and many contrib modules were updated to their latest stable version.

Special Notes

• Previous versions of DKAN provided a field "Modified Source Date" (machine name field_modified_source_date) to store Project Open Data's "modified" property. The way we capture sources' modified and issued properties has since been revamped (see #1802), and an update function was needed to remove the old field. However, previous versions had put hook_update functions in the wrong order and some sites did not recieve the change when updating. This release should clear up those remaining issues on sites that still have the field_modified_source_date field.

DKAN 1.13.3

This is a "patch" release of DKAN, containing bug fixes and minor updates, but adding no new functionality.

Improvements in this release

• Groups permissions were given better logic. Editors will now become administrators of any group they are added to, giving them permission to edit or moderate any content in that group. Also, Site Manager was given permissions to edit the order of featured groups.

• We now provide parsing options on the Datastore Fast Import. This means a user can set what delimiters and line terminators are used when importing a CSV file, avoiding some import errors users were experiencing.

• Some errors were fixed that caused installing DKAN with the browser (using /install.php rather than drush). Also, special characters were removed from default content filenames, fixing an installation bug on Windows systems.

• The Rules module was patched to prevent a "cache rebuild lock" (see https://www.drupal.org/node/2851567)

• Several bugs in the Harvest module were fixed:

  • The Site Manager can now use the main dashboard view to initiate harvest actions in bulk
  • Inconsistencies in date fields (Harvest source "modified" vs Drupal's "modified" dates) were addressed by adding new fields to the dataset content type.
  • A bug preventing the "temporal coverage" field from being harvested was fixed.

• The restws and media modules were updated to latest versions

• Several other smaller bug fixes and improvements; see the CHANGELOG for more information.

Special Notes

Changes to Harvester's date handling

This update changes the way the Harvest module reflects the issued and modified dates of harvested datasets. Project Open Data and most other metadata standards provide an "issued" or "created" date for datasets, as well as a "modified" date. The original release of Harvester simply overwrote Drupal's created and modified node properties with the source's dates for these fields, but we've run into two problems with this:

1. A node's modified date is easily overwritten by other actions in Drupal
2. It can be useful to store both the date that a dataset was issued in its source, and the date it was added to the portal harvesting it.

An existing field, field_modified_source_date, was already handeling some of this but we decided to scrap that and start from scratch. Starting with this release, a source's issued and modified dates will be stored in the new fields field_harvest_source_issued and field_harvest_source_modified. When these two fields are present, those will be the dates shown on a dataset's landing page on DKAN and in the site's data.json and DCAT RDF feeds.

All sources should be re-harvested after updating to this patch release to ensure that all date fields and properties are accurate.

7.x-1.13.2

DKAN 1.13.2 Release Notes

This is a "patch" release of DKAN, containing bug fixes and minor updates, but adding no new functionality.

See full 1.13 release notes here.

Improvements in this release

• Updated documentation.
• DKAN Datastore: fixed null values being imported as 0.
• Fixed errors related to the open_data_schema_apis_features_rebuild() function.
• Fixed errors when users with content creator role are editing or adding resources.
• Fixed access to the POD validation screen for site managers.
• Fixed access to the featured groups block for anonymous users.
• DKAN Harvest: fixed permissions for the site manager role, they now have access to the cache, delete, harvest, and migrate bulk operations from the Harvest Dashboard.
• DKAN Harvest: added support for importing contact name and contact email.
• Updated contrib modules: services, visualization_entity and views.

7.x-1.12.14

DKAN 1.12.14 Release Notes

This is a "patch" release of DKAN, containing bug fixes and minor updates, but adding no new functionality.

See full 1.12 release notes here.

Improvements in this release

• Added css for chart visualizaitons, fixes issue in IE.
• Updates drupal core to 7.53
• Upgrade visualization_entity to 7.x-1.1
• Upgrade services module (via datastore module) to 3.19

1.13.1

This is a "patch" release of DKAN, containing bug fixes and minor updates to DKAN 1.13, but adding no new functionality. It was released very shortly after 1.13 to address bugs that surfaced during deployments and upgrades. If you have not yet upgraded to 1.13, upgrade directly to this release and skip 1.13; if you have already upgraded, we recommend updating to 1.13.1 immediately.

See full 1.13 release notes here.

Improvements in this release

• Fix validation page permission check using wrong permission name odsm.
• Fixed a bug in the home page conversion function
• Fixed the page title (<head><title>) so that it's just the site name (not the node or panel title)
• Updated the DKAN API link on dataset pages to use the new documentation site page.
• Fixed error messages appearing on homepage after upgrade
• Fixed panelizer permissions to hide the "Customize Display" button for Site Managers.
• Additional minor bug fixes to code and tests

1.13

DKAN 1.13 Release notes

Read up on our latest release, 1.13! This is a huge release and we have new features and enhancements that will make using DKAN better than ever. Major new features include a Harvester, API improvements, new data previews, and DCAT-AP support.

Have questions or thoughts? Let us know on our public DKAN Repo in the issues queue or chat with us in our Gitter room.

What's new

DKAN Harvest

DKAN's new Harvest module gives DKAN administrators and site managers the ability to harvest datasets from other sources for their DKAN site. Built on the widely-used Drupal Migrate system, the Harvest module can fetch data and publish it to a DKAN site.

DKAN Harvest is fully configurable via the DKAN web interface. Harvest sources can be added like any other type of content, and data can be overridden or manipulated before being published. Migrations can be managed via an intuitive dashboard interface or triggered with a cron job.

Get more details on DKAN Harvest by reading its documentation pages.

DKAN Dataset REST API

Previous DKAN APIs have been read-only and focused on CKAN compatibility. A new RESTful API for DKAN datasets provides an endpoint that allows other applications to interact with DKAN, including creating and modifying datasets and resources.

Get more details on the Dataset REST API.

DCAT-AP Compliance

DKAN's DCAT RDF publishing has been reengineered to be more consistent with its other open-data standards implementation, and, more importantly, to implement the new DCAT-AP protocol. DKAN's fields can be easily mapped to DCAT-AP fields using the Open Data Schema Mapper. Now, DKAN meets US standards (Project Open Data) as well as European Union standards (DCAT-AP).

New Default Content

On a basic install of DKAN, the site came with only a few example Datasets and Resources. We added more default content in order to demonstrate the full capabilities on DKAN and give new users a better experience in testing out various DKAN features. The default content module is part of core DKAN, so all new installs of DKAN contain the content. Simply disabling the "DKAN Default Content" module will remove all the default content to give you a blank slate to start building.

What we've improved

The functionality isn't new, but it's working even better than before. These improvements make DKAN better on the whole including user experience, efficiency, and scale.

Site Managers can assign roles

The Site Manager role is part of a core group of user roles on DKAN. Site Managers have a high degree of control over the entire site, but are expected to be less technical than a full admin. Previously, it wasn't possible for Site Managers to assign roles to new users. By adding the RoleAssign module we have provided this more fine-grained permission.

Site Managers can add pages

One of the most basic content types on DKAN is a page. And though the content type is straightforward it can have implications for the structure and appearance of the site. Originally, Site Managers were not able to add or manage Pages; in DKAN 1.13 Site Managers now have this permission.

Even more permissions for Site Managers

• Theme Settings: Includes page elements and ability to add custom logo and favicon from "Site Configuration > Theme Settings" at /admin/appearance/settings
• Colorizer: Ability to create custom color schemes for a DKAN site from "Site Configuration > Colorizer" at /admin/appearance/colorizer
• Open Data Schema Mapper (ODSM): Ability to add, edit, and delete APIs and their mappings to DKAN fields from "Site Configuration > Open Data Schema Mapper" at /admin/config/services/odsm
• Menus: Permission to manage the main menu links by adding, editing, and deleting links from "Site Configuration > Menu" at /admin/structure/menu
• Topics Icons: When adding or editing terms in the Topics taxonomy at path /admin/structure/taxonomy/dkan_topics, site managers can choose an "Icon Type" of either "Font Icon" or "Image." If they choose "Font Icon," the edit form displays a list of available icons from which to choose. If they choose "Image," the form lets them upload an image.
• Enabling External Previews: Site Managers can enable previews so that site visitors can look at Resource contents with visualization tools, Carto and ArcGIS. To enable External Previews, go "DKAN > Data Previews" at /admin/dkan/dataset_preview
• DCAT and POD validation: Site Managers can select the settings for how Datasets are validated against Project Open Data and DCAT-AP standards.
• Site managers can confirm that the website's data.json feed is working correctly by using the POD online validator at https://labs.data.gov/dashboard/validate
• Site managers can confirm that the website's DCAT feed is working correctly from Site Configuration > Open Data Schema Mapper > DCAT Validation at /admin/config/services/odsm/validate/dcat or by using the DCAT online validator at http://www.dcat.be/validator/
• Order of Groups if using the featured groups block: Site Managers can arrange the order in which Groups appear in the featured group block from "DKAN > Featured Groups Sort Order" at /admin/dkan/featured-groups-sort-order
• Recline size configuration: Manage size constraints for Recline, which powers internal previews, from "DKAN > Recline Configuration" at /admin/dkan/recline

Open Data Federal Extras (ODFE) is now part of core DKAN

Project Open Data (POD) sets a standard for the information about datasets (metadata) included when the data is published. DKAN collects metadata with fields on the Dataset form when the data is published. For most agencies, the fields reflect basic requirements. However US federal agencies are required to provide additional information about the data published. The extra fields appear as part of the Dataset form with our ODFE module.

We've moved this module into core DKAN so that it's part of every install, though not enabled by default. When enabled, the ODFE module makes federal agencies compliant with POD standards.

Get more details on Federal Extras on DKAN.

In-place Editor for Site Managers + look and feel

Previously, the In-place Editor (IPE, an interface for changing page layouts) for Site Managers included several options that weren't appropriate for their level of permissions. This UI was confusing, so we improved the interface to only include options that are relevant to non-admin user roles. We also improved the overall look and feel of the IPE to be more user-friendly.

Improved Data Previews

The Data Previews feature is designed to let site visitors take a sneak peek at the content of a Resource before downloading the file. In this release, we've improved Data Previews to support more file formats beyond CSV and XLS. Now previews also support JSON, geojson, XML, ArcGIS REST, WMS, images, PDF, and ZIP data formats.

Improvements to Data Previews also offer better support for Resources that are hosted remotely. Previously, Resources that were linked to a web source could only be previewed if they were first imported into the DKAN Datastore (which only supported CSV files). In DKAN 1.13, linked Resources can be previewed with the Data Previews feature.

Improved Datastore API

The DKAN Datastore API makes it possible to query for contents of Resources uploaded to the Datastore as detailed as a cell within a table. The improved Datastore API is enhanced to open greater possibilities of requesting complex queries from contents within a Datastore on a DKAN site as well as multiple queries in a single request.

Get more details on the Datastore API

New Dataset fields added

A number of metadata fields that are common requirements for open data standards like Project Open Data and DCAT-AP have been added to the basic dataset content type in DKAN. These fields are:

• Data Standard
• Data Dictionary Type
• Homepage URL
• Data Standard (conformsTo)
• Rights
• Language

The following fields are also included but hidden. This allows the harvester to ingest metadata with these fields and present them in data.json but they are not yet supported in the UI for locally-created datasets:

• Collection (isPartOf)
• Theme

See the Project Open Data schema for more information on these fields.

NuBoot Radix now in core DKAN

Previously, the NuBoot Radix theme on DKAN was not part of the core repository. The old Omega-based theme and its tools (the Delta module) have been removed, and NuBoot Radix is no longer maintained in a separate repository. Legacy themes will not be maintained or shipped with DKAN and require manual installation if you want to use the Omega theme. Alternatively, you can make customizations to the default theme.

Additionally, the admin theme setting is set to use the default theme rather than the NuBoot Radix theme. Because the Colorizer tool requires both the admin and default to be the same, setting the admin theme setting to use the default theme enables Colorizer to work normally for both the NuBoot subtheme and other custom subthemes without admins needing to manually override this setting.

Added POD-based...

7.x-1.12.13

This is a "patch" release of DKAN, containing bug fixes and minor updates to DKAN 7.x-1.12, but adding no new functionality. Upgrading should be straightforward.

Improvements in this release

• Fixed broken recline (resource CSV) preview embeds caused by how recline module loads bootstrap.
• Added patch to remote_stream_wrapper to avoid memory exhausted errors on big files.
• Fixed publisher token in open_data_schema_map_dkan. It was showing only URL rather than publisher name.
• Remote linked files are now proxied through a DKAN site so that we don't run into CORS issues for the visualizations. That can cause memory issues with large files. This only proxies files smaller than 50MB.

7.x-1.12.12

This is a "patch" release of DKAN, containing bug fixes and minor updates to DKAN 7.x-1.12, but adding no new functionality. Upgrading should be straightforward.

Group assignment cleanup update

This release contains a database update that may take some time to complete on larger sites. As described in #1267, on older DKAN sites it was very easy to add resources to datasets that would not be added to the parent dataset's group.

Many sites may have this issue and not even realize, but it will create problems for organizations that use DKAN's out-of-box group roles and permissions; it give users permission to edit content in their own group but not content outside of it. If a resource is not assigned to a group, this could block a group user from editing their groups' own datasets.

This patch release contains a database update that will identify datasets with resources missing their group assignments, and correct them. See pull request #1491 for the fix itself.

Other improvements in this release

• Data previews can be captured in a certain state for embedding into other web pages as an iframe, but these previews were not cached by Drupal. This has been fixed; embedded previews are now cached, which is a significant performance boost for some websites.
• Upgraded Drupal core to 7.52, ctools to 1.11, and Media to 2.0-beta13.
• Fixed links to group pages from group node teasers that broke when site has clean urls disabled
• Fixed some issues with certain filters/facets on the search page being unexpectedly collapsed.
• Fixed a bug introduced on a previous patch release that hid the body field on page nodes.
• Improved the access check and security on datastore pages - unauthorized users could perform certain datastore functions by guessing URLs.
• Fixed "page not found" errors when clicking certain topics links with spaces or special characters in them.

7.x-1.12.11

This is a "patch" release of DKAN, containing bug fixes and minor updates to DKAN 7.x-1.12, but adding no new functionality. Upgrading should be straightforward.

Improvements in this release

• Fixed a bug in Recline regarding file objects and node forms that caused errors when using the "view changes" button on Dataset or Resource edit form
• Center group images in "Groups" page and group node page sidebar
• Add "2x," "3x" etc to Dataset teasers when more than one resource of a particular format present.
• Update the default jquery library setting from 1.7 to 1.10
• Fixed topics menu and facet links if special characters are used in topic terms.
• Fixed dataset form redirect when validation fails, was sending user to node/add/dataset rather than node/%/edit
• Patch fontyourface to remove
  from the "standard text" selector, to prevent unpredictable results from this option
• Fixed issue in open_data_schema_map with "Data Dictionary" field not displaying URL in data.json file