Skip to content

Release 2022-12-08
Compare
Choose a tag to compare
@sreekzz sreekzz released this 08 Dec 15:52
· 2 commits to master since this release
2022-12-08
4393e92
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

Azure HDInsight release notes

This article provides information about the most recent Azure HDInsight release updates.

Summary

Azure HDInsight is one of the most popular services among enterprise customers for open-source analytics on Azure.

Release date: December 12, 2022

This release applies to HDInsight 4.0. and 5.0 HDInsight release is made available to all regions over several days.

HDInsight uses safe deployment practices, which involve gradual region deployment. It may take up to 10 business days for a new release or a new version to be available in all regions.

OS versions

  • HDInsight 4.0: Ubuntu 18.04.5 LTS Linux Kernel 5.4
  • HDInsight 5.0: Ubuntu 18.04.5 LTS Linux Kernel 5.4

For workload specific versions, see here.

What's New?

  • Log Analytics - Customers can enable classic monitoring to get the latest OMS version 14.19. To remove old versions, disable and enable classic monitoring.
  • Ambari user auto UI logout due to inactivity. For more information, see here
  • Spark - A new and optimized version of Spark 3.1 is included in this release.

New Regions

  • Qatar Central
  • Germany North

What's changed?

  • HDInsight has moved away from Azul Zulu Java JDK 8 to Adoptium Temurin JDK 8, which supports high-quality TCK certified runtimes, and associated technology for use across the Java ecosystem.

  • HDInsight has migrated to reload4j. The log4j changes are applicable to

    • Apache Hadoop
    • Apache Zookeeper
    • Apache Oozie
    • Apache Ranger
    • Apache Sqoop
    • Apache Pig
    • Apache Ambari
    • Apache Kafka
    • Apache Spark
    • Apache Zeppelin
    • Apache Livy
    • Apache Rubix
    • Apache Hive
    • Apache Tez
    • Apache HBase
    • Apache OMI
    • Apache Pheonix

Updated

HDInsight will implement TLS1.2 going forward, and earlier versions will be updated on the platform. If you're running any applications on top of HDInsight and they use TLS 1.0 and 1.1, upgrade to TLS 1.2 to avoid any disruption in services.

For more information, see How to enable Transport Layer Security (TLS)

End of Support

End of support for Azure HDInsight clusters on Ubuntu 16.04 LTS from 30 November 2022. HDInsight had begun release of cluster images using Ubuntu 18.04 from June 27, 2021. We recommend our customers who are running clusters using Ubuntu 16.04 is to rebuild their clusters with the latest HDInsight images by 30 November 2022.

For more information on how to check Ubuntu version of cluster, see here

  1. Execute the command “lsb_release -a” in the terminal.

  2. If the value for “Description” property in output is “Ubuntu 16.04 LTS”, then this update is applicable to the cluster.

Bug fixes

  • Support for Availability Zones selection for Kafka and HBase (write access) clusters.

Open-source bug fixes

Hive bug fixes

Bug Fixes Apache JIRA
HIVE-26127 INSERT OVERWRITE error - File Not Found
HIVE-24957 Wrong results when subquery has COALESCE in correlation predicate
HIVE-24999 HiveSubQueryRemoveRule generates invalid plan for IN subquery with multiple correlations
HIVE-24322 If there's direct insert, the attempt ID has to be checked when reading the manifest files
HIVE-23363 Upgrade DataNucleus dependency to 5.2
HIVE-26412 Create interface to fetch available slots and add the default
HIVE-26173 Upgrade derby to 10.14.2.0
HIVE-25920 Bump Xerce2 to 2.12.2.
HIVE-26300 Upgrade Jackson data bind version to 2.12.6.1+ to avoid CVE-2020-36518
Assets 2
Loading