branching.md

Ansible OpenStack Collection and its branches

Our codebase has been split into two separate release series, 2.x.x and 1.x.x:

• 2.x.x releases of Ansible OpenStack collection are compatible with OpenStack SDK 1.x.x and its release candidates 0.99.0 and later only (OpenStack Zed and later). Our master branch tracks our 2.x.x releases.
• 1.x.x releases of Ansible OpenStack collection are compatible with OpenStack SDK 0.x.x prior to 0.99.0 only (OpenStack Yoga and earlier). Our stable/1.0.0 branch tracks our 1.x.x releases.
• 2.x.x releases of Ansible OpenStack collection are not backward compatible to 1.x.x releases ⚠️

Both branches will be developed in parallel for the time being. Patches from master will be backported to stable/1.0.0 on a best effort basis but expect new features to be introduced in our master branch only. Contributions are welcome for both branches!

Our decision to break backward compatibility was not taken lightly. OpenStack SDK's first major release (1.0.0 and its release candidates >=0.99.0) has streamlined and improved large parts of its codebase. For example, its Connection interface now consistently uses the Resource interfaces under the hood. This required breaking changes from older SDK releases though. The Ansible OpenStack collection is heavily based on OpenStack SDK. With OpenStack SDK becoming backward incompatible, so does our Ansible OpenStack collection. For example, with openstacksdk >=0.99.0 most Ansible modules return dictionaries instead Munch objects and many of their keys have been renamed. We simply lack the development resources to maintain a backward compatible interface in Ansible OpenStack collection across several SDK releases.

Notable changes between release series 2.x.x and 1.x.x

When we ported our collection to openstacksdk >=0.99.0, a series of changes were applied to our master branch. We went through each module in our collection and did the following:

• Identify function calls which use openstacksdk's cloud layer, e.g. self.conn.get_network(). Change these calls to functions from openstacksdk's resource proxies, e.g. self.conn.network.find_network(), if possible. As a guideline use this decision tree:
  • If a functionality requires a single api call (to the OpenStack API), then use functions from openstacksdk's resource proxies.
  • If a functionality requires multiple api calls (to the OpenStack API), e.g. when creating and attaching a floating ip to a server, then use functions from openstacksdk's cloud layer.
  • When unsure which of openstacksdk's layers to use, then first go to resource proxies and then to its cloud layer. Mainly this applies to functions retrieving information, i.e. all calls where we get info about cloud resources should be changed to openstacksdk functions which return proxy resources. Note: Using openstacksdk's cloud layer for functionality which is not provided by openstacksdk's proxy layer is acceptable. openstacksdk's cloud layer is not going away. For example, listing functions in openstacksdk's cloud layer such as search_users() often allow to filter results with function parameter filters. openstacksdk's proxy layer does not provide an equivalent and thus using search_users() is fine.
• Functions in openstacksdk's cloud layer often have different return values then pre-0.99.0 releases. When return values have changed in any of the functions which a module uses, update RETURN variable. If a module has no RETURN variable, define it.
• Only return data types such as lists or dictionaries to Ansible. For example, the return statement self.exit_json(changed=False, floating_ips=floating_ips) in module floating_ip_info shall return a list of dict's. Use openstacksdk's to_dict function to convert resources to dictionaries. Setting its parameters such as computed to False will drop computed attributes from the resulting dict. Read to_dict's docstring for more parameters. Using to_dict might change the return values of your Ansible module. Please document changes to return values in RETURN.
• Older openstacksdk releases did not provide the to_dict function. We decided to allow breaking backward compatibility with release 2.x.x, so workarounds such as (o.to_dict() if hasattr(o, 'to_dict') else dict(o)) are not required anymore and shall be avoided.
• Manually dropping attributes such as location or link from openstacksdk resources is no longer necessary. Workarounds such as

  for raw in self.conn.block_storage.backups(**attrs):
    dt = raw.to_dict()
    dt.pop('location')
    data.append(dt)

  are no longer necessary and can be removed.
• Add tests to ci/run-collection.yml and ci/roles. Each module has a dedicated Ansible role with tests in ci/roles. Create one if no such directory exist.
• With release of openstacksdk 0.99.0 most of our CI tests in ci/ failed. To prove that module patches actually fix issues all CI tests for unrelated broken modules have to be skipped. To run CI tests for patched modules only, temporarily list the Ansible tags of all CI tests which should run in vars: { tox_extra_args: ... } of job ansible-collections-openstack-functional-devstack-ansible in .zuul.yaml (example) and send the patch for review. Once all CI tests are passing in Zuul CI, undo changes to .zuul.yaml, i.e. revert changes to tox_extra_args and submit final patch for review.
• Cherry-pick or backport patches for master branch to stable/1.0.0 branch. Both branches should divert only if necessary in order to keep maintainence of two separate branches simple. When applying patches to the stable/1.0.0 branch, it is often necessary to make changes to not break backward compatibility on the stable/1.0.0 branch. On master we use .to_dict(computed=False) which we have to change to .to_dict(computed=True) on stable/1.0.0. For example, this patch for master branch has been tweaked and cherry-picked to stable/1.0.0 branch. Backporting patches from master to stable/1.0.0 branch have been abandoned due to lack of time and resources ⚠️
• Version checks in modules are no longer necessary because we require openstacksdk >=0.99.0 globally. For example, drop min_ver/max_ver constraints on module arguments.
• Rename module parameter names to the attribute names that openstacksdk uses, e.g. shared becomes is_shared. Keep old names as aliases for input backward compatibility.
• Some modules have if-else branches for handling cases where a name is given. For most modules these can be dropped safely because names can be passed as a query parameter.
• Some modules do not use name as module parameters for resource names. For example, port module had an attribute called port instead of name. Rename those attributes to name to be consistent with other modules and because openstacksdk is doing the same. Add old attribute names as aliases to keep input backward compatibility.
• Replacing self.conn.get_* with self.conn.*.find_* functions provide a ignore_missing=False parameter. This allows to drop self.fail_json() calls in modules. Less code means less to maintain.
• Some modules pass ignore_missing=True to self.conn.*.find_* functions and then fail if the return value is None. Often this code can be simplified by changing ignore_missing to False and dropping the if-else branches.
• When module attribute that have choices, always doubt its values. The module code was probably written long ago and the choices given might be outdated. It might also make sense to drop the choices parameter completely when choices are to narrow and might soon be outdated again.
• Check comments whether they are still relevant.
• Sanity check existing integration tests. For example, return values of module calls should be tested else running a test could be useless in the first place.
• Most functions in openstacksdk's cloud layer no longer return Munch objects. Instead they return resources which should be converted to dictionaries. Update RETURN docs in modules, e.g. change from type: complex to type: dict.
• Move list of expected module results to role defaults, e.g. define a variable expected_fields. This enables easier reuse.
• Following and applying our development guide and review guide