typedefs from a submodule not included in a module

[jrsmroz]

Should entries (i.e. typedef) defined in a submodule be found from a module? It seems currently they're not. I have a yang files from a vendor, that defines typedefs in a submodule, then referes to them using a prefix for a module. And it fails to lookup the typedefs.

For example:

submodule some-vendor-foo-sub {
  belongs-to some-vendor-foo-module {
    prefix some-vendor-foo;
  }
 // ...
  typedef xg-port-id {
    type uint16;
  }
}

Then in other modules:

module some-vendor-bar-module {
  // ... 
  import some-vendor-foo-module  {
    prefix some-vendor-foo
  }
  // ...
  grouping some-grouping {
    // .. 
    list some-list {
      key xg-port-id;
      leaf xg-port-id {
        type some-vendor-foo:xg-port-id;
      }
  }
}

It fails when parsing the modules in findExternal method. I have modified findExternal method, adding a loop that checks the submodules of a modules for a typedef:

func (d *typeDictionary) findExternal(n Node, prefix, name string) (*Typedef, error) {
	root := FindModuleByPrefix(n, prefix)
	if root == nil {
		return nil, fmt.Errorf("%s: unknown prefix: %s for type %s", Source(n), prefix, name)
	}
	if td := d.find(root, name); td != nil {
		return td, nil
	}
	for _, sm := range root.Modules.SubModules {
		if td := d.find(sm, name); td != nil {
			fmt.Println("Found in a Submodule!")
			return td, nil
		}
	}
	if prefix != "" {
		name = prefix + ":" + name
	}

	return nil, fmt.Errorf("%s: unknown type %s", Source(n), name)
}

"Found in a Submodule!" is being printed. Is the vendor wrong defining typedefs in a submodule, the referring to them by module with prefix or is goyang incorectly handling submodules?

Given:
4.1

  YANG structures data models into modules and submodules.  A module
   can import data from other external modules, and include data from
   submodules.

5.1

  Submodules are partial modules that contribute definitions to a
   module.  A module may include any number of submodules, but each
   submodule may belong to only one module.

[andaru]

The vendor's YANG is semantically valid, the some-vendor-foo:xg-port-id identifier should resolve. This is used in the ietf-* modules for one, and I'm guessing you may have run into it there (particularly as the OC modules use the IETF interface definitions).

Note that where and how a definition's identifier is to be found differs between YANG 1.0 and YANG 1.1: the differences in section 5.1 of RFC6020 vs RFC7950 describe exactly how this differs. And this is not only for typedef statements, any definition identifier mentioned in a statement covered in section 6.2.1 is effected.

The tl;dr is: in YANG 1.1, the ultimate parent module's definition namespace includes all top-level identifiers (including those found via a uses) from all submodules (recursively). In YANG 1.0, only the current submodule includes those top-level identifiers (again, recursively for all included submodules; again including those via uses).

For lookup, in YANG 1.1, the identifier namespace extends up to the parent module, and down recursively to all included submodules (using the same rules within a (sub)module). In YANG 1.0, the namespace extends from the current (sub)module file, recursively down to all included submodules.

---

In YANG 1.0, the definition scope for "local" definitions (whether referenced without a prefix or using the explicit local module prefix, as I understand it) is only those found by traversing the current file (module, submodule) and that of the include statements found (recursively, if submodules are included by submodules).

The first principle approach to that is to search the current module (current statement, all children of each parent statement (either defined locally or available via a uses statement, recursively... until the local [sub]module statement), then search all included submodules top-level statements for the definition, recursively (i.e., following any included submodules in each submodule). But, you need not go up to the parent (aka belongs-to) module if you started in a submodule. This changes in YANG 1.1.

YANG 1.1 differs in that any submodule included (recursively) by its parent module has its top-level definitions pulled into the namespace of the parent module. The reason for and effect of this change is that fewer include statements are required in adjacent submodules using the same definitions.

Thus, the first principle approach for (any 6.2.1) identifier lookup is the same for a module, but when performing the lookup in a submodule, if the definition's identifier is not found in the local submodule, one also needs to search the ultimate parent module for identifiers.

So a correct solution will need to cover both the typeDictionary as well as identifier lookup for data nodes, extension statements, identity statements, and so on.

One approach, would be to build a namespace per module (only performing the dictionary insert on the parent module's identifier namespace dictionary for YANG 1.1 modules) of all 6.2.1 definitions once all initial modules and their submodules have been resolved and loaded into memory, after the top-level identifiers have been resolved. The "brute force" identifier find methods need correction per the approach mentioned above.