[v1.5.3] User manual utility and file manager bug fix

Code blocks on the in-app version of the user manual now have a new button that allows users to save that code snippet on disk. This change was made because the button to copy values to the clipboard is not working properly for all users. I still have to investigate the problem to know the cause, but it looks like it is not on Nodezator's source, since it works on some machines and not on others.

To avoid hassle to the users (one of them reported having to type the code manually), this new button now allows them to save the code directly on disk. Of course, alternatively, the users always have the online version of the manual available as well.

I also fixed a bug on the file manager that sometimes caused paths to still be submitted despite the user having pressed the Cancel button. Naturally, this was a clearly undesired behaviour that could cause all kinds of undesired side-effects as well.

[v1.5.2] Bug fix for nodes with keyword-variable parameters

I usually only write a summary of the changes here, detailing them on the commit messages themselves. However, since this was a pretty quick and simple fix, I'll just paste the commit message itself here:

Nodes with keyword-variable parameters (parameters prefixed with **) were being executed in the wrong way due to a bug that caused their arguments to be discarded when more than 01 variable keyword argument was collected.

This nasty bug could either cause all kinds of errors when executing the graph or even worse remain undetected, all due to the unexpected outputs that such nodes produce by ignoring the variable keyword arguments.

It was introduced in v1.5.0, when the entire graph execution mechanism was overhauled in order to adopt the topological sorting algorithm.

We also updated social media links to include the developer's (me) Bluesky profile and also point to the developer's links page, where several other useful links/addresses are available, including email.

[v1.5.1] Usability improvements and content update

This is an impromptu patch that came to be because of some experimentation I did with a solution to make it easier to interact with sockets using the mouse to establish new connections. I call this solution socket proximity detection. It consists in detecting nearby sockets whenever:

• clicking near them in order to establish a new connection
• releasing the mouse dragging the new connection near the other socket

In other words, now doing both things is easier, because the mouse doesn't need to be exactly over the sockets for any of those actions to take place.

Additionally, special graphics/shapes are drawn on the screen when dragging a new connection to another socket in order to signal the detection to the user.

For instance, here's what it looks like when the socket detection graphics are "reaching hands":

In the image, the mouse cursor becomes a hand that extends from the first socket. The second socket is detected due to its proximity, so a hand also extends from it to meet the hand cursor. Granted, a static image doesn't do justice to the usefulness of the feature nor to how smooth the movement is.

There's also no risk of the wrong socket connecting by mistake by moving the mouse accidentally close to another socket because the connection is only established when the mouse button is released, so you can move the cursor freely and only release it when there's visual confirmation that the socket you intend to connect to was properly detected and is in range (for this particular graphics used, we know the socket is in range when the hands hold each other at the cursor).

The user can even customize the experience by:

• picking one of the following graphics to be used:
  • assisting line
  • reaching hands
  • hands and eyes
  • baseball elements
  • baseball elements and eyes
• setting the distance at which a target socket is detected when the mouse is dragging a new connection
• setting the distance at which a connection can be established with the detected socket

A few key content pieces were update as well. The first chapter of the user manual now has a new subsection on sharing resources among nodes. A very useful and long needed resource. In addition to that, a few other parts of the chapter were improved for clarity.

The README file of the repository is much more informative now and makes usage of more images and animated GIFs to showcase Nodezator features and capabilities. Like the first chapter of the manual, the README file also had several parts of its text improved for clarity.

We also made sure to include important information and warnings to avoid common misconceptions people tend to have when learning about Nodezator for the first time. For instance, some people seem to have a hard time telling whether Nodezator is an application or a framework/GUI library at first, so in the text we made sure to emphasize what Nodezator is and what it is not.

Finally, made a few small changes to the menu subpackage to make it easier to use widgets like checkbuttons and radiobuttons on the menus. The menu subpackage always had such widgets available, but we never needed to use them before. Now we use radiobuttons to allow users to choose their preferred socket detection graphics, so it was a great opportunity to improve their back end a bit. The appearance of the radiobuttons was improved as well. They are now rendered from SVG text.

[v1.5.0] System testing, topological sorting and QOL features

The scheduled work for this release consisted of 02 features, but we delivered additional QOL (quality-of-life) features and other changes as well. The scheduled features were:

• Finishing and releasing the automated system testing service
• Replacing the makeshift graph execution algorithm by topological sorting

Among the additional QOL features, we highlight:

• Birds eye view
• The ability to jump to any node in the graph by providing its id

The remaining changes will be presented futher ahead, in a dedicated section.

In addition to the information presented in this release text, you can find much of the development process documented in this GitHub discussion: #72. It was used as a development log (devlog) detailing changes and related decisions.

New automated system testing service

This release was focused on providing a new key feature to aid in development of Nodezator: an automated system testing service.

Nodezator, being a node editor, is a software with many features, tools and a varied workflow. There are innumerable actions that users can take and underlying systems supporting it. Additionally, being an open-source software project, it is subject to external development work being merged into it. Even if there was only my own work being merged into it, the use-cases/systems are just too many to be manually tested by a single person.

In the past, despite my careful testing hundreds of times, I still couldn't manually test every possible action and it resulted in some nasty bugs being introduced in some past releases (bugs that were already solved). Although we also have unit tests, the coverage is still low and, even if it was high, the tests would only be able to ensure the isolated units work. This means we needed a tool that could test the entire system, ensuring every part works in tandem to perform each and every task the user can perform in Nodezator.

In other words, we needed a tool that ensured that, given the same actions performed by a user like clicking some element, typing something, pressing specific keys and key combinations, in order to perform a specific task, the app would be able to successfully perform that task, resulting in specific outputs and/or state.

This tool is called System testing. At the beginning, because it relies on GUI automation, we were mistakenly calling it automated GUI testing. However, because we are interested in testing the entire system, not only the graphical user interface, system testing is indeed the right term.

Performing system tests

Since version 1.4, Nodezator featured a Playback submenu on the menubar, although none of its options were available to users. In 1.5, this submenu was renamed to GUI automation and its Automated GUI tests option was renamed to System testing. This System testing submenu has all the options needed to set and perform an automated system testing session, with some keybindings assigned to the most common ones. Here's an image depicting the GUI automation submenu as well as the options available in the System testing submenu inside it:

Among the options within the System testing submenu, there's one that presents a form where a custom testing session can be set and triggered. The form is presented in the image below:

The form can be used to pick which test cases must be executed and the speed used. There's an speed option because although most people will usually want the tests to run in top speed, it might be useful to run specific tests in normal or slower speed in order to careful observe the execution of the task to help spot possible errors/mistakes.

Once an automated system testing session begins, the app starts moving by itself, executing each task. During the session, there are several controls that can be used to perform tasks like pausing/resuming the session or aborting it. You can check such controls in the general controls page (that you can access in the Help > General controls menubar option).

Once a system testing session ends, a report is presented. Here's what the initial portion of the report looks like:

The actual report is much larger since it has details about the system and each test case executed.

On the bottom of the report there are also buttons to export it to other useful formats like .html, .png and .pyl (a Python file containing only literals).

Remaining work for system testing and Nodezator in general

All that remains is to populate the system testing service with all the needed test cases, which is expected to be done in the next patch or so.

In order to aid in development of not only of the system testing feature, but the whole app, a holistic view of its design is needed. Each and every task and feature must be listed. This is specially important for system testing because we need such information in order to define and implement all the needed test cases.

Because of that, since the beginning of our scheduled work for this version, we also started producing a Software Design Document (SDD) for Nodezator, which you can find in this dedicated repository: https://github.com/IndiePython/nodezator-sdd

A system testing document is also being produced. It is kept in a separate repository although it is considered part of Nodezator's SDD: https://github.com/IndiePython/nodezator-system-testing. It is used to help manage system testing for Nodezator and to document its test cases. It is kept as a separate repository so changes to system testing can be managed and tracked separately from the rest of the SDD.

Finishing those documents is important not only for system testing, but also for Nodezator in general. For instance, once these documents are ready, we'll be able to organize the missing features in a sensible order for scheduling and implementation. We'll also be able to present a proper roadmap to users of the app and followers of the project.

Breadth-first-search-based topological sorting for graph execution

Replacing our makeshift graph execution solution by one based on the topological sorting algorithm/technique was a feature request made by Mr. Alexander and accepted after considering the merits of the request.

Although other materials on the topic were researched, the most concise and straightforward definition for the topological sorting technique/algorithm was found on Quora (A Questions & Answers platform):

Topological sort is a fundamental technique in graph theory that organizes the nodes of a directed acyclic graph (DAG) while maintaining the edge direction. The significance of topological sort stems from its ability to offer a linear ordering of nodes, which may be used to address a range of issues such as scheduling, job sequencing, and dependency resolution. (source)

In summary, topological sorting has many applications in computer science and software engineering and is an established key technique to solve many different kinds of problems. The specific algorithm picked was one based on a breadth-first search algorithm and also recommended by Mr. Alexander. This means nodes which don't require inputs are listed first, gathered in groups called generations, then the nodes immediately fed by their outputs are also listed as their own generation and so on until all the existing nodes are listed.

It should be particularly useful to us in the future when we implement parallel execution, since the algorithm groups together nodes which don't depend on each other and can thus be executed in parallel.

We implemented a custom solution adapted from this pseudocode and with insight from other researched materials. Our solution presented the same output but superior speed to an alternative solution using an established 3rd-party library for manipulating graphs, the NetworkX library. Specifically, this NetworkX tool was used.

For the record, we don't fault NetworkX for its inferior speed in this specific case, since, being a library, we assume its offers a general solution designed to meet the requirements of many environments, whereas our custom solution is free to take advantage of local services and existing data structures to achieve top performance. NetworkX actually helped us validate our custom solution, that is, because NetworkX is a renowned library and our custom solution produced the same output as its solution.

In the distant future we intend to research NetworkX further, as well as other graph manipulation libraries like igraph. This would be done in order to learn even more about our problem ...

[v1.4.6] More additions and a bug fix

Among the many changes/additions for this patch release, we have:

• code contributed by Mr. @sefgit via pull request, that consists of:
  • mousewheel pan feature
  • the addition of a .gitattributes file
• code contributed by me (@KennedyRichard):
  • the ability to confirm/dismiss/navigate the dialog using the keyboard
  • a bug fix for a dialog that was growing past the screen

The mousewheel pan feature is the ability to pan/move the graph with the mousewheel. The .gitattributes file added is a simple text file that is used to define attributes to paths. Certain operations by Git can be influenced by assigning particular attributes to paths, which is what this file is used for. Some additional changes already discussed with the contributor were also made by me.

I also added/set the ability to confirm/dismiss dialogs with the keyboard using the Enter and Escape keys, respectively (though, often, confirming will also dismiss the dialog when it is the only action available). I use the terms "added/set" because part of the feature was already implemented and just needed to be properly set in the existing dialogs defined throughout the source. Naturally, some dialogs may not offer the option to be so casually dismissed like that when the consequences of the action cannot be reversed. Currently, though, we don't have any dialog like that.

The user can now also navigate the available dialog buttons/options using arrow keys or the Tab and Shift+Tab keys.

The bug fix consisted in preventing a specific dialog from growing past the screen.

In this specific bug, we were allowing text from error reports to be displayed on the dialog. The right practice, however, would be to just use known predefined text and record the error report in a more appropriate place for the user to check. Text from error reports can grow a lot when there are too many errors, so we must not use such text in dialogs. That's what was causing the problem.

Now, instead of reporting the error(s) on the dialog, we just display an informative small message and log the errors in both the default logger and our custom user logger. This way the user can check the error report using the log files or the user log that can be accessed on the app's GUI. The user log can be accessed by pressing Ctrl+Shift+j when you are on the graph/canvas or by selecting the Help > Show user log option on the menubar.

[v1.4.5] Additions, improvements and fixes to tests

New unit tests for the function used to load nodes were provided by Mr. @BMaxV via pull request. I @KennedyRichard also refactored and complemented them.

I also used the opportunity to make some small improvements in the doctests and to fix one of them.

In order to make it more convenient to store test data, .zip files found in Nodezator's source are now also included in the source distribution. That is, they are downloaded in conjunction with the Python modules when someone installs Nodezator.

[v1.4.4] Fixed bugs in SVG exporting and change checking

The bug in SVG exporting was fixed by removing unused code. The unused code was causing a name error because it was made from repurposed pre-existing code and was trying to access a variable that didn't exist in the new spot. Since HTML exporting relies on SVG exporting, HTML exporting was also being affected.

Some other lines that weren't causing errors but were not needed either were also removed to avoid confusion and needless maintenance.

The other bug fixed is related to change checking, that is, checking whether a change in a widget's value took place. Change checking was causing errors due to missing type checks. Such type checks were missing in different modules in the source and are now implemented.

More specifically, when a number changed, if the value was the same, even though the type was different, that change wasn't being treated properly, because the app was only checking the value and reaching the conclusion that nothing changed, because the value was the same in the end. However, this is not the right thing to do. That is, we must also take the type into account, because numeric values may have the same value and still have different types.

For instance, 0, 0.0 and 0j all have the same value, although they are an integer, a float and a complex number respectively. Because of that, wherever a value was being checked to determine whether a change took place, we also added a type check so this kind of change is also taken into account.

[v1.4.3] Fixed error when executing operation nodes

The error was caused by using the wrong attribute when retrieving the existing sockets from an operation node. Using the correct attribute name, by including an if/else clause that picked the appropriate attribute when needed, fixed the error.

The error was causing execution of the graph to fail altogether whenever operation nodes were present.

[v1.4.2] Fixed error when executing collapsed nodes

The error was caused by using the wrong attribute when retrieving the existing sockets from a collapsed node. Using the correct attribute name fixed the error.

The error was causing the output of collapsed nodes to be handled in the wrong way because it wasn't able to accurately determine the number of existing output sockets. This problem could cause all sorts of other problems, like the graph failing to execute or the wrong data being passed along to other nodes.

[v1.4.1] Tiny packaging fix

This tiny fix consists of the addition of a missing empty __init__.py file for packaging purposes. The absence of this file was causing some modules to be ommited in the pip installation, making the app crash on launch due to the missing modules.