Target .Net Standard in Lieu of .Net Framework

[RichardWhitfieldTTW]

Is there a reason why the core libraries of BHoM could not target .Net Standard in lieu of .Net Framework to support operating systems other than Windows?

One would also expect that in the next few years commercial software providers may begin to move away from targeting .Net Framework for their APIs and SDKs as well so this would allow compatibility with both versions of 'full' .Net.

Given that .Net Framework 4.5.2 will go out of support in April 2022 (due to insecurities in the SHA-1 algorithm) it may be prudent to upgrade all assemblies requiring .Net Framework to version 4.7.2 which would allow use of .Net Standard 2.0 for the core libraries (per MS recommendations).

Keen to hear any thoughts.

[FraserGreenroyd]

Hi @RichardWhitfieldTTW we're currently investigating a potential move to .Net Core. We did previously look at a move to .Net Standard but the cons outweighed the pros at that time, but that was a discussion held last year, and things have moved on since then! It's on my roadmap to investigate what a move to .Net Core would do for the UIs and general deployment of the BHoM in known uses, any information you have that would support that would be great to hear 😄

[RichardWhitfieldTTW]

Thanks Fraser, I did a quick test on the BHoM repo where I changed the base BHoM project to netstandard1.2 and it all worked fine in some very basic testing (also took the opportunity to change the csproj to the SDK version which is so much shorter!). Let me know if you want me to push this here and you can take a look.

I think moving to .Net Core (or now just .Net) might be problematic for some of the adapters...for example I have previously tried to use the ETABS API from a .Net Core application and have run into issues with some of the methods in that API (giving System.AccessViolationException which appears to be a known difference between the runtimes).

If we move all the base classes to netstandard (ie DTOs and any of the low level methods which are not using any functions of the windows runtime) then we support applications in both .Net Framework (for legacy APIs) and .Net (Core) for modern development.

Happy to have more of a chat about this if it's of any interest...I'm based in Australia so we would need to sort out the timezones but apart from that it should be all good.

[FraserGreenroyd]

I'd be happy with a draft PR being raised to see what changes you made Richard to get the ball rolling on this 😄

The adapters are definitely a concern, but then maybe moving the BHoM core as you say could work well if the adapters can remain on an older version of .Net. I'm imagining the Environment Adapters (IES, XML) won't suffer this as much as they are mostly doing file interop., while the Structure Adapters (ETABS, GSA) are more likely to. Revit is also the big one, and falls into the similar category as they're doing live model interop. rather than file types as the Environment Adapters are.

April 2022 is only 9 months away (or 3 BHoM development milestones at the current calendar) so definitely not one to sit on. I'll try and draft a plan of action for this next week (I'm away over the weekend unfortunately) and post it here for input to make sure we have a plan to do these investigatory parts and leave time to make the changes before we get to the end of support cycle for 4.5.2 (I'm accepting it's not the end of the world to keep using it after that as it'll still work, but we don't want to be snails I think on any upgrades given everyone is likely to start making an upgrade to .NetStandard or .NetCore soon as the .NetFramework's begin losing their support)

[RichardWhitfieldTTW]

Thanks Fraser,

I'm actually leaving my current job on Thursday to start a business writing software for the AEC industry and was hoping to use BHoM as a base for my models so I have some skin in the game here. From Friday onwards I'll be able to spend a lot more time on this so I will go through the whole BHoM solution and change it out for .Net Standard and make a PR. I ran the portability analyser on the code and the only changes that needed to be made were to the BHoM project itself and that ended up being just changing out a couple of ForEach method calls to a traditional foreach loop.

I'm also glad the BHoM has moved to LGPL as well...I think it's getting big enough and stable enough for people like me to consider it as a base for technology but the contributions to the code should remain in the public domain.

Looking forward to working with you all a bit closer!

[al-fisher]

Hi @RichardWhitfieldTTW - above all sounds great - and exciting news about your new focus. Congratulations!

Also pleased to hear set up, including LGPL etc. is working for you!
Naturally very keen to hear your suggestions for any further improvements and additions across any aspect of the BHoM - so starting any new discussions/PRs really welcome.
Also as you suggest above - always happy to find a time to grab a chat. Can make the times work 😄

[RichardWhitfield]

Hi @FraserGreenroyd I have been through and updated all of the assemblies in the BHoM repo to target netstandard2.0 and updated the .csproj files to be the newer SDK format. On this GitHub account (I have now left TTW) I don't have permission to push this branch to origin, are you able to give me access for this? You can also remove the old TTW account if you like.

Once it's up then I can create a PR if that's your preferred way to review the code...I think that it's likely that you will want some further changes (see below). Also before merging this then all the other projects in the BHoM will need upgrading to Net Framework 4.7.2 to make use of this version of netstandard. I am happy to do this (can look at this immediately), I will also do the same and change the .csproj to be the newer SDK format as I go (it's much cleaner).

As I have been going through and making the changes I have a few queries:

1. With the versioning .json files, these don't seem to copy anywhere for me. I had a quick look through the docs for versioning and it was suggesting that these are copied to the \bin folder but typically BHoM outputs to a ..\Build folder and in any case I can't see these versioning files here. Anyway...long and short of it, I currently have them copying to the %ProgramData%\BHoM\Assemblies folder but this isn't correct since the same named .json file can exist with different content in various projects.
   Not quite sure what the best thing is to do here...open to suggestions.
2. Following point 1 I noticed that in the Structure project that version 40 was missing from the .csproj include item
3. I have typically removed the assemblyinfo.cs file and where relevant the parent Properties folder. If this is needed then I can add this back in as a build instruction. Let me know.

Rich

[FraserGreenroyd]

Hi @RichardWhitfield I'll get the accounts switched over in a moment for you and that should allow the push.

I've been looking at the .NetStandard and .Net5 and suspect we may want to go direct to .Net5 for greater compatibility with the different OS'. My concern at the moment is how the adapter toolkits will react within the UIs, that's the feasibility study I still need to try and investigate (and will need support from @pawelbaran @IsakNaslundBh @alelom and @adecler I suspect). I'm hoping to do this in the next couple of weeks, so I would propose if you raise your PR for us to crowd around and look at, and I'll draw up that plan for how to check how different toolkits react to the change in .Net. I'll try and draw it up tomorrow (I have a bit of other operational stuff to do tomorrow afternoon so will try and roll this into it!) so it's up before the weekend and we can have a couple of clear weeks to work on it as appropriate 😄

[RichardWhitfield]

Cheers @FraserGreenroyd I have pushed this branch so have a look and let me know if you want any changes made.

Regarding .Net 5 I don't think this is a viable option. Because most proprietary software packages still target .Net Framework for their SDKs and these frequently use Windows specific calls to things like the file system you will find that these calls are different in the cross platform .Net 5 (which will result in compile time errors). By all means do investigate yourself (I would like to be wrong about this) but I think that the adapters will be stuck in .Net Framework until the vendors change the SDKs. Also just for information .Net 5 is not an LTS version of .Net so I think if you could use this then you might be better to wait for a few months for .Net 6 to be released.

.Net Standard 2.0 will not adversely affect the compatibility across operating systems (it's a fully portable BCL which is 100% compatible with .Net Core and .New Framework 4.7.2+). It is simply a subset of the BCLs which are common to both .Net Core (and by extension .Net 5/6+) and .Net Framework. In particular for your data transfer objects where you require very few calls to the framework it's a perfect solution for this.

[IsakNaslundBh]

First want to say great discussion going on here!

Can probably try to answer some of your questions above:

1. The management of the versioning files, an moving of them to the correct location, is handled in full by the Versioning_Toolkit postbuild. It gathers all relevant versioning data for the current version being built from all BHoM repositories and concatenates it into a single versioning file for the current version. No post-builds are required for the projects.
2. Following up on 1 as well. After each release has been made, then the concatenated versioning file is locked and stored in the Versioning_Toolkit. This means that the Versioning_40.json file will stop having any functional impact as soon as the 4.0 beta has been released and the BHoM is versioned up to 4.1. Some files have been kept for reference, but no file other than the one matching the current milestone version need to be included in the project.

The above means that one needs to build versioning toolkit after building all other repos to get the latest versioning included, when building the code from source. When running an installer, this is ofc already handled.

Finally, great to have you even more on board @RichardWhitfield ! 😄

[FraserGreenroyd]

Today has run away from me so I'll prioritise this for early next week (Monday is filled with meetings), will keep you updated!

[FraserGreenroyd]

Ok so a bit slower than originally hoped but here we go.

For guidance, the 4.3 milestone ends on the 24th September 2021, with the 5.0 milestone starting on the 27th September 2021, so end of 4.3 can also be read as end of September.

The key bit here is outlined in the first slide, that .Net Standard is unlikely to have enough LTS for me to be happy, but .Net 5 goes out of support 3 months after .Net 6 is released, which is currently projected for November 2021. Thus, I think it makes sense to do the investigatory work with the toolkits on .Net 5 between now and November (on the basis that there are unlikely to be massive changes between .Net 5 and 6 that would impact our investigations from looking at the specs.) and then upgrade straight to .Net 6 in November.

Thoughts? Also tagging in @IsakNaslundBh @pawelbaran @peterjamesnugent as named individuals on the slides to be aware of.

[RichardWhitfield]

I had a further thought on this...with particular reference to the BHoM core repo it might be best to multi target different frameworks.

If we then make the primary way of consuming the BHoM objects as a NuGet package then NuGet will resolve the framework issue for us depending on the solution that it's being used in.

[FraserGreenroyd]

All of the investigations on this end have concluded to the same result @RichardWhitfield - so if nothing else, a consistent result for us all!

As such, I am going to agree with you on going for .NetStandard for BHoM at the very least, with an aim to upgrade as many of the BHoM_Engine projects to .NetStandard as well for consumption by .NetFramework and .Net5. This could give flexibility on the core objects and object methods - though if there becomes problems with doing that for the engine then we'll just do the BHoM repo itself for the objects.

I'm currently against doing multiple frameworks just due to the potential maintenance but could be persuaded if we have strong enough use cases.

Therefore an update to the above mentioned plan will be just to go to .NetStandard for BHoM DLLs, which I'll aim for us doing w/c 27th September. We're going for a new beta next Friday (24th September), so am against doing anything in the next 7 days that might impact that.

I also want to update the default branches that week as well so there should hopefully be a low amount of other changes happening across the organisation to try and balance. I'll get an issue raised now to capture it on the planning boards 😄

[RichardWhitfield]

Cheers Fraser...nice to know I'm not insane 🧠. I agree regarding multi-targetting...it would be my least preferred approach.

Ran the portability analyser over the engine repo and most are pretty well supported (see below)...I had a quick look at the Analytical Engine as an example and I just had to add the NuGet package for Microsoft.CSharp and it would compile (for using dynamic types) so it looks to be relatively straightforward. Not sure about the speech engine though...

[FraserGreenroyd]

I think Speech_Engine may be legacy machine learning related work, I might bump it down from BHoM_Engine to MachineLearning_Toolkit if it causes us a problem with the upgrade, as otherwise the porting to .Net Standard 2.0 with platform extensions looks like it could work swimmingly for us 😄

[RichardWhitfield]

Hi @FraserGreenroyd, one more thing which might warrant it's own topic but it also kind of related. Is there any interest in moving the core objects from 'Class' type objects to 'Record' type objects to better ensure immutability?
I like the idea of having immutability within the objects, as BHoM promotes, but since .Net Standard 2.0 doesn't actually support C#9 (I think it's 7.3) you have to override the LangVersion in the .csproj file. Record types are not actually anything to do with the language per-se but are actually a compiler feature so .Net Standard will support them perfectly well if you do this. It would also bring the benefit of nullable reference types from C# 8 into play if you want them enabled as well.
The downsides are as below:

• You need a newer version of the Roslyn compiler than the version of .Net suggests - don't see this as a huge issue, in VS it's been supported since 16.8
• Some work is needed to enable many of the other features since C# 7.3 but with somewhat limited work a lot of these can be enabled. However there are some features that are not able to be supported which if used would throw errors.

If I was making and maintaining a class library I would 100% make this change...but I'm not as convinced when it's a relatively large open source project such as this. Would be interested to know your thoughts.

Below is a link to a good post on backward compatibility of these features for .Net Standard 2.0 projects. Let me know if you want me to run any tests.

https://sergiopedri.medium.com/enabling-and-using-c-9-features-on-older-and-unsupported-runtimes-ce384d8debb

[FraserGreenroyd]

Ok so we've updated BHoM, BHoM_Engine, and BHoM_Adapter to now run on .Net Standard 2.0, and all toolkits included within the installer to run on .Net Framework 4.7.2.

Following this, compliance guidelines are updated (or will be very shortly) to say that .Net Framework 4.7.2 is the bare minimum framework a toolkit should be running on, with ideally toolkits running on .Net Standard 2.0 or higher if they can.

I would say the record discussion should be it's own topic @RichardWhitfield if that's ok, just so we can close out this line of enquiry 😄

All of the updates will be released in the 5.0 beta in December, so plenty of time to do tests and update further if needed, but I'll mark this as the 'answer' to this discussion and get ready for the next bits 😄