Rethinking about LSP

[liushuyu]

Background

I think it is time to rethink about LSP, the LMMS sharing platform.

The code of it was an ancient piece of history from the late 90s. And some of the coding conventions are no longer best practices today.

UI/UX Redesign

As time went by, design patterns changed a lot during these years. Although we have redesigned the main page and adapted the LSP website accordingly, the design is not so intuitive to the people who are familiar with "more modern websites."

Technical Practices/Debts

The LSP is a disaster in this field.

• The hash of the password is insecure (prone to multiple attacks, including but not limited to LUT attack (aka Rainbow Table Attack), length-extension attack, collision attack (demoed by Google), etc.)
• The database didn't take advantage of foreign key constraint or any other constraint checks to prevent data inconsistencies at the DBMS level
• The file drop function just making the files laying on the disk wasting inodes (I am very sure the problem won't occur!)
• The HTML structures are concatenated together, making it easy for attackers (some of the contents might not be correctly escaped) and making it challenging to translate)
• The LSP isn't Unicode aware, data is stored in Latin-1 encoding in the database and breaks when there are multibyte sequences (like CJKV and emojis)
• Can't be easily patched to have things like avatars (Enhancement: Ability to have profile avatars on the LSP. #255) or upvote-based rating system (Change the star rating system to an upvote based system #246)
• Anti-spam is a joke, a basic OCR engine can decode the security code correctly, and there are already spamming incidents in the past
• ...And much other nitty-gritty from both users and devs

Possible Improvements

Okay, enough whining now, so here are the ideas I have so far:

• Implement LSP in a different language than PHP (like Python (Falcon/Flask/bottle/Django/aiohttp/...), Ruby (Sinatra/RoR/...), Rust (actix-web/hyper.rs/...), NodeJS (ExpressJS/...), etc.) one of the reason is humans are good at reusing patterns, so this is to avoid that.
• Use a separate NoSQL database (k=v database) like LevelDB, RockDB or Redis to store the files; this can reduce the disk I/O, inode usage and presumably boost the speed. Some of these DB comes with builtin compression mechanisms.
• Migrate to probably MariaDB (MySQL compatible written by former MySQL developers) or PostgreSQL for the database, and they offer better optimization in our use-case.
• Redesign the UI and use probably specific frameworks to create a single page PWA to speed up page-load and overall smooth experience, an alternate design can be deployed side-by-side for people who disabled or don't have Javascript and CSS support

[tresf]

Although security is a concern, the platform itself could also benefit from the most basic of usability improvements. For example:

• The logins could be tied to another service (such as OAuth, forums, etc)
• A "report" option for people abusing the platform
• A way to track accreditation and history by having a unique marker inside the track
• Some form of intelligent spam title sanitization. e.g. discourage or ban 🎵🎵🎵SUPER DUBSTEP DROP DJ MIKE BEST REMIX EVER!!!🎵🎵🎵 versus Dubstep Forest Part I. May need to leverage spamassassin-type spam filtering.
• Rate limiting on voting
• Easier playback (either through bulk downloads or server-side preview rendering)

The stuff you mention is nice and will make the platform stronger, but I'd also love to see a place that encourages creativity by the musicians.

[liushuyu]

Although security is a concern, the platform itself could also benefit from the most basic of usability improvements. For example:

I have actually considered all of those features and functions, but the problem is, under the current codebase, it is difficult to implement.

For instance:

• OAuth: Instead of string matching credentials, now we need to change the login page to match tokens as well, also database schema needs to change again to store OAuth tokens
• Report: Obviously you need to store the records somewhere, database changes again
• Version tracking: Database change, and, this will exaggerate the issue with the filesystem since the files will be even more sparse
• Anti-spam: You are describing a Bayesian filter, this thing is actually computational intensive and requires training the model (get optimal statistical weights for our LSP) to identify the sentiment (whether it is a spam or not)
• The bulk download is probably going to implement easily but this I think may require rate-limiting otherwise someone could hog the disk I/O of the server the whole day

The stuff you mention is nice and will make the platform stronger, but I'd also love to see a place that encourages creativity by the musicians.

Well, making the platform stronger, faster and more secure can indeed encourages more creative works. I can't see there is any contradiction.

[liushuyu]

We could learn from the platforms like CurseForge and Nexus Mod, they have something in common with LSP: hosts user generated contents, rating, comments, search function, and they all have version tracking.

[tresf]

We could learn from the platforms

Was thinking the same thing. Modify a platform that offers everything to behave like LSP instead of trying to reinvent each component by hand.

[liushuyu]

Was thinking the same thing. Modify a platform that offers everything to behave like LSP instead of trying to reinvent each component by hand.

Unfortunately, I wasn't able to find their source code and it seems like they are PaaS solutions. Other frameworks or solutions are based on "playlists" and they only host audio files.

It seems like other solutions are homebrewed.

If anyone has better solutions, just post it here.

[GlowingUmbreon]

If you are looking for a platform which has public source code and is already similar to LMMS there is the BeatSaver website:
https://beatsaver.com/
https://github.com/lolPants/beatsaver-reloaded
It is written in typescript

[RiedleroD]

This is just a personal request, but I'd like to have the lsp work without js. This is just for performance reasons since loading with js is two extra steps, and generally slows down the website. A good example of a website being slowed down by js being js would be the whole youtube ecosystem, but most noticably youtube studio and youtube music. Looking at the network inspector, on average, there's more js code getting downloaded than any other media type, which is dreadful.

[tresf]

This is just a personal request, but I'd like to have the lsp work without js.

I think most solutions (and thus those proposed by @liushuyu) are/will be predominantly back-end-driven and won't be too heavy on client-side JS unless the framework mandates it (and in that case, is up to the developer that writes it).

If a 100% "NoScript" site is desired, please make sure you're part of the testing phase so that those considerations can be made. Personally, I find the "NoScript" argument to be a relic of the 90s, but it shouldn't be too hard to guard against if it's pointed out early enough.

[RiedleroD]

If a 100% "NoScript" site is desired, […]

My biggest concern is the load and processing times, so no js at all isn't really necessary imo. The most healthy amount of javascript for a website is in my opinion just for the things that absolutely need js (e.g. client side password strength checking), so that the website can be used without js, but the experience is enhanced if js is enabled. Something like that. Ofc I'm not an expierenced webdev, just a internet user with opinions. But I think it's worth keeping that in mind.

Also: woo my js-free website loads as fast as a lightning, take that, youtube. :P

Sorry for cluttering this thread if that's not important though.

[tresf]

just a internet user with opinions

We always need testers. As far as I know this hasn't started yet. @liushuyu has pretty much taken over website maintenance at this point so if that continues they'll be the one posting updates to this thread.