Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add HUB75 support #3777

Open
wants to merge 35 commits into
base: 0_15
Choose a base branch
from
Open

Add HUB75 support #3777

wants to merge 35 commits into from

Conversation

netmindz
Copy link
Collaborator

@netmindz netmindz commented Feb 26, 2024

Add support to drive HUB75-compatible LED matrix panels. It's a great way to get started with 2D effects in WLED at very low prices.

@netmindz netmindz marked this pull request as ready for review February 27, 2024 02:09
@robertlipe
Copy link

(I'm not a WLED maintainer. My comments mean nothing.)

Wow! I'm shocked at how cleanly that slots in. Nice. I have both a Matrix S3 and a Mesmerizer (nightdriverled.com) so I may try spinning a build. What kind of frame rates are you seeing on the 2D animations? (We have a lot of the same effects in NightDriver and I've flirted with using the Faptastic HUB75 layer, too...)

Why are ESP32_FORUM_PINOUT, ARDUINO_ADAFRUIT_MATRIXPORTAL_ESP32S3, etc. compile-time flags (and not defined in a derivation of esp32dev_hub75) and not runtme-selectable flags like APA102 vs. WS2814 vs WS2812, etc.?

Given how differently -S3 and -Nothing have to be built, can you really rely on board = esp32dev? Or is this a "draw the rest of the owl" that needs to have another custom_platform_mps3.ini written that derives from an appropriate S3 with the 8MB flash, 2MB PSRAM combination? (https://github.com/platformio/platform-espressif32/blob/master/boards/adafruit_matrixportal_esp32s3.json ?) Do you need help with that, perhaps?

Would this be likely to plan nicely with the Virtual Display (chained panels - limited only by color depth and refresh rates) and UDP/DDP inputs and such?

You can see I'm pretty excited to see this as I've traveled a lot of similar paths. I'm a bit engaged in the short term, but may be able to help land this if you need extra eyes and fingers on it.

Copy link
Collaborator

@blazoncek blazoncek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this already looks great, though I have some remarks.

The most important:

  • no modifications in platformio.ini
  • if possible avoid compile time differentiation

Unfortunately I do not own Hub75 panels but I'll try to work on some enhacnements using logic.

platformio.ini Outdated
@@ -461,3 +461,19 @@ build_flags = ${common.build_flags} ${esp32s2.build_flags} -D WLED_RELEASE_NAME=
-D HW_PIN_MISOSPI=9
; -D STATUSLED=15
lib_deps = ${esp32s2.lib_deps}


[env:esp32dev_hub75]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will require this to be moved into sample file. I want platformio.ini clean and only used for default official builds.

wled00/bus_manager.cpp Show resolved Hide resolved
wled00/bus_manager.cpp Outdated Show resolved Hide resolved
wled00/const.h Outdated Show resolved Hide resolved
wled00/data/settings_leds.htm Outdated Show resolved Hide resolved
@robertlipe
Copy link

robertlipe commented Feb 27, 2024 via email

@blazoncek
Copy link
Collaborator

Also, if you have PSRAM (and what new ESP32 doesn't these days?) are you really limited to one panel?

PSRAM is slow. Too slow for anything LED related.

@robertlipe
Copy link

robertlipe commented Feb 27, 2024 via email

@netmindz
Copy link
Collaborator Author

Wow! I'm shocked at how cleanly that slots in. Nice.
Thanks :)

I have both a Matrix S3 and a Mesmerizer (nightdriverled.com) so I may try spinning a build. What kind of frame rates are you seeing on the 2D animations? (We have a lot of the same effects in NightDriver and I've flirted with using the Faptastic HUB75 layer, too...)

I'm having real issues with wifi with the Matrix S3, so will be interesting to see if you see the same.

Why are ESP32_FORUM_PINOUT, ARDUINO_ADAFRUIT_MATRIXPORTAL_ESP32S3, etc. compile-time flags (and not defined in a derivation of esp32dev_hub75) and not runtme-selectable flags like APA102 vs. WS2814 vs WS2812, etc.?

If you are using the matrix portal, you can't change the pins, so no point asking the user to enter 14 pins. For other boards, just lazy ;)

Given how differently -S3 and -Nothing have to be built, can you really rely on board = esp32dev? Or is this a "draw the rest of the owl" that needs to have another custom_platform_mps3.ini written that derives from an appropriate S3 with the 8MB flash, 2MB PSRAM combination? (https://github.com/platformio/platform-espressif32/blob/master/boards/adafruit_matrixportal_esp32s3.json ?) Do you need help with that, perhaps?

The example env just shows what options you need to enable this feature. If you aren't using a regular esp32 then you need to add the extra options to the env for your board

Would this be likely to plan nicely with the Virtual Display (chained panels - limited only by color depth and refresh rates) and UDP/DDP inputs and such?

In theory yes you could use DDP then Hub75 output or mix of Hub75 and other output types

You can see I'm pretty excited to see this as I've traveled a lot of similar paths. I'm a bit engaged in the short term, but may be able to help land this if you need extra eyes and fingers on it.

Great

@netmindz
Copy link
Collaborator Author

Also, if you have PSRAM (and what new ESP32 doesn't these days?) are you
really limited to one panel?

You are not limited to single panels. At the moment it will allow you to select up to 4 panels of 32x32 in horizontal configuration. We can add other layouts later

The ESP32-S3s always have octal PSRAM and can
DMA to and from it

The library being used does allow PSRAM use

@robertlipe
Copy link

Rats. The email gateway ate my response:

I finally tried to check this tree out and build it for MatrixS3. I may fumble with WLED and GitHub, but I'm not an inexperienced dev, so please stick with me.

$ gh repo clone netmindz/WLED hub75-wled
Branches didn't come across, so I did
$ git checkout 421c470
and confirmed the HUB75 changes were in the git log.

$ pio run -e adafruit_matrixportal_s3_hub75

npm run build
[ ... ]

wled@0.15.0-b1 build
node tools/cdata.js

node:internal/modules/cjs/loader:1146
throw err;
^

Error: Cannot find module 'inliner'
Require stack:

  • /Users/robertlipe/src/nightdriverstrip/hub75-wled/tools/cdata.js
    at Module._resolveFilename (node:internal/modules/cjs/loader:1143:15)
    at Module._load (node:internal/modules/cjs/loader:984:27)
    at Module.require (node:internal/modules/cjs/loader:1234:19)
    at require (node:internal/modules/helpers:176:18)
    at Object. (/Users/robertlipe/src/nightdriverstrip/hub75-wled/tools/cdata.js:20:17)
    at Module._compile (node:internal/modules/cjs/loader:1375:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1434:10)
    at Module.load (node:internal/modules/cjs/loader:1206:32)
    at Module._load (node:internal/modules/cjs/loader:1022:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:142:12) {
    code: 'MODULE_NOT_FOUND',
    requireStack: [
    '/Users/robertlipe/src/nightdriverstrip/hub75-wled/tools/cdata.js'
    ]
    }

I don't do Node, so I'm stumped on that one. (I have built WLED directly for another S3 board from teh main repo about a month ago...)

If I restart it, it starts to chow down on the CPP code:
Compiling .pio/build/adafruit_matrixportal_s3_hub75/lib7c9/FastLED/FastLED.cpp.o
Compiling .pio/build/adafruit_matrixportal_s3_hub75/lib7c9/FastLED/hsv2rgb.cpp.o
: warning: "ARDUINO_USB_CDC_ON_BOOT" redefined
: note: this is the location of the previous definition
In file included from /Users/robertlipe/.platformio/packages/toolchain-xtensa-esp32s3/xtensa-esp32s3-elf/sys-include/sys/reent.h:503,
from /Users/robertlipe/.platformio/packages/framework-arduinoespressif32/tools/sdk/esp32s3/include/newlib/platform_include/sys/reent.h:17,
from /Users/robertlipe/.platformio/packages/toolchain-xtensa-esp32s3/xtensa-esp32s3-elf/sys-include/stdio.h:60,
from /Users/robertlipe/.platformio/packages/framework-arduinoespressif32/cores/esp32/esp32-hal.h:25,
from .pio/libdeps/adafruit_matrixportal_s3_hub75/FastLED/src/platforms/esp/32/led_sysdefs_esp32.h:2,
from .pio/libdeps/adafruit_matrixportal_s3_hub75/FastLED/src/led_sysdefs.h:42,
from .pio/libdeps/adafruit_matrixportal_s3_hub75/FastLED/src/FastLED.h:51,
from .pio/libdeps/adafruit_matrixportal_s3_hub75/FastLED/src/bitswap.cpp:7:
/Users/robertlipe/.platformio/packages/framework-arduinoespressif32/tools/sdk/esp32s3/include/newlib/platform_include/assert.h:20:10: fatal error: sdkconfig.h: No such file or directory

I agree. There is an assert.h there and it's trying to include an <sdkconfig.h>, but none is in the system header path.

If I look at all the sdkconfig.h's on my entire system, I think the pattern is "projects built with idf.py, not platformio", but I'm not certain of that pattern.

If I change strategies to:
$ pio run -e esp32s3dev_8MB_PSRAM_qspi
I still get the failure in node.js

and the C++ compilles until it gets to
wled00/wled_server.cpp:3:10: fatal error: html_ui.h: No such file or directory
The only html_ui.h I have is in that aforementioned wled build directory, which makes me suspect this is a generated file. I can't help but wonder if it's generated in a step that comes after that nodejs failure.

But that same assert.h is still trying to include a sdkconfig.h that still doesn't exist, so there must be something else going on. I suspect we're getting the wrong assert.h

I see adafruit_matrixportal_esp32s3.json in .platformio/platforms/espressif32/boards so I don't think this board is too new to be supported by platformio, but it would be totally on brand for them to not print "something not found" and just blunder on with the wrong one if they did. (My relationship with PlatformIO is strained.) OTOH, it's clearly finding at JSON because it's expanding out the full name:
PLATFORM: Espressif 32 (6.5.0) > Adafruit MatrixPortal ESP32-S3
HARDWARE: ESP32S3 240MHz, 320KB RAM, 8MB Flash

So I guess this is a three part question:

  1. What's the solution to the Error: Cannot find module 'inliner' ?
    1a) Is this what's preventing html_ui.h from being built? (Assuming it's generated at all since it's no in my checkout)
  2. Is there something else required to build the adafruit_matrixportal_s3_hub75 target? Maybe something magic in Platformio or arduino32?

As a sanity check $pio run -e esp32dev_hub75 has the same issue with html_ui.h and with node, but does find a happy assert.h.

So I think I just have startup blues. Should this be expected to build from this basic recipe? Is this perhaps divergence from the two wled trees?

Thanx for the hand.

@netmindz
Copy link
Collaborator Author

netmindz commented Mar 5, 2024

The simple option @robertlipe would just be to use this ready-made bin

https://github.com/netmindz/WLED/actions/runs/8056668690/artifacts/1276931005

@netmindz
Copy link
Collaborator Author

netmindz commented Mar 5, 2024

If you are trying to build from source, you need to run

npm install

To have the right modules installed needed for inliner

@robertlipe
Copy link

robertlipe commented Mar 5, 2024 via email

@netmindz
Copy link
Collaborator Author

netmindz commented Mar 5, 2024

@robertlipe - I suspect the wrong version of the platform is being used. We have the option to build on V4, which is what I use for the MoonModules#108

@lboue
Copy link

lboue commented Mar 10, 2024

Thanks @netmindz. I also have Adafruit MatrixPortal S3 board. I will try this.

@lboue
Copy link

lboue commented Mar 10, 2024

The simple option @robertlipe would just be to use this ready-made bin

https://github.com/netmindz/WLED/actions/runs/8056668690/artifacts/1276931005

Should I use the following command to flash Adafruit MatrixPortal S3 board?

esptool.py write_flash 0x10000 adafruit_matrixportal_esp32s3.bin

@netmindz
Copy link
Collaborator Author

The simple option @robertlipe would just be to use this ready-made bin

https://github.com/netmindz/WLED/actions/runs/8056668690/artifacts/1276931005

Should I use the following command to flash Adafruit MatrixPortal S3 board?

esptool.py write_flash 0x10000 adafruit_matrixportal_esp32s3.bin

Not sure about that, I just tend to goto the manual ota page of an existing WLED install and upload the bin

@lboue
Copy link

lboue commented Mar 10, 2024

Not sure about that, I just tend to goto the manual ota page of an existing WLED install and upload the bin

Which firmware file did you use for WLED install before going to the manual ota page and upload your HUB75 version?
https://github.com/Aircoookie/WLED/releases/tag/v0.14.1

  • WLED_0.14.1_ESP32-S3_16MB_PSRAM_opi.bin
  • WLED_0.14.1_ESP32-S3_8MB.bin
  • WLED_0.14.1_ESP32-S3_8MB_PSRAM_opi.bin
  • WLED_0.14.1_ESP32-S3_8MB_PSRAM_qspi.bin

@netmindz
Copy link
Collaborator Author

Not sure about that, I just tend to goto the manual ota page of an existing WLED install and upload the bin

Which firmware file did you use for WLED install before going to the manual ota page and upload your HUB75 version? https://github.com/Aircoookie/WLED/releases/tag/v0.14.1

  • WLED_0.14.1_ESP32-S3_16MB_PSRAM_opi.bin
  • WLED_0.14.1_ESP32-S3_8MB.bin
  • WLED_0.14.1_ESP32-S3_8MB_PSRAM_opi.bin
  • WLED_0.14.1_ESP32-S3_8MB_PSRAM_qspi.bin

Ah, for the matrix3 I didn't actually try a regular install, just went directly to building from source. Does the web installer give a basic working setup? https://install.wled.me/

@lboue
Copy link

lboue commented Apr 14, 2024

@netmindz
Any news about this PR?

@Linjieqiang
Copy link

Any news about this PR? I have 5 pcs led matrix board

@netmindz
Copy link
Collaborator Author

netmindz commented Apr 15, 2024

There is support for hub75 in the latest MoonModules builds, just use V4_S or V4_M builds

As for support in AirCookie, @blazoncek said he would be updating the PR to address an issue he thought was a blocker to accpeting
#3777 (comment)

@blazoncek blazoncek added enhancement keep This issue will never become stale/closed automatically labels Apr 15, 2024
@blazoncek
Copy link
Collaborator

@netmindz I've tried to come up with a better solution regarding conditional UI elements for setting options for Hub panels.
Unfortunately I was not successful as that would require redesign of whole LED settings page and bus manager logic together with available LED types. I tried that but ended efforts as there was too little gain for too much work in the end.

I can see 2 ways forward as a result of this: a) make Hub 75 panels regular option and not a conditional compile or b) not support Hub 75 at all

For a) that would probably increase code size which most users would rather see used by other usermods instead (current flash utilisation is already above 95% while we are still with ESP32 core 3.5.0, newer cores add substantial amount to this).

There is of course still possibility to keep PR as is and hide Hub panels in 99.9% of installations at the expense of few wasted bytes for never used UI elements. I cannot decide either way so I'll leave the decision to @Aircoookie. I have marked the PR with keep label so it will not become stale.

For anyone else interested please upvote this PR so we can see how much interest is there and decide upon it.

@@ -344,7 +392,7 @@ struct BusConfig {
{
refreshReq = (bool) GET_BIT(busType,7);
type = busType & 0x7F; // bit 7 may be/is hacked to include refresh info (1=refresh in off state, 0=no refresh)
size_t nPins = Bus::getNumberOfPins(type);
size_t nPins = OUTPUT_MAX_PINS;
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are using the "pins" in the config file for more than just GPIO, so always copy the full array

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That may/will cause out of bounds access.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have checked the places the array is created and replaced the magic number 5 with the proper constant

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I don't change, then we would see an out of bounds as the number of GPIO pins used is 16 but there are only 3 "pins" used in the config

@netmindz netmindz marked this pull request as ready for review September 22, 2024 15:53
@netmindz
Copy link
Collaborator Author

There are many more config options that the library supports that are not (yet) exposed through the WLED UI, but my feeling is to merge what is there, then have a later follow up PR(s) that tackle specific items, like non-horizontal layout of multiple panels

@netmindz
Copy link
Collaborator Author

... and a label indicating which pinout configuration is active. But that's blocked by the pr that deals with the extra ledType data from the bus

Copy link
Collaborator

@blazoncek blazoncek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a few minor issues but otherwise seems ok.
Will need to take more time to go through code more thoroughly.

wled00/bus_manager.cpp Show resolved Hide resolved

fourScanPanel = nullptr;

if(bc.type == TYPE_HUB75MATRIX_HS) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use space between if and (.

I would also encourage the following form:

if (condition {
  ...
} else if (condition2) {
 ...
} else {
  ...
}

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

// let's adjust default brightness
display->setBrightness8(25); // range is 0-255, 0 - 0%, 255 - 100%

delay(24); // experimental
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this really needed?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

}
}

void __attribute__((hot)) BusHub75Matrix::setPixelColor(uint16_t pix, uint32_t c) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps a standards version of [[gnu:hot]] may be better choice. I'm not sure but I think the attribute is needed in declaration, not in definition.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

uint32_t BusHub75Matrix::getPixelColor(uint16_t pix) const {
if (!_valid || pix >= _len) return IS_BLACK;
if (_ledBuffer)
return uint32_t(_ledBuffer[pix].scale8(_bri)) & 0x00FFFFFF; // scale8() is needed to mimic NeoPixelBus, which returns scaled-down colours
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reasoning is false. This function should return original value the setPixelColor() set.
The behaviour of NPB has to be reverted and is done so in BusDigital.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

display->setBrightness(_bri);
}

void __attribute__((hot)) BusHub75Matrix::show(void) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does not need to be hot. Called at 23ms intervals.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed


#include <ESP32-HUB75-MatrixPanel-I2S-DMA.h>
#include <ESP32-VirtualMatrixPanel-I2S-DMA.h>
#include <FastLED.h>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think adding FastLED to bus manager is not appropriate.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was added for one of the functions or possibly CRGB. I'll check if it's still needed

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it's still required - see the usage of CRGB

class BusHub75Matrix : public Bus {
public:
BusHub75Matrix(BusConfig &bc);
bool hasRGB() { return true; }
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add override attribute.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can't actually override as inline, bu that suggestion helped find and fix that bug

@@ -344,7 +392,7 @@ struct BusConfig {
{
refreshReq = (bool) GET_BIT(busType,7);
type = busType & 0x7F; // bit 7 may be/is hacked to include refresh info (1=refresh in off state, 0=no refresh)
size_t nPins = Bus::getNumberOfPins(type);
size_t nPins = OUTPUT_MAX_PINS;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That may/will cause out of bounds access.

@softhack007 softhack007 added this to the 0.15.1 candidate milestone Sep 28, 2024
@softhack007 softhack007 self-assigned this Oct 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement keep This issue will never become stale/closed automatically
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants