Gentoo packages for the R5900

[frno7]

@immolo, this is a continuation of issue torvalds#71. After a Gentoo/MIPS stage 3 archive has been installed, it will need to be maintained and updated regularly. How can that be done?

GCC, for instance, vital to Gentoo, typically demands much more memory than is available with the 32 MiB of hardware. QEMU/R5900 has been ported to overcome such issues. Maybe something like Gentoo/MIPS binary packages would be useful, when it's unable compile packages for itself?

[immolo]

Good idea to move it here, I was thinking I should suggest it and then saw you just did it :)

This is a problem I'm facing with the Wii port of Gentoo as I'm sure you are aware distcc isn't useful for GCC. Creating a binhost is the solution to this but currently as so little people are involved with it's been far down my list of things to correctly setup.

We would have to maintain this binhost ourselves so while I could probably get a quick one setup for the starters we would need to find a way to make it more automated in the future, which sounds like a fun challenge.

My 'new' PS2 turned up finally in the post so if you give me a bit of time to find my old SwapMagic CDs and relearn how Linux works on the PS2 again as it's nearly been two decades since I've installed Gentoo on one and I'm pretty rusty around how it works, then I'll start looking into these issues.

I'll create an issue from this to allow progress to be tracked.

[frno7]

This is a problem I'm facing with the Wii port of Gentoo as I'm sure you are aware distcc isn't useful for GCC. Creating a binhost is the solution to this but currently as so little people are involved with it's been far down my list of things to correctly setup.

Oh, I didn’t know GCC had problems with distcc, but I think there’d be additional memory issues with Python and a other tools when building packages locally on the PlayStation 2. So I rule that out anyhow. :-)

We would have to maintain this binhost ourselves so while I could probably get a quick one setup for the starters we would need to find a way to make it more automated in the future, which sounds like a fun challenge.

Is it practical to supply precompiled packages for something as massive as Gentoo? Assuming one has got a working mipsr5900el-unknown-linux-gnu, or mipsr5900el-unknown-linux-musl, I hope it’s easy to precompile packages on-demand and on a need-to-have basis. Maybe mipsr5900el-gentoo-linux-gnu made by @TobiX and @AKuHAK can be adapted to serve as a basis for precompiling these packages.

My 'new' PS2 turned up finally in the post so if you give me a bit of time to find my old SwapMagic CDs and relearn how Linux works on the PS2 again as it's nearly been two decades since I've installed Gentoo on one and I'm pretty rusty around how it works, then I'll start looking into these issues.

Sure! See #4 (comment) for modern alternatives these days.

[immolo]

This is a problem I'm facing with the Wii port of Gentoo as I'm sure you are aware distcc isn't useful for GCC. Creating a binhost is the solution to this but currently as so little people are involved with it's been far down my
Oh, I didn’t know GCC had problems with distcc, but I think there’d be additional memory issues with Python and a other tools when building packages locally on the PlayStation 2. So I rule that out anyhow. :-)

GCC has two stages the first stage can be offloaded to other devices however as the first stage builds the second stage this won't happen and you have to wait for it to build. The 730mhz CPU on the Wii took 5 days to build that second stage with 77mb of RAM and a 4GB Swapfile....

Is it practical to supply precompiled packages for something as massive as Gentoo? Assuming one has got a working mipsr5900el-unknown-linux-gnu, or mipsr5900el-unknown-linux-musl, I hope it’s easy to precompile packages on-demand and on a need-to-have basis. Maybe mipsr5900el-gentoo-linux-gnu made by @TobiX and @AKuHAK can be adapted to serve as a basis for precompiling these packages.

We can decide which packages we want to host premade so we don't have to don't have to build everything so I don't think it will be too much trouble.
I suggest we start out by recommending setting up a cross distcc environment with a binhost for just GCC, then we can scale up as we understand the issues more, as they say don't run before you can walk.
I have no idea how easy it will be to do the automated part but luckily Gentoo is looking at doing this at scale for the main distro, so I have lots of amazing people that I can learn from to avoid their early mistakes.

My 'new' PS2 turned up finally in the post so if you give me a bit of time to find my old SwapMagic CDs and relearn how Linux works on the PS2 again as it's nearly been two decades since I've installed Gentoo on one and I'm pretty rusty around how it works, then I'll start looking into these issues.

Sure! See #4 (comment) for modern alternatives these days.

Wow this come along way since 2004 so you all have done amazing to make the process this much better. Thanks for highlighting it too.

[immolo]

@frno7 I'm having a strange issue where my kernel and initramfs seem to build find but when I open the kernel via launchelf it just shows a black screen, do you have any idea to what I am doing wrong?

[frno7]

@AKuHAK, if you know the video resolution coming from wLaunchELF, you could skip set_res and its registers.

[AKuHAK]

all of what? Setting the GS registers in

At least I got some output on my PS3 now :) So at least I know that it is failing before the console on the EARLY_PRINTK stage. However, the EARLY_PRINTK stage is giving me weird graphic issues. I was trying to set up low res mode by examining GS registers but probably I am doing this somehow wrong. I tried to set up 640x512i mode in such a way:

static void set_env(void)
{
	const u32 xres = 640;
	const u32 yres = 512;

and

	const u64 smode1_ = 0x740806504;
	struct gs_smode1 smode1;

	memcpy(&smode1, &smode1_, sizeof(smode1));
	smode1.gcont = 0;	/* For YCrCb output */
	smode1.sint = 1;
	smode1.prst = 0;

	gs_write_smode1(smode1);
	gs_writeq_smode2(0x1);
	gs_writeq_srfsh(0x8);
	gs_writeq_synch1(0x7f5c21fc83030);
	gs_writeq_synch2(0x3484bc);
	gs_writeq_syncv(0xa9000502101401);
	gs_writeq_display1(0x1c79ff0184a2a9);

	gs_write_dispfb1((struct gs_dispfb) {
		.fbw = 640 / 64,
		.psm = gs_psm_ct32,
		.dbx = 120,
		.dby = 120,
	});

	gs_write_pmode((struct gs_pmode) {
		.en1 = 1,
		.crtmd = 1
	});

	smode1.prst = 1;
	gs_write_smode1(smode1);
	udelay(2500);			/* 2.5 ms FIXME: spinlock */

	smode1.sint = 0;
	smode1.prst = 0;
	gs_write_smode1(smode1);

In SCPH-70003 and PS3-CECHA00 I am getting the same weird green screen, on regular ps2 then boots normal console, but on ps3 I am stuck on that screen. In pcsx2 btw, I am getting normal printk messages. @frno7 can you, please, change early_printk from 1920x1080 to some low res mode, for example, 640x512i

[AKuHAK]

[frno7]

@AKuHAK, this green thing very much looks like you’ve changed to gs_psm_ct32 in

	gs_write_dispfb1((struct gs_dispfb) {
		.fbw = 640 / 64,
		.psm = gs_psm_ct32,
		.dbx = 120,
		.dby = 120,
	});

but you’re still on gs_psm_ct16 in

linux/arch/mips/boot/compressed/dbg.c

Line 418 in 59a11ab

const u32 psm = gs_psm_ct16;

where the FRAME_1 register is set up. I recommend getting gs_psm_ct16 to work first, and then later on, if necessary, change to gs_psm_ct32. PSM stands for pixel storage mode, and when drawing one typically wants this mode of the FRAME register to be the same as the mode of the DISPFB register. Confer pages 110 and 147 in the GS User’s Manual, version 6.0.

Can you try this? I think you’re very close now to having a good working early console. I’ve written some pieces of documentation on these concepts here:

linux/arch/mips/include/uapi/asm/gs.h

Lines 13 to 36 in 59a11ab

	/**
	* DOC: The PlayStation 2 Graphics Synthesizer
	*
	* The Graphics Synthesizer frame buffer is subdivided into rectangular pages,
	* from left to right, top to bottom. Pages are further subdivided into blocks,
	* with different arrangements for the PSMCT16 and PSMCT32 pixel storage modes.
	* Blocks are further subdivided into columns that are finally subdivided into
	* pixels.
	*
	* The frame buffer width (FBW) is defined as the pixel width divided by 64,
	* where 64 is the width of a page for PSMCT16 and PSMCT32.
	*
	* The texture base pointer (TBP), color base pointer (CBP), source base
	* pointer (SBP) and destination base pointer (DBP) are all addressed by block.
	*
	* As an example, consider the frame buffer resolution 1920x1080@16 that uses
	* 1920*1080*2 = 4147200 bytes of the 4 MiB = 4194304 bytes available memory.
	* There are 4194304-4147200 = 47104 bytes of free memory. The FBW is 1920/64
	* = 30, which means it is 30 pages wide. However, 1080/GS_PSM_CT32_PAGE_HEIGHT
	* = 1080/64 = 16.875, which means that the last row of blocks are unused in
	* the last page row. These make up 30*4 = 120 blocks of the 47104/256 = 184
	* blocks available. To make effective use of free memory for this resolution
	* in particular, it is essential to compute accurate block addresses.
	*/

[AKuHAK]

Thanks, 16bit did the trick!

[AKuHAK]

@frno7 the kernel is crashing on PSX DESR after printing that info. After that, it just froze completely.

[frno7]

@AKuHAK, the problem seems easy enough to spot, as the log says

iop-module: Direct firmware load for ps2/intrman.irx failed with error -2
iop-module: ps2/intrman.irx module version 1.2 request failed with error -2
iop-module: irqrelay dependency on intrman failed to resolve with -2
iop-module: ps2/irqrelay.irx module version 1.0 request failed with -2

So the intrman library version 1.2 (or later minor version) wasn’t found in ROM for this particular PSX DESR device, so the kernel assumed it had to be loaded as an IRX file, which wasn’t found either. Perhaps it’s related to issue #72, assuming that intrman exists at all?

By the way, it involves the logic you remarked on in #28 (comment). :-)

[frno7]

After that, it just froze completely.

To be exact, it’s most likely only OHCI interrupts that aren’t relayed from the IOP to the R5900 (that’s what the failed irqrelay module is for), so the kernel isn’t completely frozen but rather alive without any means of USB interaction. Perhaps an academic difference, but still. Had #68 been available the kernel had probably happily responded on the serial port. :-)

[immolo]

@frno7

I'm still having issue with video output when I complie my own kernel but I think that will be solved soon when my HDMI adapter comes shortly.

If you have a second could you please build a test PAL kernel with the following init so I can debug while I wait for that?

Edit: Formatting gets ruined here so I've uploaded init file to https://www.immolo.co.uk/init

This should detect correct Partition label so we don't need to worry much but feel free to check it over as you are much better at this part than I am :)

[immolo]

@frno7 Champagne is on order as I've finally got the damn thing to work with all your help.

I'll get back to work on creating the LiveUSB and hopefully have something for next weekend when I return from a work trip.

In other news my ps2hdmi does support 1080p with pretty decent results to my eyes, so I got lucky with the one I was recommended.

[frno7]

Bliss! :-)

[immolo]

Does anyone know why we only have 22mb of RAM available rather than the 32mb the PS2 offers?

I know with the Wii that IOS reserves 11mb so is this a similar thing with the PS2?

[frno7]

Does anyone know why we only have 22mb of RAM available rather than the 32mb the PS2 offers?

@immolo, I guess initramfs has claimed a good chunk of the discrepancy. Check the total size of your initramfs directory, after the kernel modules have been installed. I very much hope switch_root will reclaim it as free memory.

Some clues about memory usage are given in the kernel log:

# dmesg | grep Memory:
Memory: 20360K/37888K available (4149K kernel code, 183K rwdata, 1252K rodata, 2472K init, 154K bss, 17528K reserved, 0K cma-reserved)

I know with the Wii that IOS reserves 11mb so is this a similar thing with the PS2?

PlayStation 2 Linux discards the Sony BIOS, to claim 32 MiB of main RAM for itself. Details in issue #72, with links to additional comments.

It’s a very good idea to improve the use of available memory, and there are a lot of opportunities to explore.

[immolo]

That was the issue I was trying to search for, thank you.

Those file sizes do match up with when had 30mb of ram available back in the early 2000s when removing the initramfs size so I feel you could be on to something here.

[AKuHAK]

@frno7 can you point out to the good dropbear configuration guide? I finally got usb to ethernet adapter, would like to test it

[frno7]

I can confirm that the USB Ethernet adapter D-Link DUB-1312 works, for instance. USB 3.0 and gigabit is more than enough for PlayStation 2 hardware, of course. It needs

modprobe cdc_mbim
modprobe ax88179_178a

in /sbin/init, with CONFIG_USB_NET_CDC_MBIM=m and CONFIG_USB_NET_AX88179_178A=m (unless these modules are linked into the kernel).

I’d like to enable CONFIG_IPV6 for IPv6 too, but it seems the kernel becomes too big and won’t boot so something else needs to be removed to fit it. IPv6 comes with stateless address autoconfiguration (SLAAC), and other modern things.

[immolo]

@frno7 Have you tried this in Gentoo yet? As I'd love to see if you run into the issue where a device that doesn't have an assigned IP to mac address fails to connect to the network so I have some more information to begin debugging. I've yet again run into my curse of being promised someone would give up their time for some testing and it not happening...

The GLIBC build currently seems like it's stable enough in it's alpha form that outside this out needing confirming and PATA support that we could request this to become to be an official project so a wider audience can get see the amazing work you all have done with this kernel.

Oh before I forget can you can fix the link in the Readme.md as it currently links to the wrong wiki page for Gentoo.

[frno7]

@frno7 Have you tried this in Gentoo yet? As I'd love to see if you run into the issue where a device that doesn't have an assigned IP to mac address fails to connect to the network so I have some more information to begin debugging. I've yet again run into my curse of being promised someone would give up their time for some testing and it not happening...

I haven’t tried the live USB, but I do have Gentoo on an ATA device, to be used when the ATA driver is functional.

The GLIBC build currently seems like it's stable enough in it's alpha form that outside this out needing confirming and PATA support that we could request this to become to be an official project so a wider audience can get see the amazing work you all have done with this kernel.

Yes, why not. :-)

Oh before I forget can you can fix the link in the Readme.md as it currently links to the wrong wiki page for Gentoo.

Done, and some more: I’ve added a Gentoo live USB for the PlayStation 2 item on the top level home wiki page. This separation makes the article on installing and booting PlayStation 2 Linux simpler. Likewise the article on building Gentoo for the PlayStation 2 is separated from the live USB article.

[frno7]

By the way, it would be nice to have a wiki article on building a Gentoo live USB for the PlayStation 2.

[immolo]

It's quite a hacky solution right now so I'm sure anyone would want to do it this way :)

if I don't get the catalyst build done though then I'll post the long winded way to create one once I've confirmed my MUSL Gentoo boots.