Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set the machine field in /proc/cpuinfo to the PlayStation 2 model #12

Open
frno7 opened this issue Mar 17, 2019 · 38 comments
Open

Set the machine field in /proc/cpuinfo to the PlayStation 2 model #12

frno7 opened this issue Mar 17, 2019 · 38 comments
Labels
enhancement New feature or request

Comments

@frno7
Copy link
Owner

frno7 commented Mar 17, 2019

The machine field in /proc/cpuinfo is currently set to unknown:

# head -n2 /proc/cpuinfo 
system type		: Sony PlayStation 2
machine			: Unknown

Set the machine field to the PlayStation 2 model, such as SCPH-50004 or SCPH-77004.

Retrieving the model seems to require some level of IOP support, see #11.

Use the sysfs pseudo file system to display further hardware revision information?

@frno7 frno7 added the enhancement New feature or request label Mar 17, 2019
@sp193
Copy link

sp193 commented Apr 13, 2019

I wouldn't say that it is surely dependent on the IOP, but you need some form of access to the CD/DVD subsystem. It can also be accessed directly from the EE, although it might be good to keep things standardized.

In normal PS2 software, the model name is normally obtained via libcdvd's sceCdReadModelName() function. If the first 4 characters of ROMVER is "0100J", then the HDD Browser will default to "SCPH-10000". If "0101J", then it will read the model name from offset 0x8C808 of OSDSYS, although I haven't found a PS2 with ROM 1.01J that doesn't report itself as "SCPH-10000".

@frno7
Copy link
Owner Author

frno7 commented Apr 13, 2019

It can also be accessed directly from the EE,

That is very interesting! Would you be able to elaborate on how that could be done from the EE? I would like to implement it. Also, eventually I would like to map all address spaces, as in #13.

although it might be good to keep things standardized.

What kind of standardisation? I have, for example, removed the BIOS code used by the 2.6 PS2 Linux kernel to instead have the kernel interact directly with the SIF hardware for IOP RPCs. In my opinion, we need not be bound by official abstractions. We can invent better ones. 😄

@sp193
Copy link

sp193 commented Apr 13, 2019

You would need to issue the appropriate S-commands, by writing to the hardware. You would need to dig for the command and some code from CDVDMAN. I believe I also have some code from my OSD initialization sample because the default CDVDMAN module from the boot ROM does not support this function, but I used CDVDMAN to issue the S-command.

I know you could write your own system, but this is not an open platform and it is largely proprietary. If you deviate, you will have to support everything on your own, including figuring out why the hardware misbehaves (which is hard). While writing my own copies of some of the Sony modules, I used to think I could just change the code. But sometimes that caused more harm because the original code was sometimes designed to cater for hardware characteristics, be it bugs or some design requirement. By at least using their IOP modules to begin with, it becomes easier to compare results. You will also be able to offload processing of the hardware to the IOP and to reduce duplicated code with the PS2SDK, if you manage to use the standard modules.

The official way of using the hardware is to go through the IOP. Only PS2Linux has driver code on the EE, with interrupts and DMA routed over the SIF.

Over the years, we have also established that somehow, accessing some IOP peripherals only works well from the IOP. I do not have an explanation for this, unfortunately.

@frno7
Copy link
Owner Author

frno7 commented Apr 13, 2019

You would need to issue the appropriate S-commands, by writing to the hardware. You would need to dig for the command and some code from CDVDMAN. I believe I also have some code from my OSD initialization sample because the default CDVDMAN module from the boot ROM does not support this function, but I used CDVDMAN to issue the S-command.

Thanks! I need to learn what S-commands are to proceed with this, obviously.

By at least using their IOP modules to begin with, it becomes easier to compare results. You will also be able to offload processing of the hardware to the IOP and to reduce duplicated code with the PS2SDK, if you manage to use the standard modules.

I agree, using standard modules is the best starting point. Also, we need to figure out what kind of thing the IOP is to the Linux kernel, see #11.

Only PS2Linux has driver code on the EE, with interrupts and DMA routed over the SIF.

Linux USB device drivers cannot in general run on the IOP. They are far too complex and demand too much memory. That said, the Linux USB OHCI driver can hopefully be much more efficient, making use of DMA as suggested in #17. The same reasoning applies for Linux kernel file systems for #18 and #23 and possibly #21.

PlayStation 2 specific devices such as #22 can be implemented differently, I think.

@Ravenslofty
Copy link

@PSI-Rockin wrote an incomplete list of CDVD commands in https://psi-rockin.github.io/ps2tek/#cdvd - it could be a useful reference.

@sp193
Copy link

sp193 commented Apr 14, 2019

I don't mean that the Linux modules should run on the IOP. Not only would that be complicated, it could be slow. I meant to say that you could use some of the existing PS2 IOP libraries (IRX modules), much like how the Sony SBIOS works. Rather than re-implementing everything on the EE.

frno7 pushed a commit that referenced this issue Apr 28, 2019
[ Upstream commit 42dfa45 ]

Using gcc's ASan, Changbin reports:

  =================================================================
  ==7494==ERROR: LeakSanitizer: detected memory leaks

  Direct leak of 48 byte(s) in 1 object(s) allocated from:
      #0 0x7f0333a89138 in calloc (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xee138)
      #1 0x5625e5330a5e in zalloc util/util.h:23
      #2 0x5625e5330a9b in perf_counts__new util/counts.c:10
      #3 0x5625e5330ca0 in perf_evsel__alloc_counts util/counts.c:47
      #4 0x5625e520d8e5 in __perf_evsel__read_on_cpu util/evsel.c:1505
      #5 0x5625e517a985 in perf_evsel__read_on_cpu /home/work/linux/tools/perf/util/evsel.h:347
      #6 0x5625e517ad1a in test__openat_syscall_event tests/openat-syscall.c:47
      #7 0x5625e51528e6 in run_test tests/builtin-test.c:358
      #8 0x5625e5152baf in test_and_print tests/builtin-test.c:388
      #9 0x5625e51543fe in __cmd_test tests/builtin-test.c:583
      #10 0x5625e515572f in cmd_test tests/builtin-test.c:722
      #11 0x5625e51c3fb8 in run_builtin /home/changbin/work/linux/tools/perf/perf.c:302
      #12 0x5625e51c44f7 in handle_internal_command /home/changbin/work/linux/tools/perf/perf.c:354
      #13 0x5625e51c48fb in run_argv /home/changbin/work/linux/tools/perf/perf.c:398
      #14 0x5625e51c5069 in main /home/changbin/work/linux/tools/perf/perf.c:520
      #15 0x7f033214d09a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2409a)

  Indirect leak of 72 byte(s) in 1 object(s) allocated from:
      #0 0x7f0333a89138 in calloc (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xee138)
      #1 0x5625e532560d in zalloc util/util.h:23
      #2 0x5625e532566b in xyarray__new util/xyarray.c:10
      #3 0x5625e5330aba in perf_counts__new util/counts.c:15
      #4 0x5625e5330ca0 in perf_evsel__alloc_counts util/counts.c:47
      #5 0x5625e520d8e5 in __perf_evsel__read_on_cpu util/evsel.c:1505
      #6 0x5625e517a985 in perf_evsel__read_on_cpu /home/work/linux/tools/perf/util/evsel.h:347
      #7 0x5625e517ad1a in test__openat_syscall_event tests/openat-syscall.c:47
      #8 0x5625e51528e6 in run_test tests/builtin-test.c:358
      #9 0x5625e5152baf in test_and_print tests/builtin-test.c:388
      #10 0x5625e51543fe in __cmd_test tests/builtin-test.c:583
      #11 0x5625e515572f in cmd_test tests/builtin-test.c:722
      #12 0x5625e51c3fb8 in run_builtin /home/changbin/work/linux/tools/perf/perf.c:302
      #13 0x5625e51c44f7 in handle_internal_command /home/changbin/work/linux/tools/perf/perf.c:354
      #14 0x5625e51c48fb in run_argv /home/changbin/work/linux/tools/perf/perf.c:398
      #15 0x5625e51c5069 in main /home/changbin/work/linux/tools/perf/perf.c:520
      #16 0x7f033214d09a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2409a)

His patch took care of evsel->prev_raw_counts, but the above backtraces
are about evsel->counts, so fix that instead.

Reported-by: Changbin Du <changbin.du@gmail.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steven Rostedt (VMware) <rostedt@goodmis.org>
Link: https://lkml.kernel.org/n/tip-hd1x13g59f0nuhe4anxhsmfp@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
frno7 pushed a commit that referenced this issue Apr 28, 2019
…_event_on_all_cpus test

[ Upstream commit 93faa52 ]

  =================================================================
  ==7497==ERROR: LeakSanitizer: detected memory leaks

  Direct leak of 40 byte(s) in 1 object(s) allocated from:
      #0 0x7f0333a88f30 in __interceptor_malloc (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xedf30)
      #1 0x5625e5326213 in cpu_map__trim_new util/cpumap.c:45
      #2 0x5625e5326703 in cpu_map__read util/cpumap.c:103
      #3 0x5625e53267ef in cpu_map__read_all_cpu_map util/cpumap.c:120
      #4 0x5625e5326915 in cpu_map__new util/cpumap.c:135
      #5 0x5625e517b355 in test__openat_syscall_event_on_all_cpus tests/openat-syscall-all-cpus.c:36
      #6 0x5625e51528e6 in run_test tests/builtin-test.c:358
      #7 0x5625e5152baf in test_and_print tests/builtin-test.c:388
      #8 0x5625e51543fe in __cmd_test tests/builtin-test.c:583
      #9 0x5625e515572f in cmd_test tests/builtin-test.c:722
      #10 0x5625e51c3fb8 in run_builtin /home/changbin/work/linux/tools/perf/perf.c:302
      #11 0x5625e51c44f7 in handle_internal_command /home/changbin/work/linux/tools/perf/perf.c:354
      #12 0x5625e51c48fb in run_argv /home/changbin/work/linux/tools/perf/perf.c:398
      #13 0x5625e51c5069 in main /home/changbin/work/linux/tools/perf/perf.c:520
      #14 0x7f033214d09a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2409a)

Signed-off-by: Changbin Du <changbin.du@gmail.com>
Reviewed-by: Jiri Olsa <jolsa@kernel.org>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steven Rostedt (VMware) <rostedt@goodmis.org>
Fixes: f30a79b ("perf tools: Add reference counting for cpu_map object")
Link: http://lkml.kernel.org/r/20190316080556.3075-15-changbin.du@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
frno7 pushed a commit that referenced this issue Apr 28, 2019
[ Upstream commit d982b33 ]

  =================================================================
  ==20875==ERROR: LeakSanitizer: detected memory leaks

  Direct leak of 1160 byte(s) in 1 object(s) allocated from:
      #0 0x7f1b6fc84138 in calloc (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xee138)
      #1 0x55bd50005599 in zalloc util/util.h:23
      #2 0x55bd500068f5 in perf_evsel__newtp_idx util/evsel.c:327
      #3 0x55bd4ff810fc in perf_evsel__newtp /home/work/linux/tools/perf/util/evsel.h:216
      #4 0x55bd4ff81608 in test__perf_evsel__tp_sched_test tests/evsel-tp-sched.c:69
      #5 0x55bd4ff528e6 in run_test tests/builtin-test.c:358
      #6 0x55bd4ff52baf in test_and_print tests/builtin-test.c:388
      #7 0x55bd4ff543fe in __cmd_test tests/builtin-test.c:583
      #8 0x55bd4ff5572f in cmd_test tests/builtin-test.c:722
      #9 0x55bd4ffc4087 in run_builtin /home/changbin/work/linux/tools/perf/perf.c:302
      #10 0x55bd4ffc45c6 in handle_internal_command /home/changbin/work/linux/tools/perf/perf.c:354
      #11 0x55bd4ffc49ca in run_argv /home/changbin/work/linux/tools/perf/perf.c:398
      #12 0x55bd4ffc5138 in main /home/changbin/work/linux/tools/perf/perf.c:520
      #13 0x7f1b6e34809a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2409a)

  Indirect leak of 19 byte(s) in 1 object(s) allocated from:
      #0 0x7f1b6fc83f30 in __interceptor_malloc (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xedf30)
      #1 0x7f1b6e3ac30f in vasprintf (/lib/x86_64-linux-gnu/libc.so.6+0x8830f)

Signed-off-by: Changbin Du <changbin.du@gmail.com>
Reviewed-by: Jiri Olsa <jolsa@kernel.org>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Steven Rostedt (VMware) <rostedt@goodmis.org>
Fixes: 6a6cd11 ("perf test: Add test for the sched tracepoint format fields")
Link: http://lkml.kernel.org/r/20190316080556.3075-17-changbin.du@gmail.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
@frno7
Copy link
Owner Author

frno7 commented May 13, 2019

The system command reading the machine name is now implemented in commit e16b1ae and is used to set the /proc/cpuinfo machine field in commit a84cf79:

# head -n2 /proc/cpuinfo
system type		: Sony PlayStation 2
machine			: SCPH-50004

Reading the machine names for SCPH-10000 and SCPH-15000 are not yet implemented.

frno7 pushed a commit that referenced this issue May 20, 2019
[ Upstream commit 5712f33 ]

The spinlock in the raw3270_view structure is used by con3270, tty3270
and fs3270 in different ways. For con3270 the lock can be acquired in
irq context, for tty3270 and fs3270 the highest context is bh.

Lockdep sees the view->lock as a single class and if the 3270 driver
is used for the console the following message is generated:

WARNING: inconsistent lock state
5.1.0-rc3-05157-g5c168033979d #12 Not tainted
--------------------------------
inconsistent {IN-HARDIRQ-W} -> {HARDIRQ-ON-W} usage.
swapper/0/1 [HC0[0]:SC1[1]:HE1:SE0] takes:
(____ptrval____) (&(&view->lock)->rlock){?.-.}, at: tty3270_update+0x7c/0x330

Introduce a lockdep subclass for the view lock to distinguish bh from
irq locks.

Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>

Signed-off-by: Sasha Levin <sashal@kernel.org>
frno7 pushed a commit that referenced this issue May 20, 2019
[ Upstream commit b9abbdf ]

By calling maps__insert() we assume to get 2 references on the map,
which we relese within maps__remove call.

However if there's already same map name, we currently don't bump the
reference and can crash, like:

  Program received signal SIGABRT, Aborted.
  0x00007ffff75e60f5 in raise () from /lib64/libc.so.6

  (gdb) bt
  #0  0x00007ffff75e60f5 in raise () from /lib64/libc.so.6
  #1  0x00007ffff75d0895 in abort () from /lib64/libc.so.6
  #2  0x00007ffff75d0769 in __assert_fail_base.cold () from /lib64/libc.so.6
  #3  0x00007ffff75de596 in __assert_fail () from /lib64/libc.so.6
  #4  0x00000000004fc006 in refcount_sub_and_test (i=1, r=0x1224e88) at tools/include/linux/refcount.h:131
  #5  refcount_dec_and_test (r=0x1224e88) at tools/include/linux/refcount.h:148
  #6  map__put (map=0x1224df0) at util/map.c:299
  #7  0x00000000004fdb95 in __maps__remove (map=0x1224df0, maps=0xb17d80) at util/map.c:953
  #8  maps__remove (maps=0xb17d80, map=0x1224df0) at util/map.c:959
  #9  0x00000000004f7d8a in map_groups__remove (map=<optimized out>, mg=<optimized out>) at util/map_groups.h:65
  #10 machine__process_ksymbol_unregister (sample=<optimized out>, event=0x7ffff7279670, machine=<optimized out>) at util/machine.c:728
  #11 machine__process_ksymbol (machine=<optimized out>, event=0x7ffff7279670, sample=<optimized out>) at util/machine.c:741
  #12 0x00000000004fffbb in perf_session__deliver_event (session=0xb11390, event=0x7ffff7279670, tool=0x7fffffffc7b0, file_offset=13936) at util/session.c:1362
  #13 0x00000000005039bb in do_flush (show_progress=false, oe=0xb17e80) at util/ordered-events.c:243
  #14 __ordered_events__flush (oe=0xb17e80, how=OE_FLUSH__ROUND, timestamp=<optimized out>) at util/ordered-events.c:322
  #15 0x00000000005005e4 in perf_session__process_user_event (session=session@entry=0xb11390, event=event@entry=0x7ffff72a4af8,
  ...

Add the map to the list and getting the reference event if we find the
map with same name.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Eric Saint-Etienne <eric.saint.etienne@oracle.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Song Liu <songliubraving@fb.com>
Fixes: 1e62856 ("perf symbols: Fix slowness due to -ffunction-section")
Link: http://lkml.kernel.org/r/20190416160127.30203-10-jolsa@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
@frno7
Copy link
Owner Author

frno7 commented May 26, 2019

SCPH-10000 and SCPH-15000 are now recognised in commit cda11cb, although I'm unable to verify those machines myself.

@sp193, I have defined the system command for reading the ROMVER file in commit ab958ab but there is a one structure member that remains unknown to me:

struct rom_ver {
	int number;
	char region;
	char unknown;	/* FIXME: What is this? */
	struct {
		int year;
		int month;
		int day;
	} date;
};

I read a C for the unknown field, as in 0170EC20030227. Do you have any idea what it represents?

@sp193
Copy link

sp193 commented May 26, 2019

We have no official information on this, but to me it indicates the ROM type.

For the common PlayStation 2, it is a C for CEX.
On DebugStation consoles, this is a D for DEX.
On the PlayStation 2 TOOL and Arcade Computers, this is a T for TOOL.

Arcade Computers like System 246 seem
to have ROMs based on the TOOL version, for some reason.

@frno7
Copy link
Owner Author

frno7 commented May 26, 2019

Many thanks, @sp193! I will document this in the source code. I assume that C stands for consumer and D for developer, but what does the EX part in CEX and DEX stand for? Is TOOL an abbreviation for anything?

I have patches to use the R5900 and by extension the main processor of the PlayStation 2 with QEMU. Perhaps I could allocate another letter such as E for emulated or V for a virtual ROM in such cases. It would be used for testing the Linux kernel.

@sp193
Copy link

sp193 commented May 26, 2019

It is a boot ROM, so you cannot replace it. Even in an emulator, is there a reason to make a totally original ROM, as opposed to making one that is compatible with the CEX ROM?

It could be that this lettering was for Sony to determine the contents of the ROM for tagging purposes, as a DEX ROM needed to support any possible game region and also differ in MagicGate capabilities, while the TOOL ROM had DECI2 support, support for its hardware and no MagicGate file support.

I have no idea what the EX in CEX meant. Neither do I know what SCPH or DTL means.

@frno7
Copy link
Owner Author

frno7 commented May 26, 2019

Are there free ROM implementations available?

The ROMVER region and type fields are now nicely resolved with names in the kernel log:

# dmesg | grep -E rom.:
rom0: Found 98 files in 3900816 bytes
rom1: Found 29 files in 407408 bytes
rom0: Version 0170 Europe CEX 2003-02-27

The fields are also available via sysfs:

# ls /sys/rom/rom0/version
date    number  region  type
# cat /sys/rom/rom0/version/*
2003-02-27
0170
Europe
CEX

One can also inspect the raw ROMVER file, using /dev/mem:

# grep -l ROMVER /sys/rom/rom0/file/*/name
/sys/rom/rom0/file/3/name
# cd /sys/rom/rom0/file/3
# ls
data     extinfo  name     size
# dd if=/dev/mem bs=$(cat size) iflag=skip_bytes
	skip=$(( $(cat data) )) count=1 status=none
0170EC20030227

Or inspect any of the other files in ROM0 and ROM1:

# cat /sys/rom/rom0/file/*/name | sort | column
-		FILEIO		LOGO		ROMVER		TSIO2MAN
-		FNTIMAGE	MCMAN		SBIN		TZLIST
-		FONTM		MCSERV		SECRMAN		UDNL
-		HDDLOAD		MODLOAD		SIFCMD		VBLANK
-		HEAPLIB		OSDCNF		SIFINIT		VERSTR
-		ICOIMAGE	OSDSND		SIFMAN		XCDVDFSV
-		IGREETING	OSDSYS		SIO2MAN		XCDVDMAN
ADDDRV		INTRMANI	OSDVER		SNDIMAGE	XFILEIO
ATAD		INTRMANP	PADMAN		SSBUSC		XFLASH
CDVDFSV		IOMAN		PS1DRV		STDIO		XFROMMAN
CDVDMAN		IOPBOOT		PS1ID		SYSCLIB		XLOADFILE
CLEARSPU	IOPBTCON2	PS1VER		SYSMEM		XMCMAN
DMACMAN		IOPBTCONF	PS2LOGO		TBIN		XMCSERV
EECONF		KERNEL		RDRAM		TESTMODE	XMTAPMAN
EELOAD		KROM		REBOOT		TESTSPU		XPADMAN
EELOADCNF	KROMG		RESET		TEXIMAGE	XRMMAN2
EENULL		LIBFI		RMRESET		THREADMAN	XSIFCMD
EESYNC		LIBSD		ROMDIR		TIMEMANI	XSIO2MAN
EXCEPMAN	LOADCORE	ROMDRV		TIMEMANP
EXTINFO		LOADFILE	ROMGSCRT	TPADMAN
# cat /sys/rom/rom1/file/*/name | sort | column
CDVDFSV		EROMDRV		LOADFILE	ROMDIR		STDIO
CDVDMAN		EXTINFO		MODLOAD		ROMDRV		SYSCLIB
DVDCNF		FILEIO		PADMAN		SDRDRV		THREADMAN
DVDID		IOMAN		RESET		SIFCMD		TIMEMANI
DVDVER		LIBSD		RMMAN		SIFMAN		UDNL
EESYNC		LOADCORE	RMMAN2		SIO2MAN

@Ravenslofty
Copy link

Ravenslofty commented May 26, 2019 via email

@mirh
Copy link

mirh commented May 26, 2019

CEX/DEX is also there in ps3 scene where they supposedly mean retail and debug. Not sure if there's anything behind (though those terms are inside a disassembly maybe?)

SCPH is likely Sony Computer PlayStation Hardware

And DTL.. =debug tool?

@Ravenslofty
Copy link

Worth mentioning that I would be willing to continue with biostation if people care enough; it seems like a useful thing, but I have difficulties with motivating myself to do something if I'm the only person who will ever use it.

@mirh
Copy link

mirh commented May 26, 2019

Very simple reminder that it's just 2 decades pcsx2 has been waiting for not needing bios dumping to work.

@frno7
Copy link
Owner Author

frno7 commented May 26, 2019

@ZirconiumX, did you figure out the format of the EXTINFO ROM file? It looks like file metadata, and I would be happy to decode it. Here, for example, looking at the EXTINFO entry for the ROMDIR file:

# cd /sys/rom/rom0/file
# cat 1/name
ROMDIR
# dd if=/dev/mem bs=$(cat 1/extinfo/size) iflag=skip_bytes
	skip=$(( $(cat 1/extinfo/data) )) count=1 status=none | xxd
00000000: 0000 4c03 3230 3033 3032 3237 2d31 3933  ..L.20030227-193
00000010: 3035 302c 524f 4d63 6f6e 662c 5053 3230  050,ROMconf,PS20
00000020: 3137 3045 4332 3030 3330 3232 372e 6269  170EC20030227.bi
00000030: 6e2c 6861 6e61 4072 6f6d 2d73 6572 7665  n,hana@rom-serve
00000040: 722f 7e2f 672f 6170 702f 726f 6d00 0000  r/~/g/app/rom...

And the SBIN file:

# cat 4/name
SBIN
# dd if=/dev/mem bs=$(cat 4/extinfo/size) iflag=skip_bytes
	skip=$(( $(cat 4/extinfo/data) )) count=1 status=none | xxd
00000000: 0000 0401 0304 0220 0100 0002            ....... ....

@sp193 and @ZirconiumX, regarding QEMU: It currently does Linux user mode emulation for the R5900. This is the most convenient way to compile for the PlayStation 2, because the compiler and all other software run as native R5900 programs, avoiding all problems with cross-compilation.

I'm considering implementing QEMU system mode emulation specifically to test the R5900 Linux kernel. I imagine that the only device in such a configuration would be a serial port, perhaps also USB or ATA, so QEMU would not have an IOP, a Graphics Synthesizer, or anything like that. Like @mirh just noted, there are other emulators that have those devices, but I suspect they are not (yet?) good enough for running a Linux kernel.

@ZirconiumX, in some sense the Linux kernel is a ROM replacement, because it handles everything for the EE without running any ROM code at all. In fact, it completely discards and reclaims the SBIOS. 😄 The SIF, for example, is implemented in drivers/ps2/sif.c and is complete enough to run simple IOP services such as heap allocation that is needed for the Linux USB OHCI device driver. I'm hoping to discard and reclaim the memory for most if not all IOP modules as well, since I think Linux eventually will have better variants of them anyway.

@sp193
Copy link

sp193 commented May 26, 2019

This is a Linux project, so the IOP isn't important here I guess. But the PlayStation 2 IOP will always run Sony's proprietary IOP kernel, regardless of whether Linux advertises that or not.
It may replace the EE kernel, but it cannot run the whole PS2 without the IOP.

The EXTINFO structure is known. I also wrote my own tool for building images, called ROMIMG. PCSX2 also has its own code for working on the ROM format, which I also took some information from.

@Ravenslofty
Copy link

GXemul can emulate a PS2 running Linux to some extent, but it doesn't have USB support which the official kernel requires, so it just hung.

As for the ROM, I'd imagine you'd want to boot Linux first? Sure, you can overwrite the ROM-in-RAM, but you need to get there first.

@frno7
Copy link
Owner Author

frno7 commented May 26, 2019

GXemul can emulate a PS2 running Linux to some extent, but it doesn't have USB support which the official kernel requires, so it just hung.

Oh, nice! USB is optional with the 5.x Linux kernels I'm working on so that shouldn't be problem then.

As for the ROM, I'd imagine you'd want to boot Linux first? Sure, you can overwrite the ROM-in-RAM, but you need to get there first.

Of course. Everything up to actually starting Linux is running a combination of ROM code and other software, such as Free MC boot, and so on.

@frno7
Copy link
Owner Author

frno7 commented May 26, 2019

This is a Linux project, so the IOP isn't important here I guess.

Well, the IOP as a piece of hardware is as important as ever, also for Linux, but if memory can be reclaimed by discarding unused modules then that is an improvement. More space for useful things. After all, the Linux project is about using the hardware, and not Sony's ROM software. 😁

But the PlayStation 2 IOP will always run Sony's proprietary IOP kernel, regardless of whether Linux advertises that or not.

I suppose the IOP kernel could be replaced, if one wants to. Why not?

The EXTINFO structure is known. I also wrote my own tool for building images, called ROMIMG. PCSX2 also has its own code for working on the ROM format, which I also took some information from.

Thanks, @sp193! I found ROMIMG and will have look at the EXTINFO format.

@frno7
Copy link
Owner Author

frno7 commented May 27, 2019

@sp193, ROMIMG was indeed helpful and EXTINFO is now decoded in commit 81dd020 and shown in sysfs with commit fc83e56. Would you know what the 0x7f type means? It is EXTINFO_FIELD_TYPE_NULL in your code. Few ROM files use it. Why would they have this property?

@mirh, is the PS3 scene active with OtherOS? I have a few PS3 machines with it installed, and would be happy to work on that too.

@mirh
Copy link

mirh commented May 27, 2019

Uh, well.. Put aside Geoff, the only person doing something "active" with linux that I know is @rxrbln
You may want to check this guide then.. It may not sport mainline, but at least way noober people than you got it working.

On the other hand, the big news on the global scene is the recent invention of HFW/HEN which finally opened >3.56 minfw to the world (in addition to providing yet again a software entry point to all fat and slims owners of course).
Though (at least for the moment?) you can't get OtherOS++ without CFW.

While we are at it, I'd also like to remember of PSXLinux and vita-whatever.

@sp193
Copy link

sp193 commented May 28, 2019

0x7F seemed to be for the "-" entries, which exists to align the adjacent file at a specific address.

@uyjulian
Copy link

- files are padding files.
I have a WIP reimplementation of the modules included in the PS2 ROM located here: https://github.com/uyjulian/ps2iop Currently, stubs are pushed to the repository.
It is possible to reload the software currently running in the IOP with other versions using UDNL. When it is reset, it is running the modules from the boot ROM. A reset -> UDNL chain is possible, which allow you to make sure that the specific version of modules are loaded.
It may also be possible to run bare-metal ELFs on the IOP (like the romflash tool on PS2 TOOL models).
The IOPRP.img file used with UDNL for loading updated IOP modules has the same filesystem format as the PS2 ROM.

@frno7
Copy link
Owner Author

frno7 commented May 28, 2019

The following files in a SCPH-50004 have the 0x7f property:

ROM file index name address type
rom0 0 RESET 0x1fc00000 0x7f
rom0 32 RDRAM 0x1fc41000 0x7f
rom0 43 IOPBOOT 0x1fc4a000 0x7f
rom0 46 TBIN 0x1fc4b800 0x7f
rom0 52 KROMG 0x1fc64000 0x7f
rom0 54 KROM 0x1fc66000 0x7f
rom0 56 VERSTR 0x1fc7ff30 0x7f
rom0 58 ROMGSCRT 0x1fc80000 0x7f
rom1 0 RESET 0x1e000000 0x7f

The alignment seems to be to a page size, typically around 4096 bytes or so, with the curios exceptions of TBIN and VERSTR. Except for the RESET files at index 0, all the files above are preceded with these - padding files:

ROM file index name size
rom0 31 - 16
rom0 42 - 208
rom0 45 - 1200
rom0 51 - 2960
rom0 53 - 832
rom0 55 - 192
rom0 57 - 112

Still, the question somewhat remains as to why the 0x7f is marked in the files following the padding files and what purpose that mark could serve there. Maybe it is some kind of annotation designed for a ROM file layout tool. Perhaps we will never know.

Anyway, the original issue can now be considered fully resolved. Many thanks for all the help! 😄

@mirh, do you know if any CFW could be configured to default cold boots into OtherOS? As far as I understand it, CFWs such as Rebug only boots into GameOS, which means one always has to click through various menus every time OtherOS is booted. I'm running Gentoo on both the PS2 and the PS3, actually. 😁

Thanks for your link, @uyjulian! Looks like it is worth studying your implementation.

@mirh
Copy link

mirh commented May 28, 2019

I am under the vague impression that yes, you can (and indeed René videos don't ever seem to show gameos)

@sp193
Copy link

sp193 commented May 29, 2019

Most of those files have to exist at fixed addresses. I know that RESET must absolutely be at the start of the file, as it contains the bootstrap code that must reside at the standard MIPS reset vector.

Newer PS2s also have DECKARD, which is likely also flagged as such a file.

RDRAM is jumped to, so it must also reside at a known address.
Some of the other files like are used by PS mode. It is possible that the PS ROM also expects them at fixed addresses.

ROMGSCRT is used by PS2Linux. From its code, you can tell how it was meant to be resolved.

Kernels like TBIN, KERNEL and IOTBOOT seem to not actually need to be located at fixed places because they are actually scanned for, by RESET.

rom1 belongs to the DVD ROM. Even if that is not the BOOT ROM, it also contains RESET, although that has no code in it.

@frno7
Copy link
Owner Author

frno7 commented Jun 20, 2019

@sp193, I have just created a new iopmod repository that is specifically made for IOP modules. The ABI is compatible with Sony and PS2SDK modules, but the source code is organised differently. Modern GCC and binutils are used. I intend to write all IOP modules for Linux there, but in principle any kind of IOP module could be compiled with it.

@frno7
Copy link
Owner Author

frno7 commented Jun 30, 2019

@sp193, interestingly, some European PAL machines report J in their ROMVER files, for example 0220JC20050620 for SCPH-75004 and 0220JC20060210 for SCPH-77004. These are not Japanese machines. All other European PAL machines I have seen report E:

  • SCPH-30004
  • SCPH-30004 R
  • SCPH-39004
  • SCPH-50004
  • SCPH-70004

The Japanese SCPH-37000 L reports J. All run the Linux 5.1 kernel successfully, as I am preparing #1.

I guess some other method is needed to determine the region of the machine.

@sp193
Copy link

sp193 commented Jun 30, 2019

All European CEX machines are known to have an E there. But because the PlayStation was made by Japan, so they default to Japanese defaults when the actual value cannot be resolved.

I do not know why you need to know the region, but you should know that there are various region types for the PS2. For example, we have the regions for the ROM, discs, CSS, video mode and MagicGate. The ROMVER letter cannot be used in all cases.

In older PS2s, it used to be that the ROM was tailored for the specific region. From v1.80 (PSX) and later (SCPH-50000), the OSDSYS was changed to check OSDVER for the release region and language. From v2.20 (SCPH-75000), the ROM itself gets manipulated by the IOP to reflect input from the MECHACON.

@sp193
Copy link

sp193 commented Jun 30, 2019

If you got a 'J' by reading the ROMVER from the EE, that could be due to the gimmick by the IOP to change the letter. In that case, the only correct way to read the ROM would be to access it from the IOP itself.

frno7 pushed a commit that referenced this issue Sep 1, 2019
Revert the commit bd293d0. The proper
fix has been made available with commit d0a255e ("loop: set
PF_MEMALLOC_NOIO for the worker thread").

Note that the fix offered by commit bd293d0 doesn't really prevent
the deadlock from occuring - if we look at the stacktrace reported by
Junxiao Bi, we see that it hangs in bit_wait_io and not on the mutex -
i.e. it has already successfully taken the mutex. Changing the mutex
from mutex_lock to mutex_trylock won't help with deadlocks that happen
afterwards.

PID: 474    TASK: ffff8813e11f4600  CPU: 10  COMMAND: "kswapd0"
   #0 [ffff8813dedfb938] __schedule at ffffffff8173f405
   #1 [ffff8813dedfb990] schedule at ffffffff8173fa27
   #2 [ffff8813dedfb9b0] schedule_timeout at ffffffff81742fec
   #3 [ffff8813dedfba60] io_schedule_timeout at ffffffff8173f186
   #4 [ffff8813dedfbaa0] bit_wait_io at ffffffff8174034f
   #5 [ffff8813dedfbac0] __wait_on_bit at ffffffff8173fec8
   #6 [ffff8813dedfbb10] out_of_line_wait_on_bit at ffffffff8173ff81
   #7 [ffff8813dedfbb90] __make_buffer_clean at ffffffffa038736f [dm_bufio]
   #8 [ffff8813dedfbbb0] __try_evict_buffer at ffffffffa0387bb8 [dm_bufio]
   #9 [ffff8813dedfbbd0] dm_bufio_shrink_scan at ffffffffa0387cc3 [dm_bufio]
  #10 [ffff8813dedfbc40] shrink_slab at ffffffff811a87ce
  #11 [ffff8813dedfbd30] shrink_zone at ffffffff811ad778
  #12 [ffff8813dedfbdc0] kswapd at ffffffff811ae92f
  #13 [ffff8813dedfbec0] kthread at ffffffff810a8428
  #14 [ffff8813dedfbf50] ret_from_fork at ffffffff81745242

Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Cc: stable@vger.kernel.org
Fixes: bd293d0 ("dm bufio: fix deadlock with loop device")
Depends-on: d0a255e ("loop: set PF_MEMALLOC_NOIO for the worker thread")
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
@AKuHAK
Copy link

AKuHAK commented Feb 14, 2021

If you got a 'J' by reading the ROMVER from the EE, that could be due to the gimmick by the IOP to change the letter

it seems that Mechacon patches bootrom on boot. This can be proved by the recently discovered feature on the slim DTL-H console which allows changing console region by holding a special button combination on boot. After changing the region, mechacon patches BOOTrom on the fly by changing 2 bytes:

  • ROMVER region string (for example: 0220ED20050620)
  • VERSTRT (for example System ROM Version 5.0 09/02/00 E)
    NVRAM of course is also changed, but it is another story.

But it seems that these patches are ignored when accessing from the EE, cause from the EE side we are reading real BOOT ROM content (not patched on the fly). It looks like all BOOT ROMs in all later slims are the same.
Not patched ROM is using J as a region:

  • 0220JD20050620
  • System ROM Version 5.0 09/02/00 J

As for me reading BOOT ROM from the EE side is more correct, cause we are touching real BOOT ROM content in that way. And we can determine the region by reading NVRAM instead or call some additional command.

@frno7
Copy link
Owner Author

frno7 commented Feb 15, 2021

Thanks, @AKuHAK. Your findings make sense to me, and I agree that the real ROM appears to the correct one to use. NVRAM support remains to be implemented. By the way, the ROM is used for several purposes in the Linux kernel, for example to prelink the necessary IRX files as explained in IOPMOD issue #1.

@AKuHAK
Copy link

AKuHAK commented Dec 30, 2021

@frno7 Just some comments from ROM0 and ROM1 current implementation:

#define ROM0_BASE 0x1fc00000 /* ROM0 base address (boot) */
#define ROM0_SIZE 0x400000 /* ROM0 maximum size */
#define ROM1_BASE 0x1e000000 /* ROM1 base address (DVD) */
#define ROM1_SIZE 0x100000 /* ROM1 maximum size */

The only hardcoded value is ROM0_BASE other values are calculated:
ROM1 can be missed, for example on SCPH-10000, PSX DESR, all DTL-H machines
ROM1 also can vary in size, early units have 2Mb, while later units have 4Mb
ROM0 also isn't of fixed size:
more information about how it is actually calculated in this code snippet:

https://github.com/ps2homebrew/PS2Ident/blob/abb56af6494d827b117dd41db267bad25a0d233b/sysman/rom.c#L62

and later, ROMGetHardwareInfo() contains more info about possible scenarios.

@frno7
Copy link
Owner Author

frno7 commented Dec 30, 2021

Thanks, @AKuHAK! Indeed, we’ll have to revisit some hardware assumptions for a few models. After all, we have a page on PlayStation 2 Linux hardware model compatibility, as well as #28 needing to be fixed.

The following piece of code walks the files of a ROM, automatically terminating after the last one:

/**
* rom_for_each_file - iterate over files in given ROM directory
* @file: &struct rom_file to use as a ROM file loop cursor
* @dir: &struct rom_dir with ROM directory to iterate over
*
* The statement following the macro is executed for ROM files in the
* directory.
*/
#define rom_for_each_file(file, dir) \
for ((file) = rom_first_file(dir); \
!rom_terminating_file(file); \
(file) = rom_next_file(file))

So as the comments for ROM0_SIZE and ROM1_SIZE suggest the size is only an upper bound. The true sum of the sizes of all ROM0 files in for example a particular SCPH-50004 is easily calculated with a small Awk shell script:

# cat /sys/rom/rom0/file/*/size | awk '{ s += $1 } END { print s }'
3900175

which is close to 4 MiB, in this example. ROM1 in the same machine is much smaller, only summing to 407171 bytes. Are some models really 10 times larger than this? Which one?

The Linux kernel is more capable with the ROM files: it can for example also list all IRX symbols for all ROM object files, to determine which ones to automatically prelink when loading a supplied IRX file. So IOP device drivers need not worry about any ROM symbol dependencies at all.

I just wrote an article on the Graphics Synthesizer for the wiki. I’m going to write a similar one for the ROM files soon.

@AKuHAK
Copy link

AKuHAK commented Dec 30, 2021

ROM1 in the same machine is much smaller, only summing to 407171 bytes. Are some models really 10 times larger than this? Which one?

Oh, probably there are some problems with terminology :)
There are 2 big chips inside PS2: DEV2 and DEV1. They can be 2Mb or 4Mb in size. The size always is a multiplication of 1Mb, don't remember if I saw 1Mb chips, probably some Namco Arcades has 1Mb, but I am not sure.

DEV2 chip usually contains ROM0 (aka BOOTROM). It is possible that it doesn't fill the whole 4Mb chip and the end bytes become zero-filled. DEV2 is always mapped into memory at address 0x1fc00000. You set up the maximum size as 4Mb, but in some cases, it can be dangerous to read the 2Mb chip after its end. This can be, for example in case of PS2Tool if the user flashed incorrect chip firmware. It will be safer to calculate Maximum Size from function GetSizeFromDelay(SSBUSC_DEV_BOOTROM); which always returns DEV2 chip actual size. Also, there can be some interesting information after ROM0 ended but before DEV2 ended. In the case of PS2TOOL that has an option to write chip firmware, there can be some leftover traces from earlier firmware. This, of course, can be skipped - this is not very useful data anyway, I just want to explain how things are working in detail.

As for DEV1 - things get worse. You assume that it starts from address 0x1e000000, but this is not true, address can vary and can be different in different consoles. DEV1 mapping address can be found by calling GetBaseAddress(SSBUSC_DEV_DVDROM); Next, the size of the chip also always is larger than 1Mb, it can be calculated by GetSizeFromDelay(SSBUSC_DEV_DVDROM);. This size can be wrong and larger, in some cases, it returns mirrored chip sizes (so it can be even 16Mb for slims).

Next, you mentioned that you get ROM1 maximum size 407171, it is true, but you didn't read the whole information from DEV1:

The DEV1 contains the rom1, rom2, and erom regions, and these regions exist in this order within the DEV1 chip.
The rom2 region only exists on Chinese consoles.
(TOOL consoles have DEV1 installed, but it contains no filesystem and contains hardware IDs at specific offsets).
EROM region is encrypted (Encrypted ROM) and it is large - almost 4Mb in size. It actually contains DVD Player firmware in encrypted form.

The most critical problem - ROM1 start address, it can be wrong and you will not be able to properly read rom1. If you don't care about erom and about non-standard chip content, other functions can remain intact, but they will be only assumptions.

@frno7
Copy link
Owner Author

frno7 commented Dec 30, 2021

Linux only actually uses ROM files such as LOADCORE, SIFMAN and a handful related ones. They are for IOP memory allocations, threads, DMA transfers, and suchlike. I believe all of them are always in ROM0. Other ROM files for peripherals such as USB, gamepads, etc. are ignored since Linux has its own IRX modules for those. Maybe there will be some complications for DVD devices, as in #23, but that will come later.

So ROM1 is only ever used by people who are curious and want to examine their machines themselves. I just wrote an article on PlayStation 2 read only memory (ROM) for the wiki, with some examples on how that can be done.

Obviously the ROM1 implementation is currently a bit simplistic and imperfect, so we’ll have to revisit it when I or someone else want to deep dive into it. Never mind ROM2. 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

6 participants