Resolve PLL Programming and Div by 0 (#8)

* DKL: Fix a bug with multiple DKL phys not being seen

Each Dekel PHY is addressed through a 4KB aperture. Each PHY has more than
4KB of register space, so a separate index is programmed in HIP_INDEX_REG0
or HIP_INDEX_REG1, based on the port number, to set the upper 2 address
bits that point the 4KB window into the full PHY register space.
So, basically, if we read the value of the PLL DIV0 register and it is 0,
then that means that this aperture may need to be shifted. Once we do that,
we should re-read the value of the DIV0 register once more and this time it
should be valid. During resetting of the DKL phys, it is possible that the register index
moves back to another window and as such programming of the phy
registers has no impact. So before each programming, ensure that we are
in the right window.

* vbltest: Provide a cmd param to get Vsync interrupts on a per pipe basis

Earlier, we only used to get the vsyncs interrupts for the primary pipe.
This means, we couldn't synchronize other pipes because we didn't know
when their vsync interrupts were happening. With this change, this
program takes an extra parameter in the form of a 0 based pipe for which
we need to get the vsyncs. So the usage of vbltest is:

vbltest <number of vsyncs to get timestamp for> <0 based pipe to get for>

* Combo: Fix a divide by 0 error

According to the Bspec, we have to get the qdiv_mode first from cfgcr1
register's bit 9 and only if it is enabled, should we read qdiv_ratio
from cfgcr1's bit 17:10. Otherwise, it should be 1. We were incorrectly
not reading this bit 9 and that would cause us to read bits 17:10 even
when they were 0 which would end up causing a divide by 0 error. This
change addresses this problem.
---------

Signed-off-by: Jubilee Steinbrink <jubilee.steinbrink@intel.com>
Signed-off-by: Bill Mahoney <bill.mahoney@intel.com>
Signed-off-by: Eshe N Pickett <eshe.n.pickett@intel.com>
Signed-off-by: Satyeshwar Singh <satyeshwar.singh@intel.com>
Co-authored-by: Jubilee Steinbrink <jubilee.steinbrink@intel.com>
Co-authored-by: Bill Mahoney <bill.mahoney@intel.com>
Co-authored-by: Brian McGinn <brian.mcginn@intel.com>
Co-authored-by: Satyeshwar Singh <satyeshwar.singh@intel.com>

.gitignore

@@ -0,0 +1,41 @@
+# Temp Files
+*.swp
+*~
+
+# Prerequisites
+*.d
+
+# Compiled Object files
+*.slo
+*.lo
+*.o
+*.obj
+
+# Precompiled Headers
+*.gch
+*.pch
+
+# Compiled Dynamic libraries
+*.so
+*.dylib
+*.dll
+
+# Fortran module files
+*.mod
+*.smod
+
+# Compiled Static libraries
+*.lai
+*.la
+*.a
+*.lib
+
+# Executables
+*.exe
+*.out
+*.app
+
+tags
+.vscode
+vsync.code-workspace
+release/

LICENSE

@@ -1,2 +1,21 @@
-// Copyright (C) 2023 Intel Corporation
+// Copyright (C) 2024 Intel Corporation
 // SPDX-License-Identifier: MIT
+MIT License
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

Makefile

@@ -3,11 +3,11 @@
 
 #DIRS = $(shell find . -maxdepth 1 -type d -not -path "./.git" \
 #	   -not -path "." -not -path "./release" -not -path "./cmn" | sort)
-DIRS = lib synctest
+DIRS = lib test synctest vbltest
 .PHONY: $(DIRS)
 
 MAKE += --no-print-directory
-CFLAGS =-g
+
 all:
 	@for dir in $(DIRS); do \
 		$(MAKE) -C $$dir; \
@@ -30,4 +30,4 @@ release: distclean
 	@$(MAKE) clean
 	@$(MAKE)
 	@mkdir release
-	@cp lib/*.so synctest/synctest release
+	@cp lib/*.so test/vsync_test synctest/synctest vbltest/vbltest release

README.md

@@ -4,16 +4,174 @@
 This program allows a system to alter their vertical sync firing times.
 This is sample code only and is only supported on 11th Gen Core. 
 
-Building steps:
-1) sudo apt install libdrm-dev libpciaccess-dev
-1) Type 'make release' from the main directory. It compiles everything and creates a 
+
+# Prerequisites
+The build system assumes the following, prior to executing the build steps:
+1) The system has an [Ubuntu OS](https://ubuntu.com/tutorials/install-ubuntu-desktop#1-overview)
+1) Libraries installed: `sudo apt install -y git build-essential make`
+1) [Disable secure boot](https://wiki.ubuntu.com/UEFI/SecureBoot/DKMS)
+
+# Building steps:
+1) Git clone this repository 
+1) `sudo apt install libdrm-dev libpciaccess-dev`
+    1) If the directory /usr/include/drm does not exist, you may need to also add `sudo apt install -y linux-libc-dev`
+1) Type `make release` from the main directory. It compiles everything and creates a 
    release folder which can then be copied to the target systems.
 
 Building of this program has been succesfully tested on both Ubuntu 20 and Fedora 30.
 
-Installing and running the programs:
+# Installing and running the programs:
 1) On the system, go to the directory where these files exist and set environment variable:
-export LD_LIBRARY_PATH=.
+```console
+export LD_LIBRARY_PATH=`pwd`
+``````
 2) On the primary system, run it as follows:
-	./synctest 
-(Note by default this is set to move vsync by 1.0 ms.)
+	```console
+   ./synctest 
+   ```
+
+# Notes
+* By default this is set to move vsync by 1.0 ms. in [synctest/synctest.cpp](./synctest/synctest.cpp)
+* If experiencing screen flicker, adjust the SHIFT value in [lib/common.h](./lib/common.h)
+
+# Helpful libraries to have installed
+If you are doing debug, it helps to have the following libraries installed:
+```
+apt install -y intel-gpu-tools edid-decode
+```
+
+# Preparing two systems for PTP communication:
+In order to prepare two systems, please follow these steps:
+1) There should be a network cable directly connecting the ethernet ports of the two
+systems.
+2) Apply a patch in i915 driver which allows it to provide vsync timestamps in real time
+instead of the default monotonic time.
+3) Turn off NTP time synchronization service by using this command:
+	```timedatectl set-ntp no```
+4) Run ptp sync on both the systems as root user.
+	On the primary system, run the following command:
+	```ptp4l -i enp176s0 -m -f gPTP.cfg```
+	On the secondary system, run the following command:
+	```ptp4l -i enp176s0 -m -f gPTP.cfg -s```
+
+	There must be a gPTP.cfg file present in the same directory from where you run the
+	above two commands. The contents of this file should look like this:
+
+	```shell
+	#
+	# 802.1AS example configuration containing those attributes which
+	# differ from the defaults.  See the file, default.cfg, for the
+	# complete list of available options.
+	#
+	[global]
+	gmCapable		1
+	priority1		248
+	priority2		248
+	logAnnounceInterval	0
+	logSyncInterval		-3
+	syncReceiptTimeout	3
+	neighborPropDelayThresh 15000
+	min_neighbor_prop_delay	-20000000
+	assume_two_step		1
+	path_trace_enabled	1
+	follow_up_info		1
+	transportSpecific	0x1
+	ptp_dst_mac		01:80:C2:00:00:0E
+	network_transport	L2
+	delay_mechanism		P2P
+	```
+
+	The output on the primary system after running the above command should look
+	like this:
+
+	```console
+	ptp4l[13416.983]: selected /dev/ptp1 as PTP clock
+	ptp4l[13417.002]: port 1: INITIALIZING to LISTENING on INIT_COMPLETE
+	ptp4l[13417.003]: port 0: INITIALIZING to LISTENING on INIT_COMPLETE
+	ptp4l[13420.445]: port 1: LISTENING to MASTER on ANNOUNCE_RECEIPT_TIMEOUT_EXPIRES
+	ptp4l[13420.445]: selected local clock 844709.fffe.04f07f as best master
+	ptp4l[13420.445]: assuming the grand master role
+	```
+
+	The output on the secondary system after running the above command should
+	look like this:
+
+	```console
+	ptp4l[14816.313]: selected /dev/ptp1 as PTP clock
+	ptp4l[14816.328]: port 1: INITIALIZING to LISTENING on INIT_COMPLETE
+	ptp4l[14816.328]: port 0: INITIALIZING to LISTENING on INIT_COMPLETE
+	ptp4l[14820.220]: port 1: new foreign master 844709.fffe.04f07f-1
+	ptp4l[14820.250]: selected local clock 844709.fffe.04eb0e as best master
+	ptp4l[14822.220]: selected best master clock 844709.fffe.04f07f
+	ptp4l[14822.220]: port 1: LISTENING to UNCALIBRATED on RS_SLAVE
+	ptp4l[14822.650]: port 1: UNCALIBRATED to SLAVE on MASTER_CLOCK_SELECTED
+	ptp4l[14823.275]: rms 158899 max 317730 freq -11742 +/- 3075 delay    14 +/-   0
+	ptp4l[14824.276]: rms  820 max 1257 freq  -9067 +/- 1113 delay    14 +/-   0
+	ptp4l[14825.277]: rms 1366 max 1436 freq  -6582 +/- 359 delay    13 +/-   0
+	ptp4l[14826.278]: rms  866 max 1148 freq  -6099 +/-  34 delay    13 +/-   0
+	ptp4l[14827.279]: rms  280 max  465 freq  -6364 +/- 102 delay    12 +/-   0
+	ptp4l[14828.280]: rms   52 max   80 freq  -6666 +/-  65 delay    12 +/-   0
+	ptp4l[14829.280]: rms   82 max   88 freq  -6805 +/-  21 delay    12 +/-   0
+	ptp4l[14830.281]: rms   50 max   69 freq  -6829 +/-   3 delay    12 +/-   0
+	ptp4l[14831.282]: rms   15 max   28 freq  -6811 +/-   7 delay    12 +/-   0
+	ptp4l[14832.283]: rms    4 max    8 freq  -6795 +/-   5 delay    12 +/-   0
+	ptp4l[14833.284]: rms    7 max   10 freq  -6783 +/-   3 delay    12 +/-   0
+	ptp4l[14834.285]: rms    2 max    5 freq  -6788 +/-   3 delay    13 +/-   0
+	ptp4l[14835.286]: rms    3 max    5 freq  -6793 +/-   3 delay    13 +/-   0
+	```
+
+	Note that the rms value should be decreasing with each line and should go 
+	down to single digits.
+
+6) While step #4 is still executing, synchronize wall clocks of the system as
+the root user.
+	On both the primary & secondary systems, run the following command:
+	```phc2sys -s enp176s0 -O 0 -R 8 -u 8 -m```
+
+	In the above command, replace enp176s0 with the ethernet interface that
+	you find on your systems where the network cable is connected to.
+
+	Steps 4 & 5 will synchronize the wall clocks of the two systems. You can
+	stop these commands from executing by pressing Ctrl+C on both systems. Now
+	you are ready to execute the vsync_test.
+
+Installing and running the programs:
+1) Copy the release folder contents to both primary and secondary systems.
+2) On both systems, go to the directory where these files exist and set environment variable:
+export LD_LIBRARY_PATH=.
+3) On the primary system, run it as follows:
+	```./vsync_test pri [PRIMARY's PTP PORT]```
+3) On the secondary system, run it as follows:
+	```./vsync_test sec PRIMARY'S_NAME_OR_IP [PRIMARY'S ETH ADDR] [sync after # us]```
+
+This program runs in server mode on the primary system and in client mode on the 
+secondary system.
+
+If using PTP protocol to communicate between primary and secondary, there are some extra
+parameters required. The primary system must identify the PTP port (ex: enp176s0). The
+secondary system also requires the same PTP port (of primary) as well as this port's
+ethernet address.
+An example of PTP communication between primary and secondary looks like this:
+On the primary system, run it as follows:
+	```./vsync_test pri enp176s0```
+On the secondary system, run it as follows:
+	```./vsync_test sec enp176s0 84:47:09:04:eb:0e```
+
+In the above examples, we were just sync'ing the secondary system once with the primary.
+However, due to reference clock differences, we see that there is a drift pretty much as
+soon as we have sync'd them. We also have another capability which allows us to resync 
+as soon as it goes above a threshold. For example:  
+On the primary system, run it as follows:
+	```./vsync_test pri enp176s0```
+On the secondary system, run it as follows:
+	```./vsync_test sec enp176s0 84:47:09:04:eb:0e 100```
+
+In the above example, secondary system would sync with the primary once but after that
+it would constantly check to see if the drift is going above 100 us. As soon as it does,
+the secondary system would automatically resync with the primary. It is recommended to
+select a large number like 60 or 100 since if we use a smaller number, then we will be
+resyncing all the time. On the other hand, if we chose a really big number like 1000 us,
+then we are allowing the systems to get out of sync from each other for 1 ms. On a
+Tiger Lake system, 100 us difference is usually happening in around 60-70 seconds or
+about a minute. So this program would automatically resync with the primary every minute
+or so.

cmn/vsyncalter.h

@@ -10,6 +10,6 @@
 int vsync_lib_init();
 void vsync_lib_uninit();
 void synchronize_vsync(double time_diff);
-int get_vsync(long *vsync_array, int size);
+int get_vsync(long *vsync_array, int size, int pipe = 0);
 
 #endif

lib/Makefile

@@ -29,7 +29,7 @@ $(BINNAME): $(OBJECTS)
 		strip -x $@; \
 	fi
 
-.cpp.o: $(HEADERS)
+.cpp.o:
 	@echo $<
 	@$(CC) $(DBG_FLAGS) $(CFLAGS) $(LFLAGS) $(INCLUDES) $< 
 

lib/combo.cpp

@@ -1,3 +1,6 @@
+// Copyright (C) 2023 Intel Corporation
+// SPDX-License-Identifier: MIT
+
 #include <math.h>
 #include <debug.h>
 #include <signal.h>
@@ -18,12 +21,12 @@ combo_phy_reg combo_table[] = {
  * Description
  *  find_enabled_dplls - This function finds out which dplls are enabled on the
  *  current system. It does this by first finding out the values of
- * 	TRANS_DDI_FUNC_CTL register for all the pipes. Bits 30:27 have the DDI which
+ *	TRANS_DDI_FUNC_CTL register for all the pipes. Bits 30:27 have the DDI which
  *  this pipe is connected to. Once the DDI is found, we match the DDI with the
- * 	available ones on this platform. Then, we read DPCLKA_CFGCR0 or DPCLKA_CFGCR1
- * 	depending upon which DDI is enabled. Finally, the corresponding clock_bits
- * 	tell us which DPLL is turned on for this pipe. Now, we can go about reading
- * 	the corresponding DPLLs.
+ *	available ones on this platform. Then, we read DPCLKA_CFGCR0 or DPCLKA_CFGCR1
+ *	depending upon which DDI is enabled. Finally, the corresponding clock_bits
+ *	tell us which DPLL is turned on for this pipe. Now, we can go about reading
+ *	the corresponding DPLLs.
  * Parameters
  *	None
  * Return val
@@ -225,7 +228,7 @@ void program_combo_phys(double time_diff, timer_t *t)
 		int steps = calc_steps_to_sync(time_diff, shift);
 		DBG("steps are %d\n", steps);
 		user_info *ui = new user_info(COMBO, &combo_table[i]);
-		make_timer((long) steps, ui, t);
+		make_timer((long) steps, ui, t, reset_combo);
 #endif
 		DBG("OLD VALUES\n cfgcr0 \t 0x%X\n cfgcr1 \t 0x%X\n",
 				combo_table[i].cfgcr0.orig_val, combo_table[i].cfgcr1.orig_val);
@@ -243,8 +246,9 @@ void program_combo_phys(double time_diff, timer_t *t)
 		int pdiv = get_val_from_bit(pdiv_table, ARRAY_SIZE(pdiv_table), pdiv_bit);
 		int kdiv_bit = GETBITS_VAL(combo_table[i].cfgcr1.orig_val, 8, 6);
 		int kdiv = get_val_from_bit(kdiv_table, ARRAY_SIZE(kdiv_table), kdiv_bit);
+		int qdiv_mode = combo_table[i].cfgcr1.orig_val & BIT(9);
 		/* TODO: In case we run into some other weird dividers, then we may need to revisit this */
-		int qdiv = (kdiv == 2) ? GETBITS_VAL(combo_table[i].cfgcr1.orig_val, 17, 10) : 1;
+		int qdiv = (kdiv == 2 && qdiv_mode == 1) ? GETBITS_VAL(combo_table[i].cfgcr1.orig_val, 17, 10) : 1;
 		int ro_div_bias_frac = i_fbdivfrac_14_0 << 5 | ((i_fbdiv_intgr_9_0 & GENMASK(2, 0)) << 19);
 		int ro_div_bias_int = i_fbdiv_intgr_9_0 >> 3;
 		double dco_divider = 4*((double) ro_div_bias_int + ((double) ro_div_bias_frac / pow(2,22)));
@@ -261,8 +265,6 @@ void program_combo_phys(double time_diff, timer_t *t)
 			i_fbdiv_intgr_9_0 -= 1;
 			new_ro_div_bias_frac += 0xFFFFF;
 		}
-		combo_table[i].cfgcr0.mod_val &= ~GENMASK(9, 0);
-		combo_table[i].cfgcr0.mod_val |= i_fbdiv_intgr_9_0;
 		double new_i_fbdivfrac_14_0  = ((long int) new_ro_div_bias_frac & ~BIT(19)) >> 5;
 
 		DBG("old pll_freq \t %f\n", pll_freq);
@@ -276,7 +278,8 @@ void program_combo_phys(double time_diff, timer_t *t)
 		DBG("new fbdivfrac \t 0x%X\n", (int) new_i_fbdivfrac_14_0);
 		DBG("new ro_div_frac \t 0x%X\n", (int) new_ro_div_bias_frac);
 
-		combo_table[i].cfgcr0.mod_val &= ~GENMASK(24, 10);
+		combo_table[i].cfgcr0.mod_val &= ~GENMASK(24, 0);
+		combo_table[i].cfgcr0.mod_val |= i_fbdiv_intgr_9_0;
 		combo_table[i].cfgcr0.mod_val |= (long) new_i_fbdivfrac_14_0 << 10;
 
 		DBG("NEW VALUES\n cfgcr0 \t 0x%X\n", combo_table[i].cfgcr0.mod_val);
@@ -325,3 +328,35 @@ void program_combo_mmio(combo_phy_reg *pr, int mod)
 #endif
 }
 
+/*******************************************************************************
+ * Description
+ *  reset_combo - This function resets the Combo Phy MMIO registers to their
+ *  original value. It gets executed whenever a timer expires. We program MMIO
+ *	registers of the PHY in this function becase we have waited for a certain
+ *  time period to get the primary and secondary systems vsync in sync and now
+ *	it is time to reprogram the default values for the secondary system's PHYs.
+ * Parameters
+ *	int sig - The signal that fired
+ *	siginfo_t *si - A pointer to a siginfo_t, which is a structure containing
+ *  further information about the signal
+ *	void *uc - This is a pointer to a ucontext_t structure, cast to void *.
+ *  The structure pointed to by this field contains signal context information
+ *  that was saved on the user-space stack by the kernel
+ * Return val
+ *  void
+ ******************************************************************************/
+void reset_combo(int sig, siginfo_t *si, void *uc)
+{
+	user_info *ui = (user_info *) si->si_value.sival_ptr;
+	if(!ui) {
+		return;
+	}
+
+	DBG("timer done\n");
+	combo_phy_reg *cr = (combo_phy_reg *) ui->get_reg();
+	program_combo_mmio(cr, 0);
+	DBG("DEFAULT VALUES\n cfgcr0 \t 0x%X\n cfgcr1 \t 0x%X\n",
+			cr->cfgcr0.orig_val, cr->cfgcr1.orig_val);
+	cr->done = 1;
+	delete ui;
+}

lib/combo.h

@@ -1,3 +1,6 @@
+// Copyright (C) 2023 Intel Corporation
+// SPDX-License-Identifier: MIT
+
 #ifndef _COMBO_H
 #define _COMBO_H
 
@@ -55,5 +58,6 @@ int find_enabled_combo_phys();
 void program_combo_phys(double time_diff, timer_t *t);
 void check_if_combo_done();
 void program_combo_mmio(combo_phy_reg *pr, int mod);
+void reset_combo(int sig, siginfo_t *si, void *uc);
 
 #endif