PRIMEr

Tar heel gameplay is a tool built at UNC-Chapel Hill by Gary Bishop to allow anyone to create game like experiences with already existing gameplay footage. These are meant to be played by people with learning disabilities as they simplify the game immensely.

PRIMEr is a "primer" of sorts for this website, listed here. This means that it takes in videos of gameplay and "prepares" them to be played like video games. It does this by asking a user for "actions" which represent gameplay moments in the video. Such as moving, attacking, or jumping. It then scans the video with these examples to find all gameplay points in the video. Once a list is built containing timepoints, an editor preforms the necessary JSON formatting to allow the video to be inserted into the website.

This project's main focus is making the process of adding videos easier for the user adding them. As of now the current interface requires manual insertions of gameplay points across the entire video, a long and tedious process.

Building TemplateScanners

This project relies on another library, TemplateScanners, written in C++ with bindings for Python. This is done to take full advantage of multithreading. This library can be found here.

The process of building the template scanner module from the provided C++ code is fairly straightforward. It is highly recommended that this is built using linux as the build process with windows is significantly more complex, however, it has been confirmed to work on windows.

First install the following C++ modules and ensure their path variables are set:
OpenCV2 4.1 || Install Instructions
Note that you also will need python-dev (The libraries for python-dev should be installed already with python on windows but need to be acquired separately on linux.)
Boost/Boost::Python install instructions Linux
Boost/Boost::Python install instructions Windows

Ensure that Cmake is installed with version >= 3.12. From there build the .so file by invoking cmake in the TemplateScanners directory. Note that on windows the extension should be .pyd. Move the output file, either .pyd or .so into your python's DLLs or site-packages folder. On Windows you may need to include the following .dlls if they are unable to be detected in the path:
(the ..... are used to indicate build information that is specific to the user)
boost_python......dll
opencv_core401.dll
opencv_imgcodecs401.dll
opencv_imgproc401.dll
opencv_videoio401.dll

Unfortunately no pre-compiled libraries exist yet so this process must be followed in order to run the program. If unable to complete this, it's possible to download an outdated version of the TemplateScanners library written in Python from this commit. However, this is not going to be updated and will most likely fail to work with the current api without significant modification.

Results

3D Sonic

Mega Man

Punishment Examples:
Comparison Without Punishment ScannerV2
Comparison With Punishment ScannerV2

Another Punishment Example ScannerV2

HD Video Example

Transformers Example

Built With

openCV: https://github.com/opencv/opencv

flask: https://github.com/pallets/flask

flask-REST: https://github.com/flask-restful/flask-restful

flask-CORS: https://github.com/corydolphin/flask-cors

Boost: https://www.boost.org

Video Sources

Mario (NES): https://www.youtube.com/watch?v=rLl9XBg7wSs

Garfield (Famicom): https://www.youtube.com/watch?v=6JxeHtXmJ9s

FullMetal Alchemist (DS): https://www.youtube.com/watch?v=QxSl2XYw1jw

Megaman 2 (NES): https://www.youtube.com/watch?v=vuJ8Qr-3_zg

Transformers (NES): https://www.youtube.com/watch?v=uWjmkk5H9PE