Introduction to Shepherd
If you're reading this, it means you're probably interested in contributing to Shepherd in some capacity. We're excited to have you on our team and look forward to working with you. This page details what Shepherd is, how it works, and the roles the Shepherd team plays in the Pioneers in Engineering organization.
In short, Shepherd is the field control software that PiE uses and maintains to get the game running. Moreover, it is a framework that facilitates communication between a variety of important programs that are working together to run the game while also serving as an automated referee, keeping track of all the scores, timers, and rules for the game. As such, members of the Shepherd team work to develop this software. The team also has the additional responsibilities of creating sensor code to be used on the game field, creating coding challenges for the students, managing alliance selection, and keeping the PiE website up to date.
A typical year's Shepherd system might look like the following block diagram (don't worry, we are going to break it down):
So really all of Shepherd's parts fall into a few categories:
- State Machine
- Front End Interfaces
- Servers
- Supporting backend utilities
It's worth noting that all communication flows through the central state machine, and that many of the peripheral programs are stateless (they don't remember anything).
Shepherd is designed to run asynchronously. That means that each part of Shepherd is running in its own thread and is unaffected by a slowdown or crash that happens in another part (in theory). Therefore we need a method of communication that can send data from one part of Shepherd to another.
Here we have the same diagram as before, colored to show the form of communication used:
Here each of the greyed out blocks is initialized in its own thread.
- Red lines represent communication performed via YDL. This is a module that we wrote that is used to send data between two threads on the same machine. There is a lot more to know about Shepherd's use of YDL, which you can learn more about in its own documentation.
- Green lines represent communication via SocketIO and Proto Buffs, which is similar in functionality to JSON but is much lighter weight. SocketIO is used to pack and send this information to the robots.
- Orange lines represent communication via SocketIO and JSON. We use these two libraries to communicate with other computers on the internet (or often just on the same WiFi network).
- Blue lines represent communication via the serial ports of the computer. This is used to communicate with Arduinos, which we use to power our field sensors. See the Sensors page for more information.
- Black lines represent communication via a function call. This means that the two blocks shown are not running in separate threads and can simply be called / data returned normally.
There are several Python files that make up the state machine, the most notable being shepherd.py which facilities the entire process. As a whole these files are responsible for connecting all of Shepherd's bits together, primarily using YDL alongside the other methods of communication noted above. Shepherd.py itself keeps track of different parts of the game (current phase, score, etc.) which the other parts of Shepherd can access or update. A more in-depth explanation of each of these files can in the Python Files page.
We use various web pages during a competition in order to send information to Shepherd (through pages like the Match Creator) and also display information from it (like the Scoreboard). The most notable are pictured above in the communication diagram. These pages are run on a local server using Flask. This allows us to access the pages from any computer on the same network. You can learn how to use these webpages in [Using Shepherd](Using Shepherd.md). You can also learn more about how the server and web pages work in the Flask Server & UIs page.