-
Notifications
You must be signed in to change notification settings - Fork 5
Classifing Candidates Guide
- Setting up a connection to the MWA Pulsar Database server
- Connecting to the MWA Pulsar Database
- Ranking candidates on the MWA Pulsar Database
- Periodic Candidate Classification
- Single Pulse Candidate Classification
- Checking if a candidate is a known pulsar or a harmonic
- Rank 4 and 5 example candidates
- Rank 3 example candidates
- Rank 2 example candidates
- Rank 1 example candidates
- Rank 0 example candidates
- (Deprecated) Downloading Candidates
- (Deprecated) Running the cand_move.py script
The MWA Pulsar Database is not currently hosted on a website that is hosted on a Nimbus server at Pawsey. To get access to the server, you need a private key that was shared on the pulsar slack group. If you don't have access to slack, contact me, and I'll send it to you. Move the key to ~/.ssh and change the permissions using.
chmod go-rwx ~/.ssh/smart-web-app
Now that your ssh key is setup up so you do not have to repeat these steps again
Whenever you want to access the database and its webpage, you can forward a port to the server with the following command.
ssh -L 8000:localhost:8000 -i ~/.ssh/smart-web-app ubuntu@146.118.70.124
Leave the terminal open and in a web browser, go to the page http://localhost:8000/
, and you should have access to the database. To rank the candidates, you need an account which you can request from me (Nick). Once you have one, if you're not logging in you'll see something like this:
If you see this, click the login in link and input your username and password. If this is your first time, remember to change your password first. Once you're logged in, you should see something like this:
If so you're ready to move onto the next step.
To rank the candidates, go to the candidate ranking page using the url http://localhost:8000/candidates_ratings/
or clicking the "Candiate Rating" link on the homepage. This will bring you to a screen like this:
Click on any of the images (doesn't matter if you've already ranked them) and it should take you to the candidate ranking app:
Tick the "Is RFI?" box if you think it is RFI and any notes (explained soon) then click the 1 to 5 in buttons to give it a ranking from 1 to 5.
The ranking should be as follows:
5 An bright, excellent candidate that is definitely worth following up
4 Likely a pulsar but not extremely bright
3 A weak possible candidate
2 A very weak candidate that I have to squint to convince myself that it exists
1 Noise
RFI RFI (Radio Frequency Interference). You mark RFI by ticking (clicking) the RFI box and then giving the candidate a ranking. If you think there is a pulsar and RFI you can click RFI and give it a high ranking.
All candidates with an average ranking of 3 or more will be followed up with the full-length observation (80 minutes instead of 10 minutes of data), so feel free to be generous with the 3 rankings if you're unsure.
The "Notes (is it a known pulsar?)" window is a field that all users can edit to label candidates in any way the user sees fit. This is designed so we can use SQL queries to find all candidates with similar notes, for example, all the candidates with the same pulsar name. The following are some standard notes to label the candidates with:
KP This label is for known pulsars and their Jnames. For example, this candidate should be labelled "KP J2330-2005".
GL This label is for pulsars in the grating lobes (also known as side lobes). For example, this candidate can only be seen in part of the frequency band so this is likely a grating lobe detection so should be labelled "GL"
Harm This label is for harmonics of known pulsars. For example, this candidate is a 13/2 harmonic of J0820-1350 (will be explained in future sections) so should be labelled "Harm 6.5"
Here is a breakdown of PRESTO's periodic candidate classification plot using a bright detection of my favourite pulsar
Pulse profile: This is shown in the top left with the label "2 Pulses of Best Profile" (the pulse profile repeated twice). Here you look for a pulsar like signal. The most common is a nice delta function that is greater than the noise. Scattered pulsars (more common at DMs over ~120) show an exponential decay tail due to multi-path scattering. Millisecond pulsars (MSPs) have periods less than ~50 ms and often have much broader (up to 50% of the pulse profile) and complicated pulse profile structure.
Time plot: This is shown in the bottom left with a y-axis "Time (s)" and x-axis "Phase". We check if the pulsar is seen throughout the observation, as shown by the dark line through the middle of the example. Pulsars may not be seen in the entire observation due to the following reasons. It can either get fainter at the end of the observation or start faint and get brighter. This is because the tile beam of the MWA does a drift scan meaning that it points at a position on the sky and doesn't correct for the rotation of the earth so parts of the sky can move in and out of the beam. There are intermittent pulsars that sometimes turn "on" and "off", so if you see several bright spots on the time plot, you may be seeing such a pulsar. Be wary of Radio Frequency Interference (RFI) as this is often very limited in time, you will likely see it once in the observation, and it will last less than ~20 s.
Chi-squared plot: This is to the right of the time plot with a y-axis "Fraction of Observation" and x-axis "Reduced
Frequency plot: This is in the centre middle with y-axis "Sub-band" and "Frequency (MHz)" and x-axis "Phase". This plot is useful to check if the pulsar is continuous in frequency as RFI is often confined to a single frequency (a bandwidth less than 1 MHz). If a pulsar is scintillating strongly, you may see brighter and weaker spots along frequency, but this is fine. If a pulsar is in the side-lobe of the tied-array beam, then you may see it only in a small part of the frequency band (~5 MHz). This is because the beam response and, therefore, the side-lobes are frequency dependant. Hence, at some frequencies, there is enough sensitivity to detect the pulsar at some frequencies but not at other frequencies.
Dispersion Measure (DM) plot: This is in the bottom centre with a y-axis "Reduced
Period plot: This is the middle right plot with a y-axis "Reduced
P-dot plot: This is the top right plot with a y-axis "Reduced
Period vs P-dot plot: This is the bottom right plot with a y-axis "P-dot - (ms)" and x-axis "Period - (ms)". This is probably the less useful plot as it is only useful for checking that there isn't a second equally high peak in P/P-dot space and that your detection isn't a random fluctuation.
Data values: This is all the text in the top right. The most useful values are "P${bary}$ (ms)" (check that this value isn't a factor of 1 s as RFI often is), "Dispersion Measure (DM: pc/cm$^3$)" (check this isn't 0) and the position "RA${J200}$" and "DEC$_{J200}$". All of which is useful when checking if your candidate is a known pulsar (see section Checking if a candidate is a known pulsar or a harmonic)
In progress
To check if a candidate is a known pulsar got to the ATNF Pulsar Database
Check the Jname, DM and P0 boxes.
Below is an example of a candidate.
Scroll down to the "Condition" window and input
(DM > low_dm && DM < high_dm) && (P0 > low_period_in_seconds && P0 < high_period_in_seconds)
For this candidate, there is no clear DM or period peak, so we use the whole range (it is always best to over search rather than under search)
(DM > 73 && DM < 73.5) && (P0 > 0.166 && P0 < 0.167)
Then scroll down and click TABLE and you will get the following output.
Since J0742-2822 is within 30 degrees of the candidate position and the pulsars is only visible in part of the frequency plot, it is clear that this is a side-lobe detection of a known pulsar and should be given a rank of 4.
Below is an example of a candidate that returned no results from the above check.
There are 7 peaks in the pulse profile (or 14 if you count the repeated profile), so this is a harmonic candidate. Searching for harmonics is tricky because they're not always integer factors of the true period. It's best to first find one with a similar DM and roughly similar position and then divide the candidate period with the known pulsar period and see if it's a reasonable fraction. To do this, we only use the condition table for DM, so we input
(DM > 40 && DM < 45)
I chose these DM values because the peak in DM is between these values. Then use the "Within circular boundary" options to search 30 degrees around the RAJ and DECJ position like so:
Which after clicking TABLE you get the output:
So since our profile has 7 peaks, our period is likely 1.733495 / 7 = 0.247642. If we divide the periods of J0820-1350 by this period, we get 1.238130 / 0.247642 = 4.999677, so since that's so close to an integer, we can safely say that we detected the 5/7th harmonic of J0820-1350 and give this candidate a rank of 4.
If you still can't find the pulsar, increase the DM search range to +/- 10. This will help include known pulsars with inaccurate DMs (often due to them not being updated in decades).
The final check searches other telescopes candidate lists using the following website:
http://www.naic.edu/~deneva/tabscr/tabscr.php
If it's not found there, we are likely the first to find it, and it is cause for celebration!
This candidate can be seen faintly in the pulse profile, time plot and frequency plot. Even though it is faint, being seen in all 3 of those plots is enough to give us enough confidence to give this candidate a rank of 4 or 5 (depending on whether it is a known pulsar).
This is a very bright detection. The frequency plot shows that the pulsar is not continuous in frequency as this is a side-lobe detection of J0820-1350. If you see a detection this bright, don't get your hopes up as it's likely a known pulsar.
The frequency plot shows that the pulsar is not continuous in frequency as this is likely a side-lobe detection.
The frequency plot shows that the pulsar is not continuous in frequency as this is likely a side-lobe detection.
You can see two peaks in the pulse profile, time plot and frequency plot. This is because it is the harmonic of J0450-1248.
You can see three peaks in the pulse profile, time plot and frequency plot. This is because it is the harmonic of J0452-1759.
The pulse profile is rather wide and looks like it could be scattered, but after checking the ANTF database, it's clear that it's the 13/2 harmonic of J0820-1350 (and a side-lobe detection). J0820-1350's period is 807 ms, and it is common for harmonic candidates with periods much shorter than the true period to appear scattered or broad in its pulse profile.
This candidate can be seen faintly in at least parts of the time plot and frequency plot and shows a nice peak in the DM plot. This makes this candidate worthy of a rank of 3.
This candidate can be seen faintly in the pulse profile, at least parts of the time plot and frequency plot and shows a nice peak in the DM plot. This makes this candidate worthy of a rank of 3.
Don't be alarmed by the horizontal white lines in the time plot. These are a PRESTO plotting artifacts caused by the candidate's long period (6.1 s) and the short obs duration (600 s), meaning there aren't enough pulse rotations to fill the entire time range, so PRESTO leaves gaps. The pulse profile and the frequency plot provide good evidence that this is a pulsar. The infrequent, bright pulses in the time plot suggest it's an intermittent pulsar, so it's best to check the single pulse candidate plot with the same pointing. There is some evidence that it's RFI, such as it peaking at 0 DM, but given the other evidence that it's a pulsar, it's still worth following up, so it's given a ranking of 3.
The infrequent, bright pulses in the time plot suggest it's an intermittent pulsar, so it's best to check the single pulse candidate plot with the same pointing. It may be an RRAT that PRESTO was unable to find its true underlying period. There is some evidence that it's RFI because it peaked at 0 DM, but there is enough evidence that it is a pulsar, so it is still worth following up and is given a ranking of 3.
MSPs cause the time and frequency plots to be more pixelated and noisier. Detecting and MSP is a greater scientific achievement than finding and normal pulsar, so it's always best to be more lenient than normal pulsars. There is enough pulsar like signals in the pulse profile, time plot and frequency plot to give this candidate a rank of 3.
Two peaks can be seen in the pulse profile, the time plot and part of the frequency plot. This indicates it is likely the harmonic side-lobe detection and should be given a rank of 3.
Three peaks can be seen in the pulse profile, the start of the time plot and part of the frequency plot. This indicates it is likely the harmonic side-lobe detection that's only seen at the beginning of the observation and should be given a rank of 3.
If you squint, you may be able to see a faint signal in the pulse profile, time plot and frequency plot. This is too faint to be ranked 3, so it should be ranked 2 .
This candidate is likely RFI due to its ~1 s period and the reverse sweeps in the frequency plot, which is likely due to the RFI having a DM of 0 and it being dedispersed.
Similar to the above, this candidate is likely RFI due to its ~1 s period and the reverse sweeps in the frequency plot, which is likely due to the RFI having a DM of 0 and was dedispersed.
This candidate is likely RFI due to only being seen in one part of the time plot and frequency plot.
This candidate is likely RFI due to only being seen in one part of the time plot and frequency plot.
First, you need access to our supercomputing clusters Orthrus and Prometheus, which you should contact Sammy McSweeney (sammy.mcsweeney@gmail.com ) to create an account for you.
To find an observation have a look at the following google doc. On the column labelled "Quick pulsar search?" pick and observation with "Candidates require classification/ranking" And change that to "being classified by (your name)".
Then you want to download the observations candidates from Prometheus to your local machine using a command like this:
rsync -zru prometheus:/data/nswainston/SMART_cand_sorting/<obsid>/positive_detections <obsid>/
Then once it's downloaded the move into the obsid's positive_detections directory and run this script. The default method can be run like this:
python cand_move.py
It can be a bit tricky to install the dependencies, and the cv2 package doesn't seem to work on all laptops, so you can instead use a matplotib method using:
python cand_move.py -m matplotlib
The script should show you the plot, and then you can give it a ranking from 0 to 5, and it will put it into the subdirectory cand_grade_x where x is your ranking (if you misclick, you can manually move the files to a different directory)