Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support forced photometry #60

Open
wants to merge 12 commits into
base: master
Choose a base branch
from
Open

Support forced photometry #60

wants to merge 12 commits into from

Conversation

jvansanten
Copy link
Contributor

ZTF forced photometry is coming (at some point), and will mean three different kinds of datapoints:

  • Alert photometry, both from candidates and legacy history
  • Upper limits from legacy history
  • Forced photometry from new alert history

Of these, only alert photometry will come with unique ids (candid), making the previous convention for identifying datapoints (positive ids for candid, negative for a synthesized upper limit id) unsustainable. To better support data without unique ids, make the following changes:

  • ZiAlertSupplier: explicitly mark datapoints that come from candidate, to distinguish full differential photometry from the partial information supplied with legacy alert history.
  • ZiDataPointShaper: identify datapoints by the 64-bit hash of their sorted, bson-encoded content, and add tags to denote content type (ZTF_ALERT for a point from candidate, ZTF_DP for differential photometry, ZTF_UL for upper limit, ZTF_FP for forced photometry)
  • ZiMongoMuxer: when de-duplicating datapoints, prefer those tagged with ZTF_ALERT. This effectively reproduces the previous behavior, where attempts to insert a differential history point would instead update the alert point with the same candid.

- Import forced-photometry history verbatim in ZiAlertSupplier
- Treat flux > 5*fluxunc like a "detection" in alert photometry
This allows ZiDataPointShaper to tag candidate photometry to distinguish
it from prv_candidate (differential) photometry, which in turn allows
ZiMongerMuxer to use candidate photometry from a previous alert.
for better uniformity than blake2. This is more important than
irreversibility.
@vbrinnel
Copy link
Member

  • ... making the previous convention for identifying datapoints (positive ids for candid, negative for a synthesized upper limit id) unsustainable -> agreed
  • identify datapoints by the 64-bit (signed int) hash of their sorted, bson-encoded content -> a good solution indeed IMO
  • add tags to denote content type -> note that tag is (purposely) not indexed. origin could be used there (and would rather fit the purpose actually). origin can, but must not, be indexed. Some updates to T3 selection classes would be needed though, as these have currently no parameter to differentiate origins (as they should have).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants