The Zwicky Transient Facility (ZTF) has begun operations in 2018 and will map the Northern Galactic Plane twice nightly over the next three years, generating a massive real-time data flow of order of a million transient events daily. This creates an opportunity to make many important discoveries in stellar astrophysics if astronomers can immediately follow up these transient events. This research will use commercial open source software tools to provide the filtering and cross matching of the events with external catalogs, which contain contextual information to identify stars of interest and generate immediate alerts to the celestial community. It will provide a coordination center for the analysis of these events leading to discoveries and insights in stellar astrophysics. Graduate and undergraduate students will take part in this research which will develop their abilities in the rapidly developing fields of cloud and streaming platforms such as Kafka, Spark and Amazon Web Services, with the goal of developing a diverse, globally competitive STEM workforce.
The ZTF offers an outstanding opportunity to enhance our understanding of stellar astrophysics. To take advantage of this, an alert stream will be developed that consists of open source software tools which are used by commercial internet companies that specialize in massive real-time data flows. These tools will allow rapid crossmatching and filtering of the ZTF alert stream to identify significant Galactic transient events such as cataclysmic variables, state changes of X-ray binaries, and stellar flares, and to send out notices enabling immediate follow-up observations. Apache Kafka, an open-source messaging queue, will provide a distributed, fault-tolerant event stream at low latency. Apache Spark, an open-source cluster computing framework, will be used to provide fast general-purpose data processing. It includes the Spark Streaming interface to Kafka as well as SQL (Structured Query Language) databases necessary for cross-matching the catalogs. The alerts will result in discoveries and insights in stellar astrophysics. .
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
