Image may be NSFW.
Clik here to view.
© ASTRON
But how about classic filtering approaches, could those be useful in feature and anomaly detection? Could these techniques be combined with AI or perhaps run stand-alone? To start answering these questions we have looked into âclassicâ filters that select a subspace (compressing data) of each spectrogram in order to separate features that we see in LOFAR spectrogram plots. When features are separated it is easier to build detectors that can detect those features.
The image shows a LOFAR spectrogram example of the L580401 data set. It shows the original spectrogram on the left, with time on the vertical axes and frequency on the horizontal axis. Visible is intermittent RFI and frequency dependent gain anomalies. The right figure shows a spectrogram reconstruction after wavelet filtering that removed (separated) vertical stripes. We also tested 2D Fourier transform filters and singular value decomposition (SVD) filters, and applied these on 53 spectrograms with 'typical' features.
We found that eight of our developed filters were able to detect at least 85% of the features existing in our (small) dataset of preselected examples. Although the dataset used is small, we think these results are very promising and that this approach can lead to useful tooling to automatically generate data quality reports.
