Just apply a source filter (maybe other filters too). If there is a category that doesn't have events when those filters are applied, it still shows up but empty. I feel like this didn't used to do this but now does. I could be misremembering and this should be a feature request instead.