Agents always leave when busy

Is it me or do the agents always take a break right when it gets busy? Like really? is there anyone we could report this too? I mean this cant be approved by supervisors