Burke said the three men being removed from Australia had failed a character test, and their bridging visas were cancelled after Nauru issued long-term resettlement visas on Saturday.
He said they would not be deported within the next seven days, but it would occur "as soon as possible" once arrangements are made.
"When somebody has come and treated Australians in a way that shows an appalling character, their visas do get cancelled and when their visas are cancelled, they should leave," he told reporters on Sunday.
He signalled Nauru could take others from the cohort, saying the government there "had described these three visas as the first three".