A group of worshippers in Uganda have vowed never to set foot in a church in the country's Kayunga District after a couple was found having sex at the altar on Tuesday.

The couple, a married 23-year-old male Catholic and his recently divorced Muslim lover, were reportedly caught pants down by the man's wife who raised the alarm, drawing the attention of nearby residents.

"The two were arrested amid condemnation from residents. A headgear belonging to the woman and a T-shirt for the man are being kept at the Local Council's (LC) residence as exhibits," Bugonya LC1 chairman George William Kanda is quoted as saying by the Monitor.

"They accessed the church through the windows since the doors were shut."

Kanda added that following the incident, the culprits were frogmarched to his office by irate residents but he later set them free after filing a promiscuity case against them.

By Thursday, the man had gone into hiding; his current whereabouts remain unknown.

"A case does not rot. He will answer for his sin anytime be comes back," said Kanda.

The faithful, from the Bugonya Church of Uganda, now want church leaders to organise a cleansing ceremony to sanctify the premises.

Church leaders have since urged their flock to fast for an unknown number of days following the incident which they labeled as shocking and sinful.