A gay couple in Wisconsin is crying foul after being told to take down the rainbow flag they’d been flying outside their apartment.

Kevin Kollman and Merle Malterer have lived at the Country Oaks Apartments in Oak Creek Wisconsin, and say they’ve always had exterior decorations of various kinds.

But when they put up a rainbow spin on the American flag, they received a notice from the property owner telling them they had five days to take the flag down or they would be evicted.

“Personally, I feel like we’re being kind of singled out here because it’s what it is,” Kollman told their local NBC2 news.

The property owner told local media the issue wasn’t that the flag was a symbol of LGBT pride, but rather that it was a flag at all — that anyone flying a flag at the apartment complex would be in breach of their lease.

But other residents had flags displayed outside of their apartments at the time of the eviction notice, and numerous flags can be seen throughout the complex in the video NBC2 took after being contacted by Kollman and Malterer.

“I’ve never been told to take them down,” neighbor Ken Miller said of his own non-rainbow American flag. “I’ve been here for three years, and I’ve had these flags up for at least two years and I’ve never had any issues with the flags.”

The couple have no intention of taking their own flag down, and are considering legal action instead.

“It’s part of my house, it’s part of who I am,” said Kollman. “My neighbors don’t care. Why should he?”