"I felt discriminated against," she said. "I was appalled and angry."
Mrs Penn-Wagland was sitting in the back row of court to support a family member, who was due to appear.
She decided to feed her two-year-old daughter Chayelynne and said she was covered.
Get in front of tomorrow's news for FREE
Journalism for the curious Australian across politics, business, culture and opinion.
READ NOW"No one was looking at me," Mrs Penn-Wagland said.
"I was trying to keep her quiet and Chayelynne would have been asleep, but she (a security guard) interrupted me.
"She asked if she could see me outside.
"Then she said I might be more comfortable if I fed Chayelynne in another room.
"I was more than comfortable sitting where I was."
Mrs Penn Wagland said she had every right to feed her child in the public gallery and would do it again.
The Department of the Attorney General was advised that a Serco guard spoke to a woman in court.
"The Serco guard offered the woman a room to breastfeed her child in privacy," a spokeswoman said.
"The woman was advised that she would be recalled to court for the video link that she was interested in.
"The woman attended the video link."
The Director of Magistrates Courts will meet Serco and court staff and the magistrate at Mandurah to emphasise that the Department is supportive of breastfeeding mothers' rights to breastfeed in the court.