Brother Is Saying Me And My Mom Have To Leave The House As His Probation Officer Does A Home Search/visit

To start out the house is owned by my grandparents. Me, my mom and my brother just live there. My brother is on probation and has that house down as his residence. He is telling my grandparents to kick me and my mom out of the house because his officer told him we aren't allowed there as they visit, which I've never heard of being done before. He also is saying they are going to search ALL of our rooms even tho he doesnt occupy said rooms and never has access to them. Are we actually allowed to be forced out for no reason or is my brother lying to get us to leave? I have reason to believe he's lying because he's known to do so before but I need some advice on the situation

Update: My mom Contacted his PO and was told that they have no reason to search our rooms if my brother isn't sleeping or has access to them. Which means they know we all love together which is strange. She refused to give us more because apparently its none of our business, which is strange cause if my brother ISN'T lying then it involves us as well. Still completely left in the dark