Can My Landlord Prohibit My Boyfriend From Coming On The Property To Visit Me?

I live on property at work as a Horse trainer/nanny for my boss. My apartment is away from the house and over one of the barns. I do pay rent for my apartment but I don’t have any sort of signed rental agreement. I told my landlord that my boyfriend would be visiting and riding one of my horses (not staying overnight), and he told me that there are “no boyfriends allowed on the property”. He told me that I was told this from the beginning in my contract, which it is not and I was not told. In an email that is not my contract he states that no unannounced personal visitors are allowed and prearranged visitors close friends and family can be allowed. There’s nothing about no visitors allowed, in fact he let my best friend stay with me in November and then during this convo about my boyfriend said he may have been too lenient about my best friend staying. He has told me I have no tenant rights as a “live in nanny” but as far as I’m aware you don’t pay rent and you’re classified as a W2 which he gave me a 1099 and has me classified as an independent contractor even though I don’t have my own nannying business and treats me as an employee.

Where I am paying rent and have no rental agreement other than what I pay for rent, can he legally prevent my boyfriend from visiting me?