Driveway/right Of Way Dispute

This is probably going to be long & hard to explain.. but here we go. My husband & I own a home where we live with our 3 small children. The house is about 125 meters back off the road. We access the house by a long gravel driveway. The gravel driveway is not on property we technically own but the driveway leads to our house & our house only. Our neighbors own the property the driveway sits on. Said neighbors now want to build a fence around their entire property and have informed us they will not be putting a gate at the driveway to allow us access. My husband very nicely asked if they would consider building the fence on the other side of the driveway so we could still have access and the neighbor used the excuse that he had already had somebody come out to look at the land and give him a quote and that would just entail too much work to have them come back out and look at it all again. Basically he just said “no” and it seems like because he just doesn’t want to. There is a fence and a small tree line on the other side of the driveway that blocks us off from a rather trashy trailer court. I am not being judgmental, there have been many instances of emergency personnel showing up due to drug related incidents as well as domestic incidents. There is a small little road in that trailer court on the opposite side of said fence that you can drive down… and we technically have a wooden fence that you can open when you get to our house parallel to our attached garage… it has been at least 20 years since that “gate” has been used. But the neighbor is saying he’s OK with blocking us out from the driveway because we can use this to access our home. However, with our house being so far off the road, it’s already hard enough to find so I am worried about in the event of an emergency, emergency personnel, having easy access to our home. Not to mention the fact that it’s going to be a big pain in the ass to open and close that gate every time we come and go. We also have a boat and use a trailer frequently that we will not be able to get in and out of this “gate“. Lastly, we have a detached garage that is a little further up our property, right at our property line that sits right off of the driveway they are trying to block us access to, and it will be nearly impossible, if possible at all to use the detached garage when the neighbors build their fence. Backstory is that back in the 80s, two family members built our home and the neighbors home with the understanding that the people that lived in our house would always have the right of way via the driveway. Even when my husband’s Nana bought the house that we now live in, she was granted some sort of legal paperwork stating that the driveway could not be blocked. That paperwork is obviously not in our name, but I’m just wondering what, if any, type of legal rights we have to the driveway?