I've tried that a few times. In fact, where I work it is kind of an occupational hazard. Every eclipse or transit or whatever there's a crowd of people and telescopes in the mall down the hill from my office. I even have a couple telescopes* myself, including a nice 8-inch CAT, with GOTO, GPS, wifi interface to my phone/computer, and more optics and accessories than I care to admit to.
The problem is that it has been hard to find such a group nearby where I don't know anyone from work. I'm a fix-it guy at work, and socializing with work-friends always seems to wind up being about work.. specifically about getting me to fix something at work. Or discussing how to fix something at work. Or the best way to do something at work. Etc. Etc. Don't get me wrong. I'm happy to help in a work context, and I do enjoy my work. (At least when it isn't making me crazy.) But not so much that I want to give it my social life as well. And yes, I've tried to make rules about not doing or discussing work outside of the office. It doesn't really help. I'm happy to discuss pretty much anything having to do with anything more than 100km up, but that's a slippery slope. It seems at some point there's always a "Hey, that reminds me; I know you don't like discussing work but..." moment. As a result I find I'd rather associate with non-work people, even if meeting such is difficult.
That said, for a while now I've also been looking for a good solar power group locally. Maybe I could try that again. I have solar panels on the roof and I'd like to find like-minded people to get together with and compare notes. It is a lot easier & convenient to find people online though, especially with my work schedule. Recently my boss even has/had me working shifts again. Hard to find time for parties or club meetings when you'r'e working graveyard one day, then swings the next.
*"Frangere" and "Cat"...