Do you want a new, private forum?

[viscousmemories]

A while back dantonac started a thread illuminating the fact that this site is viewable by the general public, and that all of the content here (with the exception of the journals) is indexed by search engines.

At the time livius and I explained that yes, that is the nature of public discussion boards and the FF is a public discussion board. However, although it should have been obvious to us at the time, we didn't even think of asking you all if you'd be interested in having a single new forum (similar to Watering Hole or Lifestyle, for example) that isn't publicly viewable or indexed by the search engines.

So now we're asking. If you select other, please explain in detail.

Thanks.

[beyelzu]

I vote to see the poll before I vote on the poll. :D

[viscousmemories]

lol. Yeah it's weird that you have to post the thread before you can post the poll. :?

[Petra]

You know, I really don't mind either way, but I can see the value in it for people who need to talk about things that they might not wish to be viewed by non-members.

An example might be reprise and what's going on with her right now. Is she not returning to talk to us more about what's happening because it's too public? I don't know, of course, but it may be something to consider.

[beyelzu]

I think I would like a private forum as well, but I am unsure as to how to set it up.

I would hate to see all threads get started there, but something for personal shit might be nice.


I particularly like idea that it wouldnt be indexed by search engines, not that I give a fuck about others would think about the stuff that I have posted here, but other people might.

[viscousmemories]

To be clear, Bey, it would be a new forum on the main index. Like under "Watering Hole" for example, there would be "Members Watering Hole" or whatever. Then people could start their threads there and/or have us move any existing threads into there.

[livius drusus]

We were thinking something along the lines of a private lives forum including a handful of threads from Watering Hole and Lifestyle which seem like they might benefit from a more intimate environment, like the thread luna mentioned. It seems unlikely that we'd see all or even a disproportionate amount of threads get started there.

[beyelzu]

I think it is an excellent idea now that I have further information. One that I can support whole heartedly. Considering, that we are a fairly tight knit community, I think we would all benefit from having a more private place to talk about personal issues. Not that I have any of course. :D

[seebs]

Some sites do a forum only visible to Some Members, but I don't think I'd suggest that in this case; doesn't fit the FF feel.

[Petra]

Some sites do a forum only visible to Some Members, but I don't think I'd suggest that in this case; doesn't fit the FF feel.

Hell, no.

I'm not into that, at all.

[ApostateAbe]

Heathen Hangout is already such a forum. If any of us want a private forum, they can go there. I prefer to keep this place public. I refer to this forum in other forums.

[seebs]

HH does not serve the same crowd this place does. Helen and I, for instance, can't go there.

I am actually not that concerned; I don't spend much time posting about my private life. The stuff that people might think is a big deal (e.g., possible future polygamy) is already out enough that I'm not much concerned.

[livius drusus]

Heathen Hangout is already such a forum. If any of us want a private forum, they can go there. I prefer to keep this place public. I refer to this forum in other forums.

Sorry for not being clear, Abe. The problem is that "forum" means the whole board as well as a single subcategory of discussion. In this case, however, we're suggesting adding a single subcategory of discussion like "Lifestyle" or "Arts & Literature" which would be hidden from casual readership but viewable to all (no private clubs, seebs, you scamp) registered members. The board as you know it now would remain public.

[Dingfod]

HH does not serve the same crowd this place does. Helen and I, for instance, can't go there.Well, actually you could, but it would involve lying and deceit just to get past the sign-in page. So, its not that you can't go there, it's that you won't compromise your integrity to do so. That is to be admired, not denigrated.

[ceptimus]

I voted 'other - please explain' as there was no option for 'don't care'. My explanation is that I don't care one way or the other. :)

[livius drusus]

:LMAO:

[viscousmemories]

I've modified the wording in the OP to clarify the question.

[seebs]

Well, actually you could, but it would involve lying and deceit just to get past the sign-in page. So, its not that you can't go there, it's that you won't compromise your integrity to do so. That is to be admired, not denigrated.

Anything that went there wouldn't be me.

Er. Are we back to the deep philosophical questions here?

[HelenM]

I don't have strong feelings either way. Whatever you decide is fine with me.

Helen

[Corona688]

I don't like it very much. Remember, it was a private fora on iidb that got the whole ball rolling when it turned into a den of vipers and got closed; I like this public board far better. Creating a semi-private section will create problems you might not realize.

Right now, there are no hard bannings. There's no point -- nothing we do can prevent people from reading what we write here anyway. Creating a member-viewable-only forum will invent this need -- after all, why bother having it members-viewable only if we don't care who our members are?

If you don't want this site indexed by search engines, just mess with the robots.txt file. The whole point of that system is to exclude robots without excluding people.

[wade-w]

I don't like it very much. Remember, it was a private fora on iidb that got the whole ball rolling here, and honestly, that place was never much fun. I like this public board far better. Creating a semi-private section will create problems you might not realize.


They aren't talking about the old FPF at IIDB, Corona. What is being proposed is like The Lounge there now. Nobody would be excluded, you'd just have to be logged in to see it.

[Corona688]

I don't like it very much. Remember, it was a private fora on iidb that got the whole ball rolling here, and honestly, that place was never much fun. I like this public board far better. Creating a semi-private section will create problems you might not realize.


They aren't talking about the old FPF at IIDB, Corona. What is being proposed is like The Lounge there now. Nobody would be excluded, you'd just have to be logged in to see it. I know. That's not the point. Members can see all the fora already. Having it members only will give us the ability to exclude specific people, and absolutely no advantages beyond that. Why do we need that? Who do we want to exclude?

[seebs]

I think the main entities to exclude are search engines.

In other words, I might not want the search phrase "seebs adultery" to turn up a serious discussion of whether or not polygamy is adultery. (Actually, I don't give a fuck, but it's a good example, better yet because it's a realistic one, but the "story" is already out.)

So, there's reason to want some pages indexed by search engines, and others not.

[Corona688]

I think the main entities to exclude are search engines.

In other words, I might not want the search phrase "seebs adultery" to turn up a serious discussion of whether or not polygamy is adultery. (Actually, I don't give a fuck, but it's a good example, better yet because it's a realistic one, but the "story" is already out.)

So, there's reason to want some pages indexed by search engines, and others not. The poll implies that, to keep search engines out, we need a private forum, but that ain't true. It's possible to keep web robots out without modifying the forum at all -- just change some stuff in FF's robots.txt file (http://freethought-forum.com/robots.txt). That's what it's there for, in fact. :cool: As you can see, it excludes a bunch of stuff from crawlers already.

(more information on robots.txt available here (http://www.robotstxt.org/wc/exclusion.html))

[livius drusus]

vm has made some modifications to robots.txt file already, so I'll leave that branch of the discussion to y'all, but I did want to point out that people might want a private forum away from both search engines and actual rl people.

[Corona688]

vm has made some modifications to robots.txt file already, so I'll leave that branch of the discussion to y'all, but I did want to point out that people might want a private forum away from both search engines and actual rl people. It comes back to this, then. Having it members-only will give us the ability to exclude specific people, and no other advantages. Why do we need that? Who do we want to exclude?

[livius drusus]

It's a very good point, Corona. My answer to both your questions right now is I don't know. I'm in the rather enjoyable position of having no position, so I really appreciate your challenging the OP and get some discussion going. Thank you. :)

[viscousmemories]

Having it members-only will give us the ability to exclude specific people, and no other advantages. Why do we need that? Who do we want to exclude?
Thanks for the input, Corona. You make good points.

As mentioned the idea of a new forum would be to prevent spiders and non-members from viewing the content therein. You suggested modifying the robots.txt file to exclude spiders. That could be a problem since we don't want to exclude spiders from all the fora and I don't believe there is any way to exclude them on a forum-by-forum basis.

Your point about bannings is interesting too, I hadn't thought of that angle. We weren't considering any changes to the rules (http://www.freethought-forum.com/forum/showthread.php?t=49), so as things are the only way someone would be banned from the hidden forum would be if they were banned from the entire site.

So if it's possible to restrict spiders from a single forum, and people won't be banned from viewing that forum unless they misbehave grossly, what would be the point of restricting it to registered members? To me the only obvious benefit would be that people would be forced to register before they could view it, thereby limiting the eavesdropping of passerby.

[Adora]

I don't see the point to it. What have we got to hide?

[Ex-zombie]

I don't like the idea of a private forum at all. I left a forum that decided to have a members only posting area. It smacks of an elitism that I do not care for. However, this is not my forum. I doubt I would leave if that is what the other members want. I like the expression of thought and emotion here.

[SharonDee]

I don't get the elitist charge. It's not like it would be that hard to access such a forum. You register--a rather simple process--and poof, you're in.

On the other hand, if it's that easy to circumvent then why bother at all? :shrug:

[Corona688]

I don't get the elitist charge. It's not like it would be that hard to access such a forum. You register--a rather simple process--and poof, you're in. As long as registration is easy, yes. But in a members-only board, when it's decided to make someone go away, bingo, we can make them go away. Why do we need that, when we already have perfectly good systems for protecting privater things from prying eyes? (pm's, chat, articles, robots.txt)

It's more about principle. This is about as open as a board can get; no moderation, no bannings, no exclusion. So far it seems to work. I'd like a better reason than "why not" to tighten things up.

[AspenMama]

I'd like a private forum where everyone tells me how smart and gorgeous I am...

:D

Is the idea to have a support forum for some serious problems? I can see an advantage to folks being able to post private stuff without the prying eyes of their significant other or the general public-- and it's nice to be able to solicit a variety of opinions at times-- hence the advantage over PM's. I think it would be helpful as a support forum.

[viscousmemories]

I'd like a private forum where everyone tells me how smart and gorgeous I am...

:D
Wouldn't we all? :)

Is the idea to have a support forum for some serious problems? I can see an advantage to folks being able to post private stuff without the prying eyes of their significant other or the general public-- and it's nice to be able to solicit a variety of opinions at times-- hence the advantage over PM's. I think it would be helpful as a support forum.
I suspect that the others who have expressed an interest in it probably see that as a potential value. Personally I'm of two minds on that aspect. On the one hand it would seem that in the interest of community building we would offer an area for people to seek support.

On the other hand I personally feel that a discussion board with unrestricted membership is an inappropriate (as in not likely to be very effective) venue to seek support on sensitive, personal issues. Which isn't to say that I am at all opposed to people doing so, but I'm afraid that some might see this board as an alternative to something more beneficial like seeking counselling or something.

[HelenM]

I'd like a private forum where everyone tells me how smart and gorgeous I am...

:D

Wouldn't you rather have that posted publically? :D

Helen

[AspenMama]

I'd like a private forum where everyone tells me how smart and gorgeous I am...

:D

Wouldn't you rather have that posted publically? :D

Helen

:yup:

[AspenMama]

Personally I'm of two minds on that aspect. On the one hand it would seem that in the interest of community building we would offer an area for people to seek support.

On the other hand I personally feel that a discussion board with unrestricted membership is an inappropriate (as in not likely to be very effective) venue to seek support on sensitive, personal issues. Which isn't to say that I am at all opposed to people doing so, but I'm afraid that some might see this board as an alternative to something more beneficial like seeking counselling or something.
I hear you. And this second point is something that I always consider when replying to folks over at II-- but I think it is only in a few circumstances where someone should truly seek professional help-- I'd have to say in my past experience as a moderator of support forums that the majority of issues are not overwhelmingly serious-- and often folks are already seeking counseling and just asking for some supplemental advice.

[livius drusus]

Just to report on my thinking right now, Corona's point about the fundamental purposelessness of a private forum here is basically where I stand.

[dave_a]

vm has made some modifications to robots.txt file already, so I'll leave that branch of the discussion to y'all, but I did want to point out that people might want a private forum away from both search engines and actual rl people.

I would like to see either a private forum viewable only to registered members, OR the entire forum made imposible for search engines to cache.

The reason I brought this up some time ago is that I have used dantonac for a long time at a wide variety of forums. It seems that some people will do google searches on one's username at a forum to snoop out what that person is about.

I am not sure, but there was a forum I was banned on recently about gardening. I didn't understand what I had done to get banned and trying to discuss with the admin just got me treated very rudely. The guy is a fundi christian. It makes me wonder if he didn't come across some of my IIDB posts or something and got predjudiced against me. All speculation of course.

Also, I don't particularly care to have people know I am atheist unless I tell them. Here I couldn't care less, but some other sites it can be problematic.

Of course I could use a different username for every forum I participate in, but...

Then there was this guy who interviewed for a job where I work. His resume looked good. The boss asked me to google his email address and his username to see what turned up. Well, the guy had a site dedicated to being dominated by females. The boss got squicked by it and the guy didn't get the job.

My real issue is search engines being able to index content, I am not concerned about someone surfing this board and reading what is here, I just don't like people reading what I wrote by doing google searches.

As an analogy it is kind of like if we all rented a hall and the hall was open to the public. We are all sitting around talking comfortably and not really concerned that passersby might overhear some of what we say. Then we find out the hall has a camera in it and is broadcasting our words on 24x7 worldwide TV.

[Corona688]

I would like to see either a private forum viewable only to registered members, OR the entire forum made imposible for search engines to cache.

The reason I brought this up some time ago is that I have used dantonac for a long time at a wide variety of forums. It seems that some people will do google searches on one's username at a forum to snoop out what that person is about. And to clarify my position, I agree that excluding search engines is a good idea. I might've argued otherwise before, but I'm allowed to change my mind. :) Less worry for us, less work for google. I would leave the gateway, if possible; it's nice to have something pointing to this forum in google, but we certainly don't need it archiving the contents.

[livius drusus]

I'm bumping this cause dantonac asked whatever came of this discussion and I realized we had never resolved anything.

My thoughts on it right now are as follows: I think Corona's argument against creating a new private forum is a good one. A private forum wouldn't stop search engines from indexing the rest of the site anyway, so it would do little to alleviate dantonac's concerns unless he posted solely in the private forum.

I really don't want this forum to be entirely unindexed. I don't understand the point of a public forum that can't be found by someone randomly searching for, say, "culturally relevant metaphor". In fact, I think it's really awesome that a keyword search might turn up a thread of ours and I don't see the benefit of cutting off that potential source of new members.

I Googled dantonac (http://www.google.com/search?hl=en&q=dantonac&btnG=Google+Search) and two FF threads come up right quick. Neither of them are compromising wrt to atheism or anything else (that I can see), and even if we did stop all indexing, they'd still be there.

Your analogy doesn't work for me, dantonac, because by default a website is more of a tv studio than a public hall. Our words are de facto broadcast all over the world unless we make a point of shutting off the equipment that is always running in the studio. The only difference between this forum and IIDB or any of the other public forums you participate on is that we actually tell you when the spiders are here and what they're checking out. On the internet, the assumption is that everything you say is broadcast. Keeping a lid on that is up the individuals, I think, unless a forum has a specifically private mandate like HH does.

As for the robots.txt issue, Corona, do you know if it is possible to block crawls of certain forums only? Could be put something in robots.txt that would stop Yahoo from Slurping "Lifestyle", say, but not "Food & Drink"? I'd be willing to work with something like that if it were possible.

[wade-w]

FWIW, I found this place via google. If the site wasn't indexed, I wouldn't know it was here even now. Make of that what you will, I know some would consider that as an argument against allowing the site to be indexed.

[Corona688]

As for the robots.txt issue, Corona, do you know if it is possible to block crawls of certain forums only? Could be put something in robots.txt that would stop Yahoo from Slurping "Lifestyle", say, but not "Food & Drink"? I'd be willing to work with something like that if it were possible. I think it is possible. A line like
Disallow: http://www.freethought-forum.com/forum/forumdisplay.php?f=38 should bar it from viewing the "food and drink" forum but none other. If someone links to a thread inside "food and drink", it might get indexed, but it won't be able to view the main index. If it's possible, a more surefire way to do it would be to relocate these forums to a different subdirectory in your server. Then you could disallow that whole directory but good.

For testing purposes, I can deploy gnu wget if you like. It's a polite batch-downloader that obeys robots.txt rules when used as a spider.

edit -- ack. The unmangled URL should be put in robots.txt, not the shortened one it shows.

[viscousmemories]

As for the robots.txt issue, Corona, do you know if it is possible to block crawls of certain forums only?
I think it is possible.
Actually I don't think it is. Everything I've been able to find about the Robots standard says you can only exclude files and directories. I haven't found anything that says you can exclude dynamic content. I'll keep my eyes open, though, and of course if you or anyone else can find more info about this I'd appreciate it. This (http://www.robotstxt.org/wc/faq.html) seems to be the definitive source for info on the standard.

If it's possible, a more surefire way to do it would be to relocate these forums to a different subdirectory in your server. Then you could disallow that whole directory but good.
Unfortunately that's not possible either. All the dynamic forum information (and most of the static) is stored in the database, not in the directory structure. :(

At this point (based on all the contributions in this thread and without any new info) we're not going to make any changes to the way the spiders are handled. It seems to us that the arguments against it and/or neutral - combined with the technical difficulty of implementing any change - outweigh the potential benefits right now. We're definitely open to reconsidering it in the future, though.

As an aside, we do currently disallow the spiders from user profiles, the memberlist, and many other areas of the site. The only things currently indexed are posts.

[Corona688]

As for the robots.txt issue, Corona, do you know if it is possible to block crawls of certain forums only?
I think it is possible.
Actually I don't think it is. Everything I've been able to find about the Robots standard says you can only exclude files and directories. I haven't found anything that says you can exclude dynamic content. The thing is, I don't think there is a difference. Web clients don't know it's dynamic content -- HTTP servers don't know either. They're completely stateless... All they see is the filename. If the filename's constant, I think you can block it. Have a look at http://www.google.ca/robots.txt , it's got some odd stuff in it.

[viscousmemories]

The thing is, I don't think there is a difference. Web clients don't know it's dynamic content -- HTTP servers don't know either. They're completely stateless... All they see is the filename. If the filename's constant, I think you can block it.
That's the problem, though. The filename isn't constant.

If we take your example:

http://www.freethought-forum.com/forum/forumdisplay.php?f=38

It's true that would restrict the spiders from indexing the list of threads in forum #38. However, all the content in the individual threads would still be indexed because they are dynamically generated using something like this code:

http://www.freethought-forum.com/forum/showthread.php?t=1136

As you can see, only a thread ID is referenced in that command. So the only way to restrict spiders from reading that would be to disallow showthread.php (which would prevent the reading of all threads) or disallow each thread ID explicitly (as above), which is obviously impractical.

As far as I can tell there's no easy way around that...

[seebs]

I hadn't thought about the implications of the /tXXXXX URLs.

Given that, I think a forum viewable only by logged-in users may be the best way to allow bots to pick up some but not all of the site...?

[livius drusus]

Indeed, which brings us back to the OP. At this point, keeping bots out of one forum doesn't seem like reason enough to me to create a registered users only forum.

[bobeh]

I voted no opinion...as far as this forum is concerned.

But I did, out of curiosity, try to google my username. 4 hits in the first 2 pages. Not good in this case...a couple of sex related threads...etc. Anything I say in the forum is fine...for this group of people. But I have some fundy relatives, including a spouse, who might take some of it in the wrong way I imagine. So...my conclusion. Either I register a much more common nick (and therefore more likely to be lost in the noise on searches) or I be more careful what I say...here and at II. A little bit more careful...not too much. So there is my concern.

Not saying that a closed forum is the answer though.

[livius drusus]

Being careful if you have someone Googling you is probably wise nomatter where you're registered. We'd be glad to hook you up with any name you'd like, b.

[Clutch Munny]

Even supposing the search-bots can't be finessed without it, I'd rather not have a private area. Everything wide-open is just a... feel that I like about this place.

My two cents (Canadian).

[AspenMama]

Seems the tide has turned against having a private forum-- although the poll shows over half who voted in favor of it. Perhaps a new poll in light of the information presented here?

[livius drusus]

Hmm... Were you thinking we'd delete this one and replace with a new one or start a new thread, Aspen?

[AspenMama]

Well I suppose my suggestion would be to start a new poll and a new thread including the summary of techie information in layman's terms for those of us not quite up to par with the smart nerds occupying this space.

[livius drusus]

Excellent idea, Aspen. I'll definitely do that. :thankee:

[Bella]

I voted "other" - as long as all registered members have access to the private forum, I'm OK with it. Just as long as it doesn't turn into the whole IIDB debacle, where people are nominated, have to receive votes, et cetera. IIRC, that was a HUUUUUUGGGEE problem.

[seebs]

Actually, Bree's emphasis on the question of all registered users raises an interesting meta-question.

What about banned users? I mean, I don't know that we anticipate any. But let's say we did ban someone. Should that user still have access to a hypothetical Top Sekrit clubroom?

[dave_a]

Seems the tide has turned against having a private forum-- although the poll shows over half who voted in favor of it. Perhaps a new poll in light of the information presented here?

I dunno, I still prefer the posts to not be googleable. I do understand it might reduce this site's exposure to the world and I also understand there may be technical issues making this difficult to pull off.

I respect those facts and am certainly willing to do without a private forum or not banning the search spiders, but that doesn't change my actual preference at all.

In other words I want what I want. I can understand and accept that there are valid reasons why I can't have what I want, but I still want it.

Kind of like my kid wants every toy advertised on TV for Christmas. He isn't going to get all that and he understands why, but that doesn't stop him from wanting it.

[AspenMama]

dave_a I've also not changed my mind.