Massive Idea

I don't want to think about how many gigs of data that'd be.

Actually, That's kinda interesting. How big is GDnet?

Quote:

Original post by C-Junkie How big is GDnet?

very big [grin]

prolly guess about 5 or 6 TB.

5 or 6 TB?!?!?!

You've got to be kidding... let us do a simple reasonability test.

The largest of 30+ forums appears to be the Lounge which is 657,132 but most are only in the order of magnitude of 10,000 while some are even only a few hundred posts.

With the generous estimates of 50,000 posts per forum, 30 forums, (an even more generous) 10,000 characters per post, and 2 bytes per character:

50,000 * 30 * 10000 * 2 = 30,000,000,000 bytes

Google calculator says that equal to 27.9396772 gigabytes or 0.0272848411 terabytes

0.0272848411 is not reasonably near 5 or 6...

Hell.. even at 500,000 posts per forum:

http://www.google.com/search?hl=en&lr=&safe=off&c2coff=1&q=%28500%2C000+*+30+*+10000+*+2%29+bytes+in+terabytes

From a web developer's point of view I'd say that the proposal is a little pie-in-the-sky.

Generally, web-based content (forums, etc) isn't designed for use outside of the web site it's used in. Let's face it, there's a hell of a lot of content on the site and to be honest it wouldn't all useful to you (eg: 99% GDNet lounge, Help Wanted, Your Announcements threads, etc) The effort that goes into creating the redistributable package of it would be disproportionate to the value you'd get out as an end user. In order to keep the high value, it's likely that a group would need to be formed to select, edit and merge sections of content that would be useful; this task in itself would likely be expensive and time-consuming. I'm also unsure of the legal implications involved of creating a distributable package of what are, essentially, third-party contributions.

I've wondered myself in the past about a downloadable articles collection and such, so don't think I'm panning the idea. Whilst it seems like a good idea I just really don't think it's feasible because of the sheer amount of work required to pull off.

Quote:

Original post by evolutional From a web developer's point of view I'd say that the proposal is a little pie-in-the-sky.

Seriously, how big is the database?

Quote:

Generally, web-based content (forums, etc) isn't designed for use outside of the web site it's used in. Let's face it, there's a hell of a lot of content on the site and to be honest it wouldn't all useful to you (eg: 99% GDNet lounge, Help Wanted, Your Announcements threads, etc) The effort that goes into creating the redistributable package of it would be disproportionate to the value you'd get out as an end user. In order to keep the high value, it's likely that a group would need to be formed to select, edit and merge sections of content that would be useful; this task in itself would likely be expensive and time-consuming.

Nobody's looking for a finished product. If I ordered one of these things, I'd even be disappointed if it wasn't just a gigantic DB dump. I want to SELECT posts from it!@

Quote:

I'm also unsure of the legal implications involved of creating a distributable package of what are, essentially, third-party contributions.

Obviously not legal advice, but there's already an implicit contract between posters that lets you put their words up on a website and make money from advertising. I'm certain that a not-for-profit database copy would be legal, and I'm sure even a for-profit one would still be legal.

Quote:

Original post by evolutional Generally, web-based content (forums, etc) isn't designed for use outside of the web site it's used in. Let's face it, there's a hell of a lot of content on the site and to be honest it wouldn't all useful to you (eg: 99% GDNet lounge, Help Wanted, Your Announcements threads, etc)

I dunno, there are some threads that just make your burst out laughing [lol]. Ok ok seriously, I agree, it would be a lot of work and most of the end content in those forums would be of little value. But all the refrences in the programming forums could be a little benefitial. I agree that a raw product would be nice compared to a finished product, simply because I think a finished product would be inhumanely impossible!

What about a GDev script or something to make saving multiple posts easy. Maybe that'd make a potential future GDNet++ feature [wink]. Right now, it is easy to save posts as ".mht" and its all there, but when you want to do a lot, you have to go page by page. Still probabally infeasble but just some thoughts. Maybe I should try making me a program to do it for me! Weeeeeee. Ok all thanks for the feedback.

[Edited by - Drew_Benton on May 25, 2005 9:42:06 PM]

The database is a little over 10 gigs.

---
Michael Tanczos

Quote:

Original post by C-Junkie Obviously not legal advice, but there's already an implicit contract between posters that lets you put their words up on a website and make money from advertising. I'm certain that a not-for-profit database copy would be legal, and I'm sure even a for-profit one would still be legal.

It doesn't really work like that. When you post, you're giving GDNet (non-exclusive) license to use your work for the purpose of maintaining a public forum. You are at no point agreeing to give GDNet the right to publish your work (see sections 3(c), 3(d) under "Customer Obligations" of the site's terms of use). The act of posting something in a public forum does not deprive you of the right to profit from it.

Do what I do, ever since I started programming i always saved webpages, File->Save AS ->web Archive, the reason for this to start with was beause I didn't have my own online computer, so I would save the webpages and burn to disk. My tutorials folder -along with all the webpages I've saved is between 1/2 - 1Gb, now with Google desktop, I can often find things on my own computer.

Quote:

Original post by Michael Tanczos The database is a little over 10 gigs. --- Michael Tanczos

Just out of curiosity, do you purge/backup any of the older posts/data? or is that 10 gigs the whole of GDNet's existance?

Having had a quick browse through the profiles, the oldest is around summer 1999... meaning 10gb for 5yrs worth of posts?

Cheers,
Jack