🔒 Locked

[.net] Beginner Multithreading Question

Started by ArchG Jan 28, 2006 at 12:39 PM 4 replies 750+ views

Original Post

Jan 28, 2006 at 12:39 PM

Hello. I'm curious if what I want to do is possible to do with Mult-threading, if it's a good time for mult-threading and maybe some insights on how to do it in MultiThreading.. ------------- I will have an array of about 10,000 objects called cLink..here is sort of the contents of cLink


class cLink
{
public string Title;
public string Address;
public string StatusCode;
...
};

; and I have one

 CheckLink(cLink LinkToCheck)

function that checks the status of the link, and also saves the bad links into an XML file with an XML Writer. -------------- Right now this takes hours and hours to complete...and it's something that we do almost everyday...I just thought maybe having 10 different threads running at the same time would increse the overall efficiency. That way if one link times out...instead of wasting 30 seconds on it, the program will keep on chuggin. Thanks a Bunch ArchG

ArchG

Author

ArchG

Jan 28, 2006 at 1:21 PM

What if I put all those 10,000 objects of cLink, into an XML File...then did something like this

in my CheckLink Function

public void CheckLink(){   lock (this)   {      cLink Link = ReadALink("Links.XML"); // Load Link into memory      DeleteThatLinkFromXMLFile("Links.XML"); //delete it from disk   }   Link = GoToWebAndCheckLinkStatusCode(Link)   if (Link.StatusCode != 200) //BAD LINK   {        lock (this)       {          SaveToXMLFile(Link, "BadLinks.XML");       }   }}

I'm not sure if that's how it's done or not...but it seemed like a good idea when I thought of it. Would something like that work? I would try it and test it out..but this project is a rather big one and would take more than just a couple lines of code to get it going..(the example is pretty cut down just to illustrate what i'm trying to do).

Again any help is appreciated and you will be rewarded via a small ratings increase ;-)

Promit

Mod

1,082

Promit

Jan 28, 2006 at 1:24 PM

First you have to determine which part of the process is slow, although at a guess I'd say it's the file IO? Depends on how things are getting buffered internally, I guess. You could try using asynchronous IO (look up BeginWrite), or also writing them into a MemoryStream instead and serializing that to disk at the end. Multithreading will help if you're on a multiprocessor system, otherwise you're better off just doing async IO.

DrGUI

Jan 28, 2006 at 3:38 PM

He said it was the waiting for the links to verify / timing out, so he's not IO bound, it's internet-bound.

Your code looks good; just make sure that the functions you're using in your GoToWebAndCheckLinkStatusCode are multithreading-safe (should say in the docs). I guess they are but best be safe!

Promit

Mod

1,082

Promit

Jan 28, 2006 at 6:03 PM

Oh...internet links. I was wondering what the heck his "links" were.

In that case, yes, multithread it.

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.