Skip to main content
GameDev.net gamedev.net
🔒 Locked

Replication RAID functionality (symlinks)

Started by jbadams Mar 18, 2010 at 10:22 AM 1 replies 1.4k views
Original Post
jbadams
jbadams
I'm trying to solve an odd little problem and would like some feedback on my proposed solution. Given a large collection of files on multiple disk drives, I need to present a single point-of-access for the files as if they were on a single volume. I'm not allowed to use RAID to achieve this. The directory structure needs to be maintained, but there are no duplicate directories across different drives; the complete contents of any given directory are all on the same disk and no other disk contains a directory with the same name. Once created files will not be removed unless the entire dataset is scrapped, but new files may be added. After a quick bit of research my proposed solution is to create a utility that does the following:
  1. Duplicate the complete directory structure from all of the disks on a single volume; this will be done on an additional disk, not one of the ones containing the data.
  2. Populate the newly created directory structure with Symbolic Links to the original files.
  3. Given the above has already been done, the utility will search through the existing set of files and symlinks and create additional links to any newly added files.
As I understand it the symbolic links will act as the original files for purposes of I/O, and as files won't be removed individually (only if the whole set is removed - in which case all the symlinks can go) I shouldn't have to worry about the source files not being there, which seems to be a potential problem with symlinks. I'm confident that the above will work, but does anyone else see any potential issues or perhaps have suggestions (other than the obvious "use RAID you idiot!") for a more elegant solution? Thanks guys. [smile]
- Jason Astle-Adams
cache_hit
cache_hit
Why don't you just use a directory junction to the original directory rather than symlinks to the individual files inside the directory? This makes maintenance a non-issue, any changes that happen on the original disk will automatically be reflected.
jbadams
jbadams
Because I hadn't heard of them, I'll look into it. Thanks for the tip, I suspected there might be some simpler alternatives I wasn't aware of!
- Jason Astle-Adams

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.