Multiple servers, multiple clients, one file inventory

[merendo]

I am working on a software, which will, once done, do the following:

Replicate and maintain sync of one single file inventory (a filesystem with directories, subdirectories and large contents) across multiple servers. Furthermore, it will allow access to these files to clients, and whenever one client accesses a file from any one server, this file will be locked for further access across all servers, so as to prevent several users accessing the same file at the same time. The clients can also create new files and store them on one server, it will then replicate across all servers. The whole point is to allow users in different geographical locations access to the same inventory of files, while not having to wait for a slow VPN link but instead experiencing full performance of a server on the same LAN.

The files are kept in sync across all servers using Robocopy - each server will create an instance of Robocopy for every other server, in order to create a bidirectional sync system. This will mean that, for two servers, there will be two Robocopy channels running, for three servers six channels, for four servers twelve channels and so forth... (k=n!/(n-2)!)

The clients themselves will never have direct access to the files on the server. Instead, they will see some sort of links, replicated on their local machine, through which they can request to access a file, and then be either told that the file is already being accessed by someone else, or have that file downloaded on their system and do what they want with it. When done, they can upload the file back onto the server and from there, the updated version will be synced onto all other servers, and once that has happened, the file will be released for further access by anyone else.

The software is nowhere near ready yet. I am doing extensive testing and planning right now. Perhaps someone else has done something similar before and can give me a few hints, or maybe think of technical issues I might encounter I haven't thought of yet. Please feel free to ask or make comments, everything (constructive) is appreciated!

[kinglestat]

Not sure why you need to use a program, windows since NT 4.0 has a very good replication service which with 2008 has become very easy to use

You can use DFS + replication and let windows do the rest, and windows will actually choose the "closest" server to the user. An added advantage is that replication will only sync the changed parts of the files whereas robocopy will copy the whole file which is not practical for large files.

[merendo]

Well, these features apparently only come with the larger business versions of Windows servers and therefore cost too much. The solution we're working on will be more competitively priced. So, if you will, it'll be a poor mans version of a syncing server as the software will run on any computer running at least WinXP.

As for delta encoding: I simply don't know how to do that. I did once try to write my own routine for file syncing which would give me greater control but this failed dismally once the amount of data grew beyond a few GBytes. So I am now handing this task over to a tool proven for nearly ten years now.

[blueznl]

Should I think of a localized version of Dropbox?

For personal / commercial use, open source, or even to make a little cash?

[merendo]

It's what I do at work. It's for a customer, but to be sold later on. The thing is, I will be leaving my current company by the end of the month so I'm somewhat pressed for time.

A localised version of dropbox with your own servers, providing full network performance and storing large amounts of data. Also shows when a file is already in use by someone else...

[Rings]

sorry , but not a real anouncement(except the idea). moved to offtopic for now.