Distributed CVS
A few weeks ago I received an email from a person called Giorgio who stumbled across one of the blog I posted about my thesis work and the uses of MantaRay in that kind of environment. We started talking about an idea he had to create a distributed file repository; in other words, a distributed CVS. A repository such as the one he proposed wouldn’t have to relay on a centralized server!
I have spent the last three years designing and developing distributed applications that had a lot of similarities with the one Giorgio proposed, and as a result I was able to point out some of the many potential obstacles and pitfalls one had to overcome before such an idea could become a reality. The first thing that crossed my mind was of course an offline scenario. In a scenario like that, a file is stored in a machine that is currently offline / disconnected from the network. How can someone locate the file? In order to sidestep simple problems like that one, we would need to store multiple backups on separate machines in order to minimize the chance that a file could not be found. Then there's the issue of versioning (different versions on different machines) – which can be solved by timestamps (using simple solutions like NTP), searching and of course the ever popular 'split brain' phenomena which pops up every time one starts dealing with distributed systems.
From Sumit Mital Rice University personal web space you can download a PDF presentation too; here is the link:
Distributed Concurrent Versions System (DCVS).
Into such a document the researcher purpose a couple of still unanswered questions that put the accent on how should act nodes when connectivity might cause indefinite failure or put penalty on the availability of dir and file replicas.
When should node redistribute data among each other on a node failure, immediately, or should the do it lazily, hoping that the failed node will be available soon?
Also, when a new node joins, should the node immediately delete the CVS files for which they are no longer responsible or should they do it lazily, still serving some requests?
A software capable of theses features, would cause a Gestalt in a production environment. Only daily use gives advancement suggesting solutions and optimizations keeping in account that the tipical user is a developer as well.
Formal Claims Citation:
1. DCVS is consistent and trasparent.
2. It is fault tolerant for up to K/2 failures per directory, where K is the replication factor.
I would really appreciate if someone reading the paper confirm my impression of what algorithm govern the Consistency "attitude" of DCVS. I guess it is a Byzantine agreement algorithm described by Castro and Liskov [2000] and since it was my intention to make a post apart treating this and other crucial arguments, endorsements of my intuition is the only reason to anticipate Paxos consensus variations algorithm topic.
The network stub of DCVS is implemented using Pastry (from Rice University) and all the whole code is developed using Java. Pastry architecture is another "key technology", a real substrate for peer-to-peer applications. Expect to find a presentation anyway on this Blog if Yuval allow me to post. :)
Pastry provides the following capabilities. First, each node in the Pastry network has a unique, uniform random identifier (nodeId) in a circular 128-bit identifier space. When presented with a message and a numeric 128-bit key, a Pastry node efficiently routes the message to the node with a nodeId that is numerically closest to the key, among all currently live Pastry nodes. The expected number of forwarding steps in the Pastry overlay network is O(log N), while the size of the routing table maintained in each Pastry node is only O(log N) in size (where N is the number of live Pastry nodes in the overlay network). At each Pastry node along the route that a message takes, the application is notified and may perform application-specific computations related to the message.
Second, each Pastry node keeps track of its L immediate neighbors in the nodeId space (called the leaf set), and notifies applications of new node arrivals, node failures and node recoveries within the leaf set. Third, Pastry takes into account locality (proximity) in the underlying Internet; it seeks to minimize the distance messages travel, according to a scalar proximity metric like the ping delay. Pastry is completely decentralized, scalable, and self-organizing; it automatically adapts to the arrival, departure and failure of nodes.
Scalability/reliability positive issues
Peer neighborhood does the trick replicating directories more smartly than a regular p2p framework can do. Node proximity is a key factor realizing a fault tolerance CVS; Sumit Mital PDF argue DCVS gives better result compared to regular CVS systems.
I stop here anticipating, follow the link now if these hints resulted of any interest.
Best regards, Giorgio.
Well, I found this nice PPT which talks about distributed CVS
A Manifesto for Collaborative Tools
Eugene identified four necessary steps for improving collaborative tools:
Even though proclaimed points may appear more language oriented, there is a redundant key word in the text; it rises interesting points on the reasons behind DCVS tools. Statements proclaim that we, as developer, need to develop tools collaboratively minded. Doing this fashion strength the comunity, improve our tools and sometime improve ourself too.
The Eugene article has been written 2+ years ago, but it is going to be more actual than before.
Best regards, Giorgio.