Balanced Adaptive Relay Network (Getting rid of 'dedicated' servers safely)

Balanced Adaptive Relay Network (Getting rid of dedicated servers safely)

Chat systems are in their current implementation mostly implemented with the use of central server systems. In the past the IRC protocol put a real step in the right direction by creating a network of relaying servers. Unfortunately with the large amount of clue-less windows programmers that have come onto the Internet over the span of the past decade, the methods used in chat applications like ICQ etc take us back to the time before IRC.
What this project aims to do is to lay the foundations of a generic layer of relaying p2p nodes that future implementations of chat, game and other now 'central server' based protocols could use to build their services on.

The basic design of this network is based on one important principle: Central servers put a large strain on the resources needed to implement a software system, making the central server principle into a bottleneck for progress.
Peer2Peer software tries to address this problem, but unfortunately not in the best way. Most peer2peer applications are not true p2p, that is they still make use of dedicated servers for coordination and/or name resolution. Next to this Peer2Peer software in its current form, and certainly in a 'pure' form has got two important security issues: 'containment' and 'trust model', these two have great interdependencies, and are hardly addressed in any of the current p2p implementations.
Further having each application create and use its own network is also very inefficient.

This project tries to :

Define a strict p2p security and trust model for use in p2p based on containment and democratic principles.
define protocols for application independent p2p namespace resolution, message based communication and node interconnection.
create an open-source multi platform library that could be used as a global application independent p2p networking layer, and that addresses the main part of the containment issues.
Create a secure global Balanced Adaptive Relay Network network that in its startup phase will make use of the IRC networks, but after reaching sufficient mass will be a true p2p network independent of any dedicated servers

P2P and containment

Where with server security you can say 'a server is not a client', and can based on this contain and detect any intruder or worm (For more information on containment please read this text on Worms,DDOS agents and containment) , on p2p networks you can not say this, in fact every server will be a client, at least at a network (l3+l4) level, and what is even worse, on p2p networks the network itself could facilitate things like worm in spreading.
In order to implement some level of containment we need to start thinking differently of p2p.
At this moment the whole p2p concept is mostly implemented completely in single user-space applications containing both server and client capabilities. If you think of the lower layers of p2p networks as an additional L3+L4 layer facilitating the p2p network, than we see that the complete L3+L4 stack, and both the server and the client are implemented in a single application, and this for every p2p application. As we can see clearly see containment can not be implemented on a network level but only on a host level, with current implementations there can be absolutely no effective containment. By defining a containment model based on separate components that depend on the containment possibilities of the operating system that implements an application independent p2p L3+L4 to applications that are either server or client it would be possible to do at least some host based containment of p2p applications, this will never waterproof, but it will make security much tighter than it is now.

Pure P2P trust-model

Next to the containment problem we need to look at the pure-p2p trust-model. The trust model of true p2p is in its essence very difficult, and in fact I realize that I (or for that matter people 10 times smarter than me) might not be able to create a waterproof trust-model not based on a static hierarchy but on a principle of democratic dynamic hierarchies and mutual distrust. However if this problem/challenge can be solved, than this could possibly take away most of the perceived astronomic security issues surrounding p2p.

Three layers of networks

To create the global true-p2p infrastructure we need to differentiate between the different functions of the network, and realize that extended functionality in one big global network could lead to the use of high amounts of resources on some parts of the balanced network. To avoid this we should define 3 types of networks that have by themselves limited functionality, but that together provide the needed functionality without the dangers of resource bottlenecks. Further these three types and their functionality are made part of the containment model thus reducing the security impact of bugs in the implementation.

Global application independent network: this network basically needs to provide and build the infrastructure and the top level namespace resolution for application specific networks.
Namespace networks: These networks are the application-namespace networks that provide name resolution to the Maze networks.
Maze networks: These networks provide communication channels and distributed random-number-generation within a sub namespace or between individual nodes.

The Global and namespace networks will be partially hierarchical, the maze network will not.

Network design goals

The following design goals apply to all 3 types of networks

The implementation should be done by ways of 'adding' a p2p network layer to the client system , and providing 'global-network' functionality even if there are no p2p applications running.
The clients are all servers, clients that are not also servers are not allowed on the networks. (please note that this only counts for the functionality provided and needed for providing the L3+4 P2P layer services, on application level the opposite should count for containment reasons)
A client is a single-node network
Networks need to take some action in order to locate other networks on the Internet that they could merge with.
Server tasks will be shared amongst the nodes of a network, this includes the scanning for other networks. This will keep down the amount of scanning overhead used by the protocol.
Nodes are assumed to have a low q.o.s, therefore the network will set up highly redundant connections between nodes.
The network will exist of dynamic 'node clusters' that serve as a single hierarchical and logical entity with a decent q.o.s. that approximates to that of a redundant server cluster (at least 99.9%).
The network must take security measures to lower the possibilities of network mis-use, takeover and DoS. (This is a bad-ass specification that would probably take 90% of the time to solve this in a watertight manner.)

Design goals for the hierarchical networks

Nodes in clusters lower in hierarchy will have knowledge of the the node-clusters on higher levels that could be used if a network loses communication with its upstream peers in order to re-merge with the main network.
The network has got fall-back 'flat-remote-links' between clusters on the same level that are in opposite sections of the hierarchy. This is to make the loss of otherwise crucial hierarchical clusters self-repairable.
Nodes will propagate up/down the hierarchy in order to 'fill gaps' and balance the hierarchy.

The global application independent network

The first goal is the creation of a global 'root namespace' network. This network should be without dedicated servers, be redundant, reliable and provide the top namespace that is needed to facilitate the application dependent namespace networks.
For this the following design goals should be reached:

Each node is part of a 'root' network, and may be part of one or more name-space networks, it may also be part of one or more 'maze' network.
The 'root' networks are meant to try to merge into a single Internet-wide relaying network that has as its main server tasks:
- Using simple low-bandwidth (fixed port UDP based) network service scanning to locate peer networks to merge with.
- Merging with other networks using both the UDP based scanning and existing relaying servers networks and protocols (IRC etc) to locate peer networks to merge with. (Ones the number of nodes reaches a reasonable number the use of other networks will no longer be needed, but while in startup this would be crucial)
- Facilitating 'name-space' networks in the process of locating peer name-space networks to merge with.
- Facilitating in name-space-network merging in case of network merging.

Namespace networks

The 'name-space' networks are each meant to merge into a network that includes each node that is within that name-space within the root network. This network itself has the following basic tasks:

Facilitating 'sub-name-space' networks in the process of locating peer name-space networks to merge with (this may be nested).
Node addressing within a (sub) name-space, and the facilitation of the creation of 'maze' networks between nodes.
Handling of name-space collisions (this is a tricky part that needs much work)

Maze networks

The 'maze' networks are fully mazed network between a small group of nodes (n > 1) that all fall inside of the same sub-name-space.

Byte based (talk like) Peer-to-Peer communications.
Message based Multi-cast/Peer-to-Peer communications (IRC like channels, private chat and simple transaction data transfer).
Distributed random number generation. In order for client applications to be able to implement 'no cheating' distributed servers, it is needed that all clients are equal, and are able to check up on the operations of peer nodes. For this reason any randomness action needed by the server should be created and checked by the combination of all nodes. A second protocol should be constructed for the usage of the basic 'distributed random number generation' in distributed server applications, this is however a rather extensive subject, that falls outside of the scope of this project.
Providing node to node addressing.

Layer 4p2p

The 3 network types together can be considered to provide a new layer 3 interface between the different nodes. The networks do not provide sub-node addressing for the inter node communication. For this reason it is essential that this function be implemented somewhere in the design. The exact place for this is still to be determined.

Open protocol

If a open protocol can be constructed that contains all of the concepts and goals as described above, this protocol could facilitate a new p2p layer for use in chat, game and other applications that require the participation of multiple clients, and that normally would require a dedicated server. The use of a single open protocol for this that supports multiple name-spaces could help to create a global application independent p2p network

As for now all I have are these sets of concepts, some unpublished possibly dangerous algorithms and concepts, if you would like to participate in the development of this protocol, the library or more important the creation of the trust model needed for a safe implementation of this protocol, please contact me. I have come to the conclusion that the most important thing we need is a close to watertight true-p2p trust model based on democratic principles and distributed random number generation, before this is realized the publication of draft protocols would be dangerous, and any coordinated work on this for that reason impossible. For this reason I would like to ask anyone who has some knowledge of trust-models to participate in the design of a trust-model that would be suitable for the BARN construct.