🔗Yggdrasil’s Limitations

To start, it should be noted that Yggdrasil is an experimental routing protocol (based on DHT’s), which is currently still in an alpha stage. We very much admire the fact that they have been able to create a standalone network with worldwide participants from scratch. Over time, the network has grown, in part due to the grid, but also because a lot of individual participants joined and started running a node.

After investigating, we reached the conclusion that the current architecture has some scalability issues. While we have no doubt that these will eventually be solved, the importance of the planetary network in the grid is unfortunately too big to be able to wait for this. Additionally, Yggdrasil uses a technique called crypto key routing, and uses a tree structure for its routing info. This means data follows the “shortest path” in the tree, rather than the physical shortest path, which might deviate from this. As such, we have decided to implement our own Ipv6 overlay network, Mycelium.

🔗Implementing Mycelium

As previously pointed out, the main goal of our network is to be efficient, and end-to-end encrypted. Every node in the network is identified by a key pair. The address of a node is derived from this key pair, and data traffic between 2 nodes is encrypted using a secret derived from the key pair of both these nodes. As a result, every 2 nodes in the system use a different secret for encryption, and we can verify that the address used by a node is indeed owned by it (i.e. it has the corresponding private key). We have based our initial implementation (loosely) on the babel routing protocol.

Therefore, sending data to a remote node will follow the shortest path in terms of latency (i.e. the fastest route). The network adjusts itself in case nodes disappear or join, and multiple paths to a given destination are possible.

We are also considering implementing more advanced use cases in the future, though this remains to be seen at this stage. We are gearing up for some larger scale testing on the grid in the near future (after all, we do have multiple thousand nodes at our disposal to verify scalability), and will adjust as needed once these are done.

🔗The Rise & Fall of Platforms

The world of digital platforms can leave us feeling a bit deflated. We invest time and effort in creating content and building our online presence across platforms, only to see them come and go – either in terms of popularity or existance. Years of efforts can go down the drain in an instant. And the introduction of new platforms means we feel almost pressured to create new profiles yet again. Take Threads, for example, recently released by Meta. I, personally, have not downloaded the app or created a profile myself. Nor do I plan to.

🔗Algorithmic Shifts & Manipulation

Algorithms, like shifting weather, change direction without warning. What used to be familiar suddenly feels like a puzzle we can’t solve. Further, algorithms on social media and other platforms are designed to keep us engaged. They show us content that’s likely to keep us scrolling and clicking. This can lead to addictive behaviors, with us spending more time on platforms than we initially intended.

🔗Platform Redundancy

Why, in this digital age, do we have countless platforms doing essentially the same thing? It’s as if we’re at a party with multiple dance floors, all playing the same music. This fragmented landscape demands our memory’s acrobatics as we juggle passwords, navigate different interfaces, and figure out where our friends are hanging out. A simpler, unified experience seems a distant dream.

🔗Products in a Machine

We pour our creativity and stories into platforms, only to realize we don’t truly own them. Our content, our data, our followers – they belong to the platform. It’s like building a sandcastle at the beach, but it washes away with the tide. The digital space, which once held the promise of empowerment, has subtly transformed us into products within a larger system.

🔗Decentralization: Reclaiming Digital Sovereignty

Imagine a digital world where you’re the true owner of your data and content, where your digital footprint isn’t erased by platform shifts. A space where the power to shape the experience is in your hands, where you’re not a product but a participant in a collaborative journey.

This reimagined Internet isn’t a dream; it’s an evolving reality. Decentralization represents a paradigm shift – a chance to reclaim our digital autonomy. We need to foster an environment where platforms don’t dictate, but rather, serve as tools shaped by the collective. This is a call to action, a reminder that a more inclusive, user-centric Internet is within reach.

The Internet has always been a canvas for human connection and creativity. Let’s paint a picture where we’re not just contributors but true owners, where we’re not lost in a labyrinth but are the architects of our digital landscape. Let’s embrace a decentralized Internet, because we deserve an Internet that truly serves us – the people.

🔗Technology as a tool for humanity

Technology… In today’s world, technology holds a central role. Younger generations in developed countries grow up amidst technological wonders of all sorts, truly immersed in an evolving digital landscape. Any information is a click away, offering immense intellectual capital unlike ever before – and on the other hand, great distraction, misinformation, and potential for division.

Technology is a word dating back to the early 17th century, formed from the Greek tekhnologia meaning ‘systematic treatment,’ itself formed from tekhnē meaning ‘art, craft.’ Technology is the application of knowledge to reach practical goals in a specifiable and reproducible way. The word technology may also mean the product of such an endeavor, as it is mostly used today.

As technologies enter our world they either add on incrementally to already existing technologies (i.e., tablets) or shake everything up (i.e., television). Technology can be found in most industries and at different levels, ranging from vehicles to home appliances, passing through healthcare, education, and even entertainment. Sometimes technology replaces human labor by replicating it, other times it facilitates processes and makes labor less intensive. Some technology even surpasses human precision.

While technology has had an important role in the creation of today’s societies, it might have an even greater role for our future. Two opposite pictures could be painted: a utopia or a dystopia. The utopia of a world full of values-driven, well-constructed technologies promises an increased quality of life for all, with digital tools which, for instance, could provide or enable equal educational opportunities for all, mobility everywhere, a universal basic income and the promise of never having to work again (e.g. in the sense of wage labor).

On the other hand, a dystopia could be the result of reckless technological advancement. Questions are raised, like, will most jobs be dominated by artificial intelligence (AI) in 20 years? Will privacy become a myth, with personal data being controlled by a handful? Will consumerism rule societies and reinforce social struggles?

One technology that has been slowly taking the dystopian route is the Internet. While major Internet and Cloud infrastructure advancements were made since the beginning of the millennium through hyper-scale data centers, the current Internet infrastructure model has many flaws:

• It is centralized: The GAFAM (Google, Amazon, Facebook, Apple and Microsoft) own 80% of the global compute, storage and network infrastructures. Also the global infrastructure is concentrated in the U.S in large majority, as well as Europe and China.

• It is inefficient: Most of the data centers are located in the U.S, forcing the rest of the world to connect via Internet sea cables and having their data flowing outside of their borders and back.

• It is not scalable: Emerging technologies require the global Internet infrastructure to triple in size in the next 4 years. It took 20+ years to build the current Internet infrastructure. So, we need to build 3 times more in 5 times less time.

• It is not affordable: It costs billions of dollars and enormous amounts of energy and time to build, manage and productize hyper-scale data centers. Only the richest countries in the world build data centers and the rest of the world become their clients.

• It is not private: Anyone can access Google Chrome or Facebook for free. But as outlined by movies such as the Social Dilemma or The Great Hack, if it is free, you are the product. The web’s Ad model is threatening our privacy and the net neutrality of the global internet.

• It is not secure: The current IT stack depends on human error and it has been considered as the major cause of cybersecurity breaches with more than 95% of the cases. Human errors allow attackers to access encrypted channels and sensitive information

• It is not sustainable: Data centers are extremely power hungry, consuming over 2 trillion gigabytes of storage and consuming around 205 TWh - More than the State of New York. According to statistics, the Internet consumes between 10-20% of global energy resources.

• It cannot support future demand: To meet projected demand for 2022, more than 4,000 new data centers, costing an estimated $5 trillion, will be needed to build and consume an additional 400 GW/h. This is roughly equivalent to Japan’s energy consumption and would place an impossible strain on the global electricity grid. The rising demand for Internet storage is already causing hardware shortages and price hikes.

At ThreeFold, we want to encourage the creation of a utopian future driven by thoughtful technology. Our founders took part in the way the Internet was created, full of beautiful promises, and have seen first-hand how those promises have been corrupted throughout its evolution. By building a new Internet infrastructure from scratch, ThreeFold goes back to those original promises while having learnt from the development mistakes.

Kristof, our CEO, wrote in a blog:

“The Internet allows the expansion of our village life to become a global community. It allows us to act as a global tribe with all information at our fingertips, it offers the possibility to get equal access to all planet resources. It has the promise for global free communication, instant availability of information, unlimited opportunities for growth and education. But what happened to our Beautiful Internet?”

“The Internet allowed us to become more efficient in depriving mother earth from her resources. It allowed us to become more addicted to immediate gratification, examples are social media networks and the adult entertainment industry. Beautiful new possibilities like digital currencies became a gambling game and a source of wealth for the smartest hackers. The communication capabilities of the Internet allowed the few to manipulate the many.”

By putting people, planet and authenticity at the core of the development of our technology, we want to gift societes a remade internet. Features of this technology include sovereignty & decentralization, security & privacy, unlimited scalability, green & self healing properties, cost effectiveness, and reliability. The human is put at the center of our Internet model.

Technology should not be a tool to extort humanity, but one at the service of humanity.

The Internet is one of humanity’s greatest inventions. In its early days, it used to be free and decentralized by nature. No single entity was able to control access to the web as it was built on the foundation of a peer-to-peer network of nodes owned by developers and researchers. Until the dot-com era. Throughout the early 2000s, large corporations took over the web. New platforms like YouTube and MySpace were launched and new infrastructure needed to be built at an extreme pace, starting the data revolution and changing the digital landscape forever. 

Twenty years down the line, the Internet’s architecture has become highly centralized – owned by a handful of large corporations that act as the gatekeepers of our digital world. What was once an open and free platform accessible to anyone has long since built up barriers and lost its neutrality. Users are trapped in a cyberspace that exploits their personal data and allows censorship, hate speech, disinformation and political targeting to thrive. 

🔗Going down the Rabbit Hole

The Wall Street Journal’s investigation “The Facebook Files” and The New York Times’ audio series “Rabbit Hole” are only some of the latest examples that depict the shocking impacts of the manipulation, mental health problems, and extremism that social media algorithms are fuelling. Nevertheless, these incidents are only a glimpse into all the ways we’re being affected and influenced about the way we see the world – by the tech giants and their algorithms. Tech giants like Google and Facebook have been surrounded by a constant stream of data privacy scandals as more and more whistleblowers expose their misconduct. However, there are plenty of other, hidden players using the tools and algorithms of social media platforms for their own agendas.

The Great Hack and the Social Dilemma are great documentaries that shed light on how the Internet as we know it became compromised. They depict how algorithms are used to mine user data, control information feeds and manipulate our digital experiences. The Trump Election and the Brexit vote were heavily influenced by private companies like Cambridge Analytica that manipulated mass audiences through social media.

By purchasing thousands of personal data points collected by social media platforms and analyzing them, companies like Cambridge Analytica are able to not only predict but influence the behaviour of each and everyone of us. Data collection and analysis to such an extent allow these companies to craft and push perfectly targeted messages to users in a way that’s optimized to sway their opinions in favour of whatever agenda they have.  

🔗Can a neutral and free Internet still exist?

On 24th of March 2022, the European Parliament and Council negotiators agreed upon the Digital Markets Act (DMA). It seeks to give users more choice by demanding the most popular apps to open up and interoperate with smaller apps. Moreover, the act aims to ensure a fairer market competition by prohibiting certain gatekeeper practices like self-preferencing and targeted advertising. This set of new EU regulations is likely to impact the daily business of tech giants significantly once it becomes law. While the DMA promises to mitigate some of the issues discussed, it is doubtful that the act will be enough to fundamentally change the current system. This is a step in the right direction, but much more needs to be done.

The Internet has become our digital home. It’s impacting our lives in many ways by creating new opportunities, and changing how we communicate and connect. Therefore, it is crucial to restore the Internet’s neutrality to ensure a fair and equal environment. We need an Internet where we can rely on data security and privacy. An Internet where we’re empowered to truly and completely have ownership of and control of our own data. An Internet that empowers us with digital sovereignty.

The Internet used to be a free movement of culture and the Web. It started with an open-source and peer-to-peer vision to connect people. And while Web 3.0 projects are looking to liberalize the Internet as we know it, most of them still rely on centralized Web 2.0 infrastructure to some degree. The world needs better, holistic solutions. ThreeFold is realizing the initial vision and promise of the Internet as an open-source, peer-to-peer planetary mesh network of nodes owned by humanity. It’s a 100% open-source and community-driven project that builds the critical, neutral infrastructure for a decentralized world – one that is built and owned by the people. With ThreeFold, individuals and organizations are empowered to store their data and run their applications on a peer-to-peer network of nodes in a way that ensures their data will not be manipulated, hacked or lost. 

Powered by a global community, ThreeFold is the world’s largest peer-to-peer Internet infrastructure with a few thousand 3Nodes connected globally and the neutral foundation for any current and future IT workload. ThreeFold is now on the verge of opening up a new Internet era free from censorship, user tracking and privacy breaches. And anyone can participate in ThreeFold’s model by building or buying a 3Node, and connecting Internet capacity from their home or office – creating a fully decentralized Internet. 

Similar to the enterprise blockchain hype that happened a few years ago, Web3 seems to be repeating the same story, with lots of chatter but lagging adoption. Millions have rushed to trade crypto and NFTs attached to Web3 – with a lot of this activity based on speculation and hype – but that doesn’t mean they will replace legacy platforms anytime soon. New approaches such as DeFi (decentralized finance) don’t have the same problem of the legacy firms, but they face the challenge of scaling and mass adoption. However, many so-called ‘Web3’ solutions are not as decentralized as they seem, and still have to show they are scalable, secure and accessible enough for the mass market. This all may change, but there are clear limitations that will need to be overcome.

Whether Web3 is a marketing slogan or a true technological trend, there is a lot of money and technology behind it. Financial institutions are embracing blockchain technology and we are seeing a rising adoption of digital assets across industries. While the idea of utilizing digital currencies to exchange different forms of value in an increasingly digital world certainly makes sense, it’s a bigger leap to say that blockchain alone can support a broad set of use cases, including replacing the current Internet infrastructure.

The Internet’s client-server infrastructure replicates data and workloads across many computers, and uses tons of computing resources and energy while limiting the scalability of our applications. However, the clear inefficiencies and gaps in security in today’s centralized infrastructures are being challenged by simpler means of building, storing and exchanging data.

Blockchain is a clear evolution to the centralized client-server model, but there are severe challenges sourced from the blockchain trilemma between decentralization, scalability and security. Peer-to-peer systems transcend those challenges as they don’t require any valid2ation or verification from a third party, and enable end-to-end connections that can scale limitlessly without the potential of being altered or corrupted by intermediaries. In peer-to-peer systems, all participants are equally privileged, presenting a fairer model. Peer-to-peer represents an incredibly simple and efficient paradigm for organizing our digital world, and has already proven to be the most efficient alternative to centralized information systems.

Simply put, there is no reason to replicate full sets of data and workloads across many computers, and use tons of computing resources and energy while limiting the scalability of our applications. While there is an increasing reliance on blockchain principles to organize our Internet systems, blockchain also makes things redundant, slow, complex and energy intensive. To be clear, blockchain isn’t the problem, as it is a great technology to store and exchange value in a decentralized way, but it is only one piece of the puzzle.

By inter-connecting the computing processing power and storage capacity of billions of computers worldwide in a peer-to-peer way, data transfers could happen end-to-end between people and machines without the need to be verified. Peer-to-peer is also the only way to achieve a truly self-sovereign Internet – creating a ‘trustful’ environment that allows users and developers to have real data ownership.

Peer-to-peer systems aren’t meant to replace blockchains – they represent a way to decentralize models that blockchain cannot, such as Internet infrastructures. Peer-to-peer systems and blockchains, side by side. The best of both worlds. Peer-to-peer as the most efficient and decentralized alternative for us to exchange data and Internet resources. And blockchain for establishing trust through an irrefutable record of transactions.

With the rise of emerging technologies, a new technological foundation is needed to protect our sovereignty and freedom. The rise of blockchains, while well intended, ultimately has made the technological space more complex than ever before, and still benefits a plutocratic few (relative to its promise). More than just technology, peer-to-peer opens a world of new possibilities for humanity to co-initiate new paradigms and experiences.

By applying blockchain and peer-to-peer technology to the cloud, ThreeFold solves the security and autonomy issues of the Internet and realizes the initial promise of the Internet as an open-source, peer-to-peer, planetary mesh network of nodes owned by the people.

After introducing the ThreeFold Grid in part one and diving deep into storage and compute, we’re back with part 4 to look at the networking aspects of the ThreeFold Grid.

Computer networking is a rather complex phenomenon with a thankfully intuitive outcome: you send messages from one place to another. We all understand that our digital devices become a lot less interesting when they are not connected to the Internet. However, networking is not just Internet, and when it comes to building a new Internet, it’s important to understand what that means.

The existing Internet can be defined as a global set of networks and protocols that enable us to do things like load web pages and use apps that rely on external data. It depends on lots of underlying infrastructure, including fiber optic cables running along the ocean floor, our home routers and their connection to an Internet service provider (ISP), lots of cell service towers, and an increasing number of satellites. All of that makes the Internet possible, and it’s still required for a new Internet to function.

ThreeFold uses and builds new protocols to create a network that is interoperable with the existing Internet. These include private overlay networks, the Planetary Network based on Yggdrasil, and various interfaces to the public Internet. Let’s see how they work.

🔗Private overlay networks

Every deployment on the ThreeFold Grid belongs to a network, which is one of the Grid primitives. These networks allow data to be transmitted between workloads in a private and secure manner. They are called overlay networks because they pass traffic over some underlying network. That could be within a single node or between nodes on infrastructure provided by a farmer. The Grid uses software called WireGuard for this purpose, a simple, fast, and secure option among its alternatives.

By means of encryption, the traffic on a private overlay network is unreadable to any other device on that network. Likewise, the workloads within the private network have no visibility of the underlying network. Individuals who deploy workloads can access their networks over a secure tunnel, assuming that one of the nodes in the network is reachable from the public Internet. This is suitable for workload administration and some private applications. For public services and peer-to-peer communication, the ThreeFold Grid provides other solutions.

🔗Planetary Network

The Planetary Network is an implementation of the Yggdrasil Network, which is a next generation approach to network routing. It is a peer-to-peer solution where all traffic is encrypted and takes the shortest path to its destination. These features make it highly compatible with the ethos and goals of the ThreeFold Grid: privacy, security, and efficiency. The design also allows nodes, workloads, and users to communicate freely without relying on centralized intermediaries.

Many ThreeFold Farmers connect their nodes to a home router which blocks all inbound traffic by default. This is a sensible feature that doesn’t interfere with most Internet usage (enthusiasts who want to host their own website or a gaming server might bypass this, for example). However, nodes on the Grid need to be reachable, especially in order to receive workload definitions from deployers.

The Planetary Network solves this by accepting inbound traffic through an outbound connection to a publicly accessible peer. You can think of this like making a call to an operator and then waiting for another call to be routed to you on that line. Even if your phone is set to block all incoming calls, someone can still reach you if they can reach the operator.

This also provides a way for workloads to communicate with one another and be reachable by users. Anyone can enable the Planetary Network on their own device to access sites and services in a very secure and private way. While the ThreeFold Grid is designed to be fully interoperable with the existing Internet, the Planetary Network provides an alternative which can sit side by side in a seamless way.

🔗Reliable Message Bus

Using the Planetary Network, nodes and Grid users can send messages over the Reliable Message Bus. Notably, it is used for the delivery of workload definitions as mentioned earlier. Zero-OS supports some other RMB messages as well, for doing things like checking available capacity on a node in real time. RMB is a general purpose protocol that can be used by developers in their applications. It queries the ThreeFold Blockchain as a kind of address book for the Planetary Network, enabling nodes and users to find each other in a decentralized way.

🔗Public IPs and Web Gateways

Finally, let’s see how the Grid can offer services over the public Internet using public IP addresses and web gateways. Everytime we type a domain name into our browser, like threefold.io, it is resolved to an IP address. The browser then forms a connection with that address to retrieve the site we’re looking for. Reviving a theme from earlier, that IP must be publicly reachable.

🔗Public IPs

We’re in the midst of a shift from the older IPv4 standard to the newer IPv6 standard. While IPv6 has plenty of addresses for every device on the planet and many more, it hasn’t been universally adopted yet, and IPv4 addresses are in short supply. You likely have one assigned to your router at home, and these days, even getting a second one from your ISP can be a difficult and expensive proposition.

The ThreeFold Grid allows farmers who own blocks of IPv4 addresses to rent them out to deployers. These addresses get attached to individual workloads which are then reachable over the public Internet. A deployer could then assign their own domain name to that IP and host a website there. This is a very flexible way to make Grid deployments publicly available, but it comes at a relatively high cost.

🔗Web Gateways

Web gateways are another way to make workloads on the ThreeFold Grid available over the public Internet. In this case, the farmer may provide a domain name that points to one of their nodes which is publicly reachable. Deployers are then able to reserve a subdomain, like explorer.threefold.io is a subdomain of threefold.io, that routes traffic to their workload. It is also possible for a deployer to point their own domain to the gateway node which then routes traffic to the appropriate workloads.

We glossed over a bit in the earlier description of domain name to IP address resolution. In fact, multiple domains can point to a single IP address and this is common practice among existing web hosts. Adding similar functionality to the Grid helps to ease the demand for IPv4 addresses and is a lower cost solution for deployers than renting a dedicated IP. The web gateway also serves to enhance security and reliability of Grid-hosted services.

The gateway provides a connection from the public Internet to the secure private overlay networks that all workloads use. However, this is not a standard network connection but instead a network socket, which provides security through separation. This is something like passing along the contents of mail pieces without their envelopes. Web gateways also enable redundancy, as multiple gateways can point to the same workload, and multiple workloads can serve the same set of gateways.

When we say the ThreeFold Grid scales limitlessly, this is one of the ways that’s true. Many nodes, many gateways, many applications, and many users can all fit into this model. Deployers can start with a single virtual machine and a single public IP like in the example I’ll use to wrap up this piece below, then ramp up to a globally distributed and highly redundant system as their requirements grow.

🔗Send, receive, complete

Now we’ve seen how the ThreeFold Grid handles networking. From private and secure communication between workloads, to next generation peer-to-peer communication and full compatibility with the existing Internet, these offerings are comprehensive. Not only that, but they can be pretty darn fun to play with too.

In my own recent experience, I deployed a virtual machine on the Grid with a public IP and hosted a simple website with a map of nodes on the Grid that I’d been working on. I was able to quickly share it with some colleagues and members of the community. It went from a project I was tinkering with on my own computer to a live site on the Internet in a matter of minutes, thanks especially to the networking features of the ThreeFold Grid.

For the final piece of this series, we’ll see how everything comes together to turn computer code into a meaningful digital experience. Stay tuned!