Fred Hampton 98143f5dbc Update 'README.md' 3 weeks ago
LICENSE Initial commit 1 month ago
README.md Update 'README.md' 3 weeks ago

README.md

ida:// Decentralized Protocol

The Ida protocol is named in honor of Ida B. Wells an African American teacher, journalist, civil rights pioneer and suffragist about whom it was once said: “She has plenty of nerve; she is as smart as a steel trap, and she has no sympathy with humbug.”

ida:// is the decentralized protocol that powers the Douglass OS and the Douglass Browser

The Ida protocol provides distributed secure semantically enhanced file sharing with historical versioning.

When files are added to the Ida protocol and shared with anyone in the Douglass community all versions of the files are saved and are accessible to any member that has been granted permission to access the files.

All files and date that are added to te Ida protocol are semantically tagged and added to a distributed data graph.

With Ida and the Douglass OS you can Build and host web sites and apps with no ISP.

Key properties of Ida:

  • Content Integrity - The integrity of Ida files and data is verified by the use of signed hashes.

  • Decentralized Redundancy - When users in the Douglass community share the same Ida files they are automatically discovered using the Douglass context vector mapping protocol. This enables the files to exchange semantic data in a swarm.

  • Peer to Peer Network Privacy - Ida guarantees peer to peer end-to-end encryption.

  • Progressive Versioning - Ida enabled files and data can be synced in real time to multiple nodes in the network.

  • Data, Files, and Applications are secured with cryptography.

  • The Ida protocols Decentralized Redundancy gives the ability to transfer large amounts of secure data to scale.

  • With Progressive Versioning, Ida guarantees that every node hosting Ida content has real time dynamically updated versions.

Why Ida?

When sharing files, current tools have tradeoffs: lower costs and ease of use, or security and speed. Cloud services, such as Dropbox or GitHub, force users to store data on places outside of their control. Until now, it has been very difficult to avoid centralized servers without major sacrifices. Ida’s unique distributed network allows users to store data where they want. By decentralizing storage, Ida also increases speeds by downloading from many sources at the same time.

Having a history of how files have changed is essential for effective collaboration and reproducibility. Git has been promoted as a solution for history, but it becomes slow with large files and a high learning curve. Git is designed for editing source code, while ida is designed for sharing files. With a few simple commands, you can version files of any size. People can instantly get the latest files or download previous versions.

Distributed Network

Ida works on a distributed network unlike cloud services, such as Dropbox or Google Drive. This means Ida transfers files peer to peer, skipping centralized servers. Ida’s network makes file transfers faster and more secure. You can even use Ida on local networks for offline file sharing or local backups. Ida reduces bandwidth costs on popular files, as downloads are distributed across all available computers, rather than centralized on a single host.

Data History

Ida makes it easy for you to save old versions of files. With every file update, Ida automatically tracks your changes.

Security

Ida transfers files over an encrypted connection using state-of-the-art cryptography. Only users with your unique link can access your files. Your ida link allows users to download and re-share your files. To write updates to a n ida, users must have the secret key. Ida also verifies the hashes of files on download so no malicious content can be added.

Distributed Network

Ida goes beyond regular archiving through its distributed network. When you share data, Ida sends data to many download locations at once, and they can sync the same data with each other! By connecting users directly Ida transfers files faster, especially sharing on a local network. Distributed syncing allows robust global archiving for public data. To maintain privacy, the ida link controls access to your data. Any data shared in the network is encrypted using your link as the password.

Version History

Ida automatically maintains a built in version history whenever files are added. Ida uses this history to allow partial downloads of files, for example only getting the latest files. There are two types of versioning performed automatically by Ida. Metadata is stored in a folder called .ida in the main folder of a repository, and data is stored as normal files in the main folder.

Ida uses append-only registers to store version history. This means all changes are written to the end of the file, growing over time.

Metadata Versioning

Ida acts as a one-to-one mirror of the state of a folder and all it's contents. When importing files, Ida grabs the filesystem metadata for each file and checks if there is already an entry for this filename. If the file with this metadata matches exactly the newest version of the file metadata stored in Ida, then this file will be skipped (no change).

If the metadata differs or does not exist, then this new metadata entry will be appended as the new 'latest' version for this file in the append-only SLEEP metadata content register.

Content Versioning

The metadata only tells you if or when a file is changed, not how it changed. In addition to the metadata, Ida tracks changes in the content in a similar manner.

The default storage system used in Ida stores the files as files. This has the advantage of being very straightforward for users to understand, but the downside of not storing old versions of content by default.

In contrast to other version control systems, like Git, Ida only stores the current set of files, not older versions. Git, for example, stores all previous content versions and all previous metadata versions in the .git folder. But Ida is designed for larger datasets.

Storing all history on content could easily fill up the users hard drive. Ida has multiple storage modes based on usage.

Ida Privacy

Files shared with Ida are encrypted (using the link) so only users with your unique link can access your files. The link acts as a kind of password meaning, generally, you should assume anyone with the link will have access to your files.

Make sure you are thoughtful about who you share links with and how. Ida ensures links cannot be intercepted through the Ida network. If you share your links over other channels, ensure the privacy & security matches or exceeds your data security needs.

ida:// links

Ida links have some special properties that are helpful to understand.

Traditionally, http links point to a specific server, e.g. http://douglass.io server, and/or a specific resource on that server. Unfortunately, links often break or the content changes without notification (this makes it impossible to cite nytimes.com, for example, because the link is meaningless without a reference to what content was there at citation time). Ida links, on the other hand, never change. You can update data in an ida and use the same link to download the changes.

Here is an example dat link:

ida://aa44725120c5f3c5gs5063e4f68d14a25k21bf48a7b12126d5d588becde88a93

ida: - the protocol

The first part of the link is the link protocol, Ida. The protocol describes what "language" the link is in and what type of applications can open it. You do not always need this part with Ida but it is helpful context.

aa44725120c5f3c5gs5063e4f68d14a25k21bf48a7b12126d5d588becde88a93- the unique identifier

The second part of the link is a 64-character hex strings (ed25519 public-keys to be precise). Each Ida archive gets a public key link to identify it. With the hex string as a link we can assume a few things:

  1. Encrypt the data transfer
  2. Create a persistent identifier, an ID that never changes, even as file are updated (as opposed to a checksum which is based on the file contents).

ida://aa44725120c5f3c5gs5063e4f68d14a25k21bf48a7b12126d5d588becde88a93

All together, the links can be thought of similarly to a web URL, as a place to get content, but with some extra special properties. When you download an Ida link:

  1. You do not have to worry about where the files are stored.
  2. You can always get the latest files available.
  3. You can view the version history or add version numbers to links to get an permanent link to a specific version.