Getting Started

Source repos

cweb-schemas - Thrift schema definitions
cweb-core - Core library implementing Cweb protocols
cweb-cli - Command Line Interface
cweb-conversations - StoneAge Conversations Android messenger

Building the core library. Requires Java 8 or above, and Apache Thrift 12.

$ git clone https://gitlab.com/cweb-repos/cweb-schemas.git
$ git clone https://gitlab.com/cweb-repos/cweb-core.git
$ cd cweb-core
$ mvn clean install

Building the CLI. Requires pre-built core library.

$ git clone https://gitlab.com/cweb-repos/cweb-cli.git
$ cd cweb-cli
$ mvn clean install

Command Line Interface (CLI)

Cweb CLI allows creating Cweb identities and invoking (some) protocol operations from the command line. CLI can be built from sources (see above) or downloaded as a Linux/MacOS runnable cweb-cli. CLI stores local data under ~/.cweb/ by default, the location can be overridden via the -localPath option.

Usage: cweb-cli [options] [command] [command options]

CLI Playground mode allows trying out protocols while simulating remote storage in the local file system. It does not require an actual object storage and network access. Playground mode is enabled via the -playground option.

Although the examples use the playground mode they would also work with real online storage.

Creating New Identities

In the playground mode, any properly-formatted private storage profile is accepted (and ignored). In the non-playground mode, you will need a valid private storage profile (See Storage Setup), otherwise the command will error out.

# Create identity A
$ cweb-cli -playground identityCreateNew -privateStorageProfile 's3!https://s3-us-west-2.amazonaws.com!us-west-2!aws-cli-test1!test1!temp-test1!!!xxx!yyy'
{
  "idStr" : "8XiKJdThhgwgoBrBVAEgo-Il5kQqU_4EW-jAx0WiQ8E="
}

# Create identity B
$ cweb-cli -playground identityCreateNew -privateStorageProfile 's3!https://s3-us-west-2.amazonaws.com!us-west-2!aws-cli-test1!test1!temp-test1!!!xxx!yyy'
{
  "idStr" : "QC3QvAlAF0TZQOjs75JUXv7wdKXjYGEV38vHlgkd_AQ="
}

Every invocation of identityCreateNew generates a random and unique Cweb Id. Make sure to use the actual Cweb Ids you generated (the idStr response) and replace the ones in the examples below.

Multiple identities can reside on the same local machine and share the same local and remote storage. Every identity is fully independent and knows nothing about other identities sharing the the machine or storage. CLI commands are executed on behalf of the specific identity selected via the -ownId option, which accepts any non-ambiguous prefix of Cweb Id.

Adding Known Peers

Since there is no central discovery service, we have to explicitly cache Cweb Ids and public storage profiles in peers’ local storage. The next example retrieves identity descriptor of peer A using the identityPrintOwn command, and “passes” values of idStr and publicStorageProfileDecoded to B. B invokes storageProfileSet command that associates A’s Cweb Id with its public storage profile, and caches this information in its local storage. Then we do it the other way around - A caches B’s storage profile. Finally, we verify that A and B can read each other’s identity descriptors via the identityRead command.

# Print A's identity descriptor
$ cweb-cli -playground -ownId 8Xi identityPrintOwn
{
  "identityDescriptorDecoded" : {
    "idStr" : "8XiKJdThhgwgoBrBVAEgo-Il5kQqU_4EW-jAx0WiQ8E=",
    "publicStorageProfileDecoded" : "s3!https://s3-us-west-2.amazonaws.com!us-west-2!aws-cli-test1!test1!temp-test1!!",
    "ownProperties" : [ ],
    "endorsements" : [ ]
  },
  "idUrlSafe" : "6f4iujou4gdayifadlaviajauprclzsefjj74bc35damorncipaq9999",
  "publicStorageProfileUrlSafe" : "af4jzy3bmdrgmydembqfbsrifeusrnws24xtnvrnfxlc2tznf3itluslzrg2zswpjmwc7vslz3h6kztaaivojbgloiztaa7zxraftxpetrgn2evauaqtoaylkcgblstgawzlssssomfsbeqmbqaoika2u4999999"
}

# Add A to B's known peers
$ cweb-cli -playground -ownId QC3 storageProfileSet -id 8XiKJdThhgwgoBrBVAEgo-Il5kQqU_4EW-jAx0WiQ8E= -publicStorageProfile 's3!https://s3-us-west-2.amazonaws.com!us-west-2!aws-cli-test1!test1!temp-test1!!'

# Print B's identity descriptor
$ cweb-cli -playground -ownId QC3 identityPrintOwn
{
  "identityDescriptorDecoded" : {
    "idStr" : "QC3QvAlAF0TZQOjs75JUXv7wdKXjYGEV38vHlgkd_AQ=",
    "publicStorageProfileDecoded" : "s3!https://s3-us-west-2.amazonaws.com!us-west-2!aws-cli-test1!test1!temp-test1!!",
    "ownProperties" : [ ],
    "endorsements" : [ ]
  },
  "idUrlSafe" : "iaw5bpajialujwka5dwo7esul37pa5ff4nqgcfo7zpdzmci57qca9999",
  "publicStorageProfileUrlSafe" : "af4jzy3bmdrgmydembqfbsrifeusrnws24xtnvrnfxlc2tznf3itluslzrg2zswpjmwc7vslz3h6kztaaivojbgloiztaa7zxraftxpetrgn2evauaqtoaylkcgblstgawzlssssomfsbeqmbqaoika2u4999999"
}

# Add B to A's known peers
$ cweb-cli -playground -ownId 8Xi storageProfileSet -id QC3QvAlAF0TZQOjs75JUXv7wdKXjYGEV38vHlgkd_AQ= -publicStorageProfile 's3!https://s3-us-west-2.amazonaws.com!us-west-2!aws-cli-test1!test1!temp-test1!!'

# Verify that A knows B
$ cweb-cli -playground -ownId 8Xi identityRead -id QC3QvAlAF0TZQOjs75JUXv7wdKXjYGEV38vHlgkd_AQ=
{
  "identityDescriptorDecoded" : {
    "idStr" : "QC3QvAlAF0TZQOjs75JUXv7wdKXjYGEV38vHlgkd_AQ=",
    "publicStorageProfileDecoded" : "s3!https://s3-us-west-2.amazonaws.com!us-west-2!aws-cli-test1!test1!temp-test1!!",
    "ownProperties" : [ ],
    "endorsements" : [ ]
  }
}

# Verify that B knows A
$ cweb-cli -playground -ownId QC3 identityRead -id 8XiKJdThhgwgoBrBVAEgo-Il5kQqU_4EW-jAx0WiQ8E=
{
  "identityDescriptorDecoded" : {
    "idStr" : "8XiKJdThhgwgoBrBVAEgo-Il5kQqU_4EW-jAx0WiQ8E=",
    "publicStorageProfileDecoded" : "s3!https://s3-us-west-2.amazonaws.com!us-west-2!aws-cli-test1!test1!temp-test1!!",
    "ownProperties" : [ ],
    "endorsements" : [ ]
  }
}

Exchanging Direct Messages

Once A and B know each other, they establish a communication session and exchange messages.

# A establishes a session and sends a message to B
$ cweb-cli -playground -ownId 8Xi commSessionEstablish -id QC3QvAlAF0TZQOjs75JUXv7wdKXjYGEV38vHlgkd_AQ=
{
  "success" : true
}

$ cweb-cli -playground -ownId 8Xi commSessionMessageSend -id QC3QvAlAF0TZQOjs75JUXv7wdKXjYGEV38vHlgkd_AQ= -message "Hello World"
{
  "message" : {
    "type" : "CUSTOM",
    "service" : "CLI",
    "customType" : "Message",
    "data" : {
      "dataSize" : 11,
      "dataHash" : -733462557,
      "dataHead" : "Hello World"
    }
  },
  "success" : true
}

# B receives the message
$ cweb-cli -playground -ownId QC3 commSessionMessageRead -id 8XiKJdThhgwgoBrBVAEgo-Il5kQqU_4EW-jAx0WiQ8E=
{
  "messages" : [ {
    "type" : "CUSTOM",
    "service" : "CLI",
    "customType" : "Message",
    "data" : {
      "dataSize" : 11,
      "dataHash" : -733462557,
      "dataHead" : "Hello World"
    }
  } ]
}

# Another message read gets nothing since no more messages were send
$ cweb-cli -playground -ownId QC3 commSessionMessageRead -id 8XiKJdThhgwgoBrBVAEgo-Il5kQqU_4EW-jAx0WiQ8E=
{
  "messages" : [ ]
}

Note the extra metadata that allows messages from other protocols and applications to be multiplexed in the session. One example is the file sharing message demonstrated next.

The fileUpload command encrypts and uploads a file. The decryption key is stored locally. The list of uploaded files can be inspected using fileUploadsList command, and files can be deleted from the online storage using fileUploadsDelete command.

# A uploads a file
$ cweb-cli -playground -ownId 8Xi fileUpload -filePath document.pdf
{
  "fileId" : "HS8JndD9Stjcv04RqA2QayfW2FdN7ZvYNOqyzJZTQmo=",
  "key" : "fE-6y1s3tVn7Hhki-Q9D66D_wR-0UOxdyH68MUdEqmd8dFia64zkDR4w0PTc9NEchEylz4w-Nh90UImL5QqTBg==",
  "fileMetadata" : {
    "name" : "document.pdf",
    "createdAt" : "2019-06-06 08:33:21.020",
    "size" : 1784683
  },
  "uploadState" : {
    "uploadedFraction" : 1.0,
    "corrupted" : false
  },
  "tookTime" : 160
}

# A lists uploaded files
$ cweb-cli -playground -ownId 8Xi fileUploadsList
{
  "fileUploadStates" : [ {
    "fileId" : "HS8JndD9Stjcv04RqA2QayfW2FdN7ZvYNOqyzJZTQmo=",
    "fileMetadata" : {
      "name" : "document.pdf",
      "createdAt" : "2019-06-06 08:33:21.020",
      "size" : 1784683
    },
    "uploadState" : {
      "uploadedFraction" : 1.0,
      "corrupted" : false
    },
    "fileReference" : {
      "fromId" : "8XiKJdThhgwgoBrBVAEgo-Il5kQqU_4EW-jAx0WiQ8E=",
      "fileId" : "HS8JndD9Stjcv04RqA2QayfW2FdN7ZvYNOqyzJZTQmo=",
      "key" : "fE-6y1s3tVn7Hhki-Q9D66D_wR-0UOxdyH68MUdEqmd8dFia64zkDR4w0PTc9NEchEylz4w-Nh90UImL5QqTBg==",
      "metadata" : {
        "name" : "document.pdf",
        "createdAt" : "2019-06-06 08:33:21.020",
        "size" : 1784683
      }
    }
  } ]
}

The fileShare command sends the file reference that includes decryption key via the communication session (has to be established prior to sharing), and the recipient caches it locally. From there on, the recipient can download the file at any time and decrypt it using the cached key.

# A shares uploaded file with B
$ cweb-cli -playground -ownId 8Xi fileShare -fileId HS8JndD9Stjcv04RqA2QayfW2FdN7ZvYNOqyzJZTQmo= -shareDirectlyToId QC3QvAlAF0TZQOjs75JUXv7wdKXjYGEV38vHlgkd_AQ=
{
  "success" : true
}

# B tries to downloads the file, but fails since it hasn't fetched the message from A containing the file reference
$ cweb-cli -playground -ownId QC3 fileDownload -fromId 8XiKJdThhgwgoBrBVAEgo-Il5kQqU_4EW-jAx0WiQ8E= -fileId HS8JndD9Stjcv04RqA2QayfW2FdN7ZvYNOqyzJZTQmo= -localPath document_received.pdf
{
  "error" : "Error downloading metadata: FILE_REFERENCE_NOT_FOUND",
  "success" : false,
  "tookTime" : 0
}

# B fetches messages, the file reference is silently consumed by the file sharing protocol and cached
$ cweb-cli -playground -ownId QC3 commSessionMessageRead -id 8XiKJdThhgwgoBrBVAEgo-Il5kQqU_4EW-jAx0WiQ8E=
{
  "messages" : [ ]
}

# This time B successfully downloads and decrypts the file
$ cweb-cli -playground -ownId QC3 fileDownload -fromId 8XiKJdThhgwgoBrBVAEgo-Il5kQqU_4EW-jAx0WiQ8E= -fileId HS8JndD9Stjcv04RqA2QayfW2FdN7ZvYNOqyzJZTQmo= -localPath document_received.pdf
{
  "fileMetadata" : {
    "name" : "document.pdf",
    "createdAt" : "2019-06-06 08:33:21.020",
    "size" : 1784683
  },
  "success" : true,
  "tookTime" : 103
}

# Confirm file integrity
$ diff document.pdf document_received.pdf

# A deletes the file from its online storage
$ cweb-cli -playground -ownId 8Xi fileUploadsDelete -fileId HS8JndD9Stjcv04RqA2QayfW2FdN7ZvYNOqyzJZTQmo=
{
  "fileId" : "HS8JndD9Stjcv04RqA2QayfW2FdN7ZvYNOqyzJZTQmo=",
  "success" : true
}

The -metadataOnly option of the fileDownload command allows to only see file metadata (name, size), and doesn’t download the file itself.

Source repos

Command Line Interface (CLI)

Creating New Identities

Adding Known Peers

Exchanging Direct Messages

File Storage and Sharing