Storage Setup

Cweb supports any storage that implements Amazon S3 REST API. Below are examples of configuring a self-hosted MinIO S3 server, and AWS S3.

Note: The initial storage setup requires a desktop OS (Linux, Mac, Windows).

MinIO

MinIO is an S3-compatible storage server software that you run on a machine permanently connected to the internet - your own PC, Raspberry PI, or an instance from AWS, GCP, Azure, DigitalOcean.

Follow MinIO Quickstart Guide for spinning up a server instance, creating a bucket, and configure read and write access keys. Once you create a bucket and obtain its HOST, PORT, BUCKET_NAME, and access keys, the private storage profile is constructed as follows:

s3!http://HOST:PORT!us-east-1!BUCKET_NAME!cweb!READ_ACCESS_KEY_ID!READ_SECRET_ACCESS_KEY!WRITE_ACCESS_KEY_ID!WRITE_SECRET_ACCESS_KEY

Example of creating new bucket and configuring access keys:

# Create a bucket
mc mb myminio/my-bucket

# Create a reader access key
mc admin user add myminio my-bucket-cweb-reader <READ_SECRET_ACCESS_KEY>
mc admin policy set myminio readonly user=my-bucket-cweb-reader 

# Create a writer access key
mc admin user add myminio my-bucket-cweb-writer <WRITE_SECRET_ACCESS_KEY>
mc admin policy set myminio readwrite user=my-bucket-cweb-writer

Construct private storage profile from the information obtained above:

s3!http://35.126.38.145:9000!us-east-1!my-bucket!cweb!my-bucket-cweb-reader!<READ_SECRET_ACCESS_KEY>!my-bucket-cweb-writer!<WRITE_SECRET_ACCESS_KEY>

AWS S3

Amazon provides S3 storage as a service, i.e., you do not need to manage S3 server yourself.

Create AWS account at aws.amazon.com/free. Create an admin user that is authorized to create S3 buckets: In AWS Console, navigate to Services → IAM → Users → Add User → Enter name (say, admin) and check “Programmatic access” → Next → Attach existing policies directly → select IAMFullAccess and AmazonS3FullAccess → Next:Review → Create User. Save user credentials.

Install and set up AWS CLI using the credentials of the admin user you created above.

Download the cweb-create-aws-s3-bucket.sh shell script, and invoke it as follows:

$ cweb-create-aws-s3-bucket.sh <AWS_REGION> <BUCKET_NAME>

Pick AWS Region from the region list - any region would do, but there are some price differences, and closer regions may have lower access latency.

Example invocation:

$ cweb-create-aws-s3-bucket.sh us-west-2 my-bucket
Created bucket my-bucket
Created user my-bucket-cweb-reader
Created access key for my-bucket-cweb-reader
Attached policy to my-bucket-cweb-reader
Created user my-bucket-cweb-writer
Created access key for my-bucket-cweb-writer
Attached policy to my-bucket-cweb-writer
-------------- privateStorageProfile --------------
s3!https://s3-us-west-2.amazonaws.com!us-west-2!my-bucket!cweb!AXCETV2Y5DIDF!CSlCeTRjdn1eYLH7CY5OQF/38BtJ+w!AXCETV2Y5DIDF!CSlCeTRjdn1eYLH7CY5OQF

Testing Storage Profile

To test if a storage profile is valid, use Cweb CLI:

$ cweb-cli storageProfileTest -privateStorageProfile 's3!https://s3-us-west-2.amazonaws.com!us-west-2!my-bucket!cweb!AXCETV2Y5DIDF!CSlCeTRjdn1eYLH7CY5OQF/38BtJ+w!AXCETV2Y5DIDF!CSlCeTRjdn1eYLH7CY5OQF'
{
  "success" : true
}

Keeping the storage secure and revocation

Keep in mind that private storage profile gives its holder read and write access to the bucket, so share only with ones that you trust will not abuse it, e.g., by using it as free public storage.

In the steps above we have created *-cweb-writer and *-cweb-reader users with read+write and read-only access respectively, whose access keys are embedded in storage profiles. To revoke the access simply delete the users or rotate their keys.