Difference between revisions of "Cloudron"

From Free Knowledge Free Technology Wiki
Jump to: navigation, search
(→‎Installing apps)
(→‎Create a *new* cloudron app and import backup)
 
(46 intermediate revisions by 2 users not shown)
Line 24: Line 24:
 
===Rocket.chat===
 
===Rocket.chat===
 
We click the Rocket.chat button in the store and it asks us under what subdomain we want to install it and what users will be able to use it (we choose: all). Then there is a warning: "The first user to login becomes the Rocket.Chat administrator This administrator can mark other users as administators."
 
We click the Rocket.chat button in the store and it asks us under what subdomain we want to install it and what users will be able to use it (we choose: all). Then there is a warning: "The first user to login becomes the Rocket.Chat administrator This administrator can mark other users as administators."
 +
  
 
===Phabricator===
 
===Phabricator===
Line 35: Line 36:
 
This app is configured to accept files upto 512MB. Note that large files need to be dragged and dropped (instead of the file upload button).
 
This app is configured to accept files upto 512MB. Note that large files need to be dragged and dropped (instead of the file upload button).
 
See Q216"
 
See Q216"
−
* ISSUE: how to configure Phabricator? We should somehow configure LDAP authentication inside Phabricator in such way that it authenticates against the cloudron's LDAP user accounts.  
+
* ISSUE: how to configure Phabricator? We should somehow configure LDAP authentication inside Phabricator in such way that it authenticates against the cloudron's LDAP user accounts.
 +
 
 +
====Configure LDAP====
 +
 
 +
Configure here https://phabricator.femprocomuns.cat/auth/
 +
 
 +
I copied these params from RocketChat LDAP config
 +
 
 +
    LDAP Hostname, LDAP Port, Base Distinguished Name, and then Search Attributes: username
  
 
===Roundcube===
 
===Roundcube===
Line 71: Line 80:
 
===MediaWiki===
 
===MediaWiki===
 
Installation message: "This app provides a private wiki that can be edited by any Cloudron user. It is possible to tweak the access control settings to make it a public facing wiki or a public wiki edited only by Cloudron users. Use the CLI to edit the values in /app/data/AccessControl.php. Setting a custom icon The CLI tool can be used to set a custom icon at /app/data/images/wiki.png."
 
Installation message: "This app provides a private wiki that can be edited by any Cloudron user. It is possible to tweak the access control settings to make it a public facing wiki or a public wiki edited only by Cloudron users. Use the CLI to edit the values in /app/data/AccessControl.php. Setting a custom icon The CLI tool can be used to set a custom icon at /app/data/images/wiki.png."
 +
 +
===Minio===
 +
Installation message: "This application does not integrate with Cloudron authentication. Please use the following credentials to login: AccessKey: admin;  SecretKey: secretkey Also make sure to generate new credentials under setting immediately."
 +
 +
===Resolved Issues===
 +
The Phabricator was running very slow and self detected that it was a default low buffer size. On the host machine we adjusted in
 +
    $ /etc/mysql/mysql.conf.d
 +
adding a line at the end (suggested to be 25% of total RAM of the vm):
 +
    innodb_buffer_pool_size=2400M
 +
 +
==The Cloudron CLI==
 +
 +
Install Cloudron CLI on your PC
 +
 +
https://cloudron.io/blog/2017-03-03-cli-part1.html
 +
 +
https://git.cloudron.io/cloudron/cloudron-cli
 +
 +
 +
===Enter a container with Cloudron CLI===
 +
 +
 +
    $ cloudron list
 +
    $ cloudron exec --app <app_name>
 +
  
 
==Helpful commands==
 
==Helpful commands==
 +
 +
===Enter a container from within the host===
 +
 
Docker images and containers are not named in a descriptive manner. You can list the Container ID(s?) running at mail.example.net like this
 
Docker images and containers are not named in a descriptive manner. You can list the Container ID(s?) running at mail.example.net like this
  
Line 80: Line 117:
  
 
     # docker exec -it <container_id> bash
 
     # docker exec -it <container_id> bash
 +
 +
Create a script called /usr/local/sbin/container (chmod +x)
 +
 +
  #!/bin/bash
 +
  output=`docker ps --filter "label=location=$1" | sed -n 2p`
 +
  if [ -n "$output" ]; then
 +
    array=(${output// / })
 +
    docker exec -it "${array[0]}" bash
 +
  fi
 +
 +
Now we can enter the container of app running under the subdomain 'chat' with the command
 +
 +
  # container chat
 +
 +
 +
===MySQL===
 +
 +
You can access the mysql command line on the mysql container from within an app container.
 +
 +
    $ cloudron list
 +
    $ cloudron exec --app <app_name>
 +
    $ mysql --user=${MYSQL_USERNAME} --password=${MYSQL_PASSWORD} --host=${MYSQL_HOST} ${MYSQL_DATABASE}
 +
 +
==Integrations==
 +
===PiWik===
 +
with the PiWik admin account I created a website tracking id for WordPress and for Rocket.chat and configured the settings of the PiWik plugin inside these both apps. That should them start being tracked.
 +
 +
==ToDo Action Items==
 +
Some of the things that we need to do before going into production:
 +
* configure SMTP relay
 +
* set up a registration form and workflow to allow people to become users of the platform [https://git.cloudron.io/cloudron/box/issues/60 Issue 60]
 +
* set up a Minio backup server on some other server and configure backups (possibly vice versa)
 +
* make sure users can choose to use the native email account or an external one [https://git.cloudron.io/cloudron/box/issues/60 Issue 60], [https://git.cloudron.io/cloudron/box/issues/267 Issue 267]
 +
* MediaWiki: by default Cloudron configures MW as a private wiki, but we want it to be public, i.e. accessible by all.
 +
 +
DONE:
 +
* updated to new version from host machine with $ sudo npm install -g cloudron@0.42.1
 +
* stopped minio app with CLI: $ cloudron stop --app minio
 +
 +
==Authentication==
 +
 +
===LDAP===
 +
 +
The LDAP service is not a fully fledged LDAP server like OpenLDAP.
 +
 +
A node.js script that 'emulates' what we would call a traditional LDAP server, listens on port 3002 (not 389 or 636). User credentials are kept in a mariadb database on the same host.
 +
 +
LDAP code: https://git.cloudron.io/cloudron/box/blob/master/src/ldap.js Library http://ldapjs.org/
 +
 +
Cloudron starts the node app on boot here:
 +
 +
      /etc/systemd/system/box.service
 +
 +
===OAuth 2.0===
 +
 +
The same node.js that runs LDAP interface also exposes port 3000 and 3001. (3001 is used for admin user tasks). This nodejs script exposes an API. https://cloudron.io/references/api.html#profile
 +
 +
Nginx handles requests on port 443 coming from containers and forwards it to 3000.
 +
 +
    /etc/nginx/applications/default.conf
 +
 +
==Backup and Restore==
 +
 +
====Using minio (no cloudron)====
 +
[[minio|Minio Server install]]
 +
 +
https://docs.minio.io/docs/minio-client-complete-guide
 +
 +
=== Configure Cloudron backup ===
 +
 +
https://cloudron.io/documentation/backups/#backing-up-to-minio
 +
 +
    Endpoint: https://min.my-domain.com
 +
 +
===Create a *new* cloudron app and import backup===
 +
 +
Our cloudron server is configured to save backups on a minio server so, Cloudron will pull the backups from that minio server. If, for example, the cloudron server is not configued to use a remote backup server, the backup data need to be copied to thje local cloudron server.
 +
 +
app_id: The long string of chars does not help to know what id (backup file you need to use). :(
 +
 +
From my PC with the cloudron CLI installed..
 +
 +
    $ cloudron install --backup app_75b3615a-cbb8-4e80-bb12-b6e59e9983de --appstore-id org.wordpress.cloudronapp
 +
 +
This will fail. If we look at the nginx logs on the minio server we'll see
 +
    "HEAD /femprocomuns/test-server/app_75b3615a-cbb8-4e80-bb12-b6e59e9983de.tar.gz HTTP/1.1" 404 0 "-" "aws-sdk-nodejs/2.98.0 linux/v6.11.2 callback"
 +
 +
Where: femprocomuns is a Minio bucket and test-server is the bucket prefix.
 +
 +
Let's help nginx find the file
 +
 +
  cd /opt/minio/data/femprocomuncs/test-server
 +
  ln -s 2017-09-09-220000-031/app_75b3615a-cbb8-4e80-bb12-b6e59e9983de_2017-09-09-220003-079_v0.9.1.tar.gz ./app_75b3615a-cbb8-4e80-bb12-b6e59e9983de.tar.gz
 +
 +
Note: The cloudron app version is inclueded in the backup filename. v0.9.1
 +
 +
Now the cloudron install command will work. We can also define the app version we want to use.
 +
    $ cloudron install --backup app_75b3615a-cbb8-4e80-bb12-b6e59e9983de --appstore-id org.wordpress.cloudronapp@0.9.1
 +
 +
===Restore an app on a Cloudron virtualized (virtualbox) at locahost===
 +
 +
Min 2G ram, Min 20G disk (5.8GB used after fresh cloudron installation)
 +
 +
Local virtualbox machine for testing. Install ubuntu server 16.04. Install Cloudron
 +
 +
====DNS concerns====
 +
If you don't have a dns server set up somewhere then
 +
 +
On the virtual ubuntu cloudron machine
 +
    /etc/hostname localcloudron.net
 +
    /etc/hosts 127.0.0.1 localcloudron.net my.localcloudron.net www.localcloudron.net
 +
On your PC edit /etc/hosts
 +
    192.168.1.44 localcloudron.net my.localcloudron.net www.localcloudron.net
 +
 +
If you don't include the ".net" (or "com", etc) we'll get invalid email errors later when we try to boot apps.
 +
 +
 +
On you PC connect to the new installation
 +
    $ cloudron login --allow-selfsigned
 +
    Cloudron Hostname: localcloudron.net
 +
 +
 +
Restore an app on a local Cloudron instance running on Virtualbox.
 +
 +
This does not work
 +
    $ cloudron install --backup app_83069568-d742-4017-a09f-7aa6d857b9a1 --appstore-id org.wordpress.cloudronapp@1.0.0
 +
    Location (subdomain): www
 +
    ERROR Failed to install app. Billing Required (402)
 +
 +
No billing info so we need to pull the image and install it manually. https://cloudron.io/documentation/cli/#installing-an-app-from-source-code-using-cloudron-cli
 +
 +
On your PC with Cloudron CLI installed
 +
    cd /tmp
 +
    git clone https://git.cloudron.io/cloudron/wordpress-app.git
 +
    cd wordpress-app
 +
    cloudron build
 +
    cloudron install
 +
 +
Now, 1) Copy your backup to /var/backups, 2) From the Cloudron web interface initiate a backup of the newly installed app 3) Look at /var/backups and see what was created 4) Replace the newly created backup with the backup file that contains your precious information 5) From the web interface Restore.
 +
 +
App restored!
 +
 +
==Orquestration==
 +
* https://kubernetes.io/ open-source system for automating deployment, scaling, and management of containerized applications.
 +
* https://www.kontena.io developer friendly container platform with all batteries included that is extremely easy to use and works for everybody on any cloud.

Latest revision as of 18:04, 20 September 2017

Cloudron is a powerful free-as-in-freedom cloud platform that allows you to self-host webapps efficiently while providing unified user accounts, various automated sysadmin services, tested update pathways for dozens of mature free software apps in the Cloudron appstore and a RESTful API.

At the FKI, together with a range of actors in Barcelona we are currently testing the Cloudron.

Installation instructions

Cloudron's reference documentation contains installation instructions.

We start with a clean Ubuntu machine provided by our community datacentre partner La Mar De Bits.

We point the main domain name A record with a wildcard (*) to the IP of the server. Some ISPs (like Digital Ocean and AWS/Route 53) provide an API to their DNS servers to which the Cloudron makes calls to manage the domain name. In this case we have a domain at https://dinahosting.com/, which seems to have an API for their DNS. We could check with the Cloudron team to see if they can add support for this ISP.

We download and run the installation script with: $ wget https://cloudron.io/cloudron-setup $ chmod +x cloudron-setup $ sudo ./cloudron-setup --provider generic

And after a while (15-30") we browse to the IP address of the server where we accept the generated and unrecognised SSL certificate. Then we can create an admin user and we are in the my. subdomain where we can configure all the rest.

We want to enable the email server, but when doing so a message appears: "No DNS provider is setup. Displayed DNS records will have to be setup manually." Logically so, as we had only pointed a wildcard A record to the server. So we manually add DNS records at our domain dame service provider for the MX, TXT records as detailed in the Email setup section under settings (check the "advanced" link underneath each record in settings). Only I'm not sure how to deal with the PTR .in-addr.arpa address: where can one register such reverse address lookup?

Installing apps

Cloudron has an apps store that runs by default at cloudron.io and provides dozens of mature free software apps ported to the Cloudron platform.

Rocket.chat

We click the Rocket.chat button in the store and it asks us under what subdomain we want to install it and what users will be able to use it (we choose: all). Then there is a warning: "The first user to login becomes the Rocket.Chat administrator This administrator can mark other users as administators."


Phabricator

installation at "phabricator." subdomain. Warning that the RAM is too little. Message: "This Phabricator app does not integration with Cloudron User management. To complete the installation, setup an administrative account and setup an Auth provider. If you accidentally log yourself out before adding an Auth provider, you must use the CLI tool to recover it (or simply re-install phabricator). See T8282 for more information. We simply fill in "my.femprocomuns.cat" for the domain name and confirm the creation of this authentication provider. But no, that doesn't allow users to authenticate.

You can add users using the 'People' section on the Homepage. See the https://secure.phabricator.com/book/phabricator/article/configuringaccountsand_registration/ for more information. Uploading large files 

   How can I upload large files?

This app is configured to accept files upto 512MB. Note that large files need to be dragged and dropped (instead of the file upload button). See Q216"

  • ISSUE: how to configure Phabricator? We should somehow configure LDAP authentication inside Phabricator in such way that it authenticates against the cloudron's LDAP user accounts.

Configure LDAP

Configure here https://phabricator.femprocomuns.cat/auth/

I copied these params from RocketChat LDAP config 

   LDAP Hostname, LDAP Port, Base Distinguished Name, and then Search Attributes: username

Roundcube

We chose "mail." as its subdomain and click install. It gives a message saying: "This application is pre-setup for Cloudron email usage. Just login with your Cloudron account. Note that email is only enabled for Cloudrons using custom domains."

Email

When enabling email, all users get an email address on the server in the form of <user name<@<domain name>. This is interesting but we'd want to have that as an option and not by default. Now every email notification is sent to that new email address while users may not even have it set up and so don't receive the messages.

  • ISSUE: Email accounts should be enabled on a per user basis and not by default for all.

Account Registration

By default admins can create user accounts. However we'd want a smooth self-registration process. Som reflections / criteria:

  • a free account can be available but with limited resources (1 GB?) and strong measures to avoid SPAM
  • full accounts require one to be or become a member/partner of the cooperative. femProcomuns has various typologies with different deposits to be made.
  • additionally users can subscribe to a periodic, voluntary contribution, e.g. like Som Mobilitat does (X amount every Y months).

NextCloud

We'll want CollaboraOffice which combines own/nextCloud with LibreOffice, but for now we already set up nextCloud. " This app instance is pre-setup with an admin account. This app integrates with Cloudron authentication. Cloudron users can login and use NextCloud. However, admin status of Cloudron user is not carried over to NextCloud. For this reason, this app comes with an pre-setup admin user. This admin user can grant admin previleges to other users. username: admin password: changeme Please change the admin password on first login"

Piwik

This app provides web analytics. When installing it says: "This app integrates with Cloudron authentication. Cloudron users can login and use Piwik. However, admin status of Cloudron user is not carried over to Piwik. For this reason, this app comes with an pre-setup admin user. This admin user can grant admin previleges to other users. username: admin password: changeme Please change the admin password on first login".

Wallabag

Saves articles to read later (on mobile, tablet or desktop), stripping adds and non-article based added info. Message during installation: "Wallabag is not integrated with the Cloudron user management. The app comes with a pre-setup admin account with the following credentials: username: wallabag password: wallabag Please change the admin password on first login."

Mailtrain

App for managing subscriptions and sending out mass mails and newsletters. Message during installation: "Mailtrain is setup to send emails using the Cloudron SMTP server. An external server can be configured within the app. To revert from an external email server setup to the Cloudron setup, set the SMTP Hostname to localhost and then restart the app. This will inject the Cloudrons SMTP credentials automatically. Customizations. Additional settings can be added to /app/data/production.toml."

MediaWiki

Installation message: "This app provides a private wiki that can be edited by any Cloudron user. It is possible to tweak the access control settings to make it a public facing wiki or a public wiki edited only by Cloudron users. Use the CLI to edit the values in /app/data/AccessControl.php. Setting a custom icon The CLI tool can be used to set a custom icon at /app/data/images/wiki.png."

Minio

Installation message: "This application does not integrate with Cloudron authentication. Please use the following credentials to login: AccessKey: admin; SecretKey: secretkey Also make sure to generate new credentials under setting immediately."

Resolved Issues

The Phabricator was running very slow and self detected that it was a default low buffer size. On the host machine we adjusted in 

   $ /etc/mysql/mysql.conf.d

adding a line at the end (suggested to be 25% of total RAM of the vm): 

   innodb_buffer_pool_size=2400M

The Cloudron CLI

Install Cloudron CLI on your PC

https://cloudron.io/blog/2017-03-03-cli-part1.html

https://git.cloudron.io/cloudron/cloudron-cli


Enter a container with Cloudron CLI

   $ cloudron list
   $ cloudron exec --app <app_name>


Helpful commands

Enter a container from within the host

Docker images and containers are not named in a descriptive manner. You can list the Container ID(s?) running at mail.example.net like this 

   # docker ps --filter "label=location=mail"

Now, with the Container ID we can enter the container 

   # docker exec -it <container_id> bash

Create a script called /usr/local/sbin/container (chmod +x) 

  #!/bin/bash
  output=`docker ps --filter "label=location=$1" | sed -n 2p`
  if [ -n "$output" ]; then
    array=(${output// / })
    docker exec -it "${array[0]}" bash
  fi

Now we can enter the container of app running under the subdomain 'chat' with the command 

  # container chat


MySQL

You can access the mysql command line on the mysql container from within an app container. 

   $ cloudron list
   $ cloudron exec --app <app_name>
   $ mysql --user=${MYSQL_USERNAME} --password=${MYSQL_PASSWORD} --host=${MYSQL_HOST} ${MYSQL_DATABASE}

Integrations

PiWik

with the PiWik admin account I created a website tracking id for WordPress and for Rocket.chat and configured the settings of the PiWik plugin inside these both apps. That should them start being tracked.

ToDo Action Items

Some of the things that we need to do before going into production:

  • configure SMTP relay
  • set up a registration form and workflow to allow people to become users of the platform Issue 60
  • set up a Minio backup server on some other server and configure backups (possibly vice versa)
  • make sure users can choose to use the native email account or an external one Issue 60, Issue 267
  • MediaWiki: by default Cloudron configures MW as a private wiki, but we want it to be public, i.e. accessible by all.

DONE:

  • updated to new version from host machine with $ sudo npm install -g cloudron@0.42.1
  • stopped minio app with CLI: $ cloudron stop --app minio

Authentication

LDAP

The LDAP service is not a fully fledged LDAP server like OpenLDAP.

A node.js script that 'emulates' what we would call a traditional LDAP server, listens on port 3002 (not 389 or 636). User credentials are kept in a mariadb database on the same host.

LDAP code: https://git.cloudron.io/cloudron/box/blob/master/src/ldap.js Library http://ldapjs.org/

Cloudron starts the node app on boot here: 

     /etc/systemd/system/box.service

OAuth 2.0

The same node.js that runs LDAP interface also exposes port 3000 and 3001. (3001 is used for admin user tasks). This nodejs script exposes an API. https://cloudron.io/references/api.html#profile

Nginx handles requests on port 443 coming from containers and forwards it to 3000. 

   /etc/nginx/applications/default.conf

Backup and Restore

Using minio (no cloudron)

Minio Server install

https://docs.minio.io/docs/minio-client-complete-guide

Configure Cloudron backup

https://cloudron.io/documentation/backups/#backing-up-to-minio 

   Endpoint: https://min.my-domain.com

Create a *new* cloudron app and import backup

Our cloudron server is configured to save backups on a minio server so, Cloudron will pull the backups from that minio server. If, for example, the cloudron server is not configued to use a remote backup server, the backup data need to be copied to thje local cloudron server.

app_id: The long string of chars does not help to know what id (backup file you need to use). :(

From my PC with the cloudron CLI installed.. 

   $ cloudron install --backup app_75b3615a-cbb8-4e80-bb12-b6e59e9983de --appstore-id org.wordpress.cloudronapp

This will fail. If we look at the nginx logs on the minio server we'll see 

   "HEAD /femprocomuns/test-server/app_75b3615a-cbb8-4e80-bb12-b6e59e9983de.tar.gz HTTP/1.1" 404 0 "-" "aws-sdk-nodejs/2.98.0 linux/v6.11.2 callback"

Where: femprocomuns is a Minio bucket and test-server is the bucket prefix.

Let's help nginx find the file 

  cd /opt/minio/data/femprocomuncs/test-server
  ln -s 2017-09-09-220000-031/app_75b3615a-cbb8-4e80-bb12-b6e59e9983de_2017-09-09-220003-079_v0.9.1.tar.gz ./app_75b3615a-cbb8-4e80-bb12-b6e59e9983de.tar.gz

Note: The cloudron app version is inclueded in the backup filename. v0.9.1

Now the cloudron install command will work. We can also define the app version we want to use. 

   $ cloudron install --backup app_75b3615a-cbb8-4e80-bb12-b6e59e9983de --appstore-id org.wordpress.cloudronapp@0.9.1

Restore an app on a Cloudron virtualized (virtualbox) at locahost

Min 2G ram, Min 20G disk (5.8GB used after fresh cloudron installation)

Local virtualbox machine for testing. Install ubuntu server 16.04. Install Cloudron

DNS concerns

If you don't have a dns server set up somewhere then

On the virtual ubuntu cloudron machine 

   /etc/hostname localcloudron.net
   /etc/hosts 127.0.0.1 localcloudron.net my.localcloudron.net www.localcloudron.net

On your PC edit /etc/hosts 

   192.168.1.44 localcloudron.net my.localcloudron.net www.localcloudron.net

If you don't include the ".net" (or "com", etc) we'll get invalid email errors later when we try to boot apps.


On you PC connect to the new installation 

   $ cloudron login --allow-selfsigned
   Cloudron Hostname: localcloudron.net


Restore an app on a local Cloudron instance running on Virtualbox.

This does not work 

   $ cloudron install --backup app_83069568-d742-4017-a09f-7aa6d857b9a1 --appstore-id org.wordpress.cloudronapp@1.0.0
   Location (subdomain): www
   ERROR Failed to install app. Billing Required (402)

No billing info so we need to pull the image and install it manually. https://cloudron.io/documentation/cli/#installing-an-app-from-source-code-using-cloudron-cli

On your PC with Cloudron CLI installed 

   cd /tmp
   git clone https://git.cloudron.io/cloudron/wordpress-app.git
   cd wordpress-app
   cloudron build
   cloudron install

Now, 1) Copy your backup to /var/backups, 2) From the Cloudron web interface initiate a backup of the newly installed app 3) Look at /var/backups and see what was created 4) Replace the newly created backup with the backup file that contains your precious information 5) From the web interface Restore.

App restored!

Orquestration

  • https://kubernetes.io/ open-source system for automating deployment, scaling, and management of containerized applications.
  • https://www.kontena.io developer friendly container platform with all batteries included that is extremely easy to use and works for everybody on any cloud.
Retrieved from "http://wiki.freeknowledge.eu/index.php?title=Cloudron&oldid=461"