Difference between revisions of "Cloudron"
m (→Create a *new* cloudron app and import backup) |
(→Create a *new* cloudron app and import backup) |
||
(5 intermediate revisions by the same user not shown) | |||
Line 180: | Line 180: | ||
==Backup and Restore== | ==Backup and Restore== | ||
− | ==== | + | ====Using minio (no cloudron)==== |
+ | [[minio|Minio Server install]] | ||
− | + | https://docs.minio.io/docs/minio-client-complete-guide | |
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
=== Configure Cloudron backup === | === Configure Cloudron backup === | ||
Line 242: | Line 193: | ||
===Create a *new* cloudron app and import backup=== | ===Create a *new* cloudron app and import backup=== | ||
− | Our cloudron server | + | Our cloudron server is configured to save backups on a minio server so, Cloudron will pull the backups from that minio server. If, for example, the cloudron server is not configued to use a remote backup server, the backup data need to be copied to thje local cloudron server. |
app_id: The long string of chars does not help to know what id (backup file you need to use). :( | app_id: The long string of chars does not help to know what id (backup file you need to use). :( | ||
Line 264: | Line 215: | ||
Now the cloudron install command will work. We can also define the app version we want to use. | Now the cloudron install command will work. We can also define the app version we want to use. | ||
$ cloudron install --backup app_75b3615a-cbb8-4e80-bb12-b6e59e9983de --appstore-id org.wordpress.cloudronapp@0.9.1 | $ cloudron install --backup app_75b3615a-cbb8-4e80-bb12-b6e59e9983de --appstore-id org.wordpress.cloudronapp@0.9.1 | ||
+ | |||
+ | ===Restore an app on a Cloudron virtualized (virtualbox) at locahost=== | ||
+ | |||
+ | Min 2G ram, Min 20G disk (5.8GB used after fresh cloudron installation) | ||
+ | |||
+ | Local virtualbox machine for testing. Install ubuntu server 16.04. Install Cloudron | ||
+ | |||
+ | ====DNS concerns==== | ||
+ | If you don't have a dns server set up somewhere then | ||
+ | |||
+ | On the virtual ubuntu cloudron machine | ||
+ | /etc/hostname localcloudron.net | ||
+ | /etc/hosts 127.0.0.1 localcloudron.net my.localcloudron.net www.localcloudron.net | ||
+ | On your PC edit /etc/hosts | ||
+ | 192.168.1.44 localcloudron.net my.localcloudron.net www.localcloudron.net | ||
+ | |||
+ | If you don't include the ".net" (or "com", etc) we'll get invalid email errors later when we try to boot apps. | ||
+ | |||
+ | |||
+ | On you PC connect to the new installation | ||
+ | $ cloudron login --allow-selfsigned | ||
+ | Cloudron Hostname: localcloudron.net | ||
+ | |||
+ | |||
+ | Restore an app on a local Cloudron instance running on Virtualbox. | ||
+ | |||
+ | This does not work | ||
+ | $ cloudron install --backup app_83069568-d742-4017-a09f-7aa6d857b9a1 --appstore-id org.wordpress.cloudronapp@1.0.0 | ||
+ | Location (subdomain): www | ||
+ | ERROR Failed to install app. Billing Required (402) | ||
+ | |||
+ | No billing info so we need to pull the image and install it manually. https://cloudron.io/documentation/cli/#installing-an-app-from-source-code-using-cloudron-cli | ||
+ | |||
+ | On your PC with Cloudron CLI installed | ||
+ | cd /tmp | ||
+ | git clone https://git.cloudron.io/cloudron/wordpress-app.git | ||
+ | cd wordpress-app | ||
+ | cloudron build | ||
+ | cloudron install | ||
+ | |||
+ | Now, 1) Copy your backup to /var/backups, 2) From the Cloudron web interface initiate a backup of the newly installed app 3) Look at /var/backups and see what was created 4) Replace the newly created backup with the backup file that contains your precious information 5) From the web interface Restore. | ||
+ | |||
+ | App restored! | ||
==Orquestration== | ==Orquestration== | ||
* https://kubernetes.io/ open-source system for automating deployment, scaling, and management of containerized applications. | * https://kubernetes.io/ open-source system for automating deployment, scaling, and management of containerized applications. | ||
* https://www.kontena.io developer friendly container platform with all batteries included that is extremely easy to use and works for everybody on any cloud. | * https://www.kontena.io developer friendly container platform with all batteries included that is extremely easy to use and works for everybody on any cloud. |
Latest revision as of 18:04, 20 September 2017
Cloudron is a powerful free-as-in-freedom cloud platform that allows you to self-host webapps efficiently while providing unified user accounts, various automated sysadmin services, tested update pathways for dozens of mature free software apps in the Cloudron appstore and a RESTful API.
At the FKI, together with a range of actors in Barcelona we are currently testing the Cloudron.
Contents
Installation instructions
Cloudron's reference documentation contains installation instructions.
We start with a clean Ubuntu machine provided by our community datacentre partner La Mar De Bits.
We point the main domain name A record with a wildcard (*) to the IP of the server. Some ISPs (like Digital Ocean and AWS/Route 53) provide an API to their DNS servers to which the Cloudron makes calls to manage the domain name. In this case we have a domain at https://dinahosting.com/, which seems to have an API for their DNS. We could check with the Cloudron team to see if they can add support for this ISP.
We download and run the installation script with: $ wget https://cloudron.io/cloudron-setup $ chmod +x cloudron-setup $ sudo ./cloudron-setup --provider generic
And after a while (15-30") we browse to the IP address of the server where we accept the generated and unrecognised SSL certificate. Then we can create an admin user and we are in the my. subdomain where we can configure all the rest.
We want to enable the email server, but when doing so a message appears: "No DNS provider is setup. Displayed DNS records will have to be setup manually." Logically so, as we had only pointed a wildcard A record to the server. So we manually add DNS records at our domain dame service provider for the MX, TXT records as detailed in the Email setup section under settings (check the "advanced" link underneath each record in settings). Only I'm not sure how to deal with the PTR .in-addr.arpa address: where can one register such reverse address lookup?
Installing apps
Cloudron has an apps store that runs by default at cloudron.io and provides dozens of mature free software apps ported to the Cloudron platform.
Rocket.chat
We click the Rocket.chat button in the store and it asks us under what subdomain we want to install it and what users will be able to use it (we choose: all). Then there is a warning: "The first user to login becomes the Rocket.Chat administrator This administrator can mark other users as administators."
Phabricator
installation at "phabricator." subdomain. Warning that the RAM is too little. Message: "This Phabricator app does not integration with Cloudron User management. To complete the installation, setup an administrative account and setup an Auth provider. If you accidentally log yourself out before adding an Auth provider, you must use the CLI tool to recover it (or simply re-install phabricator). See T8282 for more information. We simply fill in "my.femprocomuns.cat" for the domain name and confirm the creation of this authentication provider. But no, that doesn't allow users to authenticate.
You can add users using the 'People' section on the Homepage. See the https://secure.phabricator.com/book/phabricator/article/configuringaccountsand_registration/ for more information. Uploading large files
How can I upload large files?
This app is configured to accept files upto 512MB. Note that large files need to be dragged and dropped (instead of the file upload button). See Q216"
- ISSUE: how to configure Phabricator? We should somehow configure LDAP authentication inside Phabricator in such way that it authenticates against the cloudron's LDAP user accounts.
Configure LDAP
Configure here https://phabricator.femprocomuns.cat/auth/
I copied these params from RocketChat LDAP config
LDAP Hostname, LDAP Port, Base Distinguished Name, and then Search Attributes: username
Roundcube
We chose "mail." as its subdomain and click install. It gives a message saying: "This application is pre-setup for Cloudron email usage. Just login with your Cloudron account. Note that email is only enabled for Cloudrons using custom domains."
When enabling email, all users get an email address on the server in the form of <user name<@<domain name>. This is interesting but we'd want to have that as an option and not by default. Now every email notification is sent to that new email address while users may not even have it set up and so don't receive the messages.
- ISSUE: Email accounts should be enabled on a per user basis and not by default for all.
Account Registration
By default admins can create user accounts. However we'd want a smooth self-registration process. Som reflections / criteria:
- a free account can be available but with limited resources (1 GB?) and strong measures to avoid SPAM
- full accounts require one to be or become a member/partner of the cooperative. femProcomuns has various typologies with different deposits to be made.
- additionally users can subscribe to a periodic, voluntary contribution, e.g. like Som Mobilitat does (X amount every Y months).
NextCloud
We'll want CollaboraOffice which combines own/nextCloud with LibreOffice, but for now we already set up nextCloud. " This app instance is pre-setup with an admin account. This app integrates with Cloudron authentication. Cloudron users can login and use NextCloud. However, admin status of Cloudron user is not carried over to NextCloud. For this reason, this app comes with an pre-setup admin user. This admin user can grant admin previleges to other users. username: admin password: changeme Please change the admin password on first login"
Piwik
This app provides web analytics. When installing it says: "This app integrates with Cloudron authentication. Cloudron users can login and use Piwik. However, admin status of Cloudron user is not carried over to Piwik. For this reason, this app comes with an pre-setup admin user. This admin user can grant admin previleges to other users. username: admin password: changeme Please change the admin password on first login".
Wallabag
Saves articles to read later (on mobile, tablet or desktop), stripping adds and non-article based added info. Message during installation: "Wallabag is not integrated with the Cloudron user management. The app comes with a pre-setup admin account with the following credentials: username: wallabag password: wallabag Please change the admin password on first login."
Mailtrain
App for managing subscriptions and sending out mass mails and newsletters. Message during installation: "Mailtrain is setup to send emails using the Cloudron SMTP server. An external server can be configured within the app. To revert from an external email server setup to the Cloudron setup, set the SMTP Hostname to localhost and then restart the app. This will inject the Cloudrons SMTP credentials automatically. Customizations. Additional settings can be added to /app/data/production.toml."
MediaWiki
Installation message: "This app provides a private wiki that can be edited by any Cloudron user. It is possible to tweak the access control settings to make it a public facing wiki or a public wiki edited only by Cloudron users. Use the CLI to edit the values in /app/data/AccessControl.php. Setting a custom icon The CLI tool can be used to set a custom icon at /app/data/images/wiki.png."
Minio
Installation message: "This application does not integrate with Cloudron authentication. Please use the following credentials to login: AccessKey: admin; SecretKey: secretkey Also make sure to generate new credentials under setting immediately."
Resolved Issues
The Phabricator was running very slow and self detected that it was a default low buffer size. On the host machine we adjusted in
$ /etc/mysql/mysql.conf.d
adding a line at the end (suggested to be 25% of total RAM of the vm):
innodb_buffer_pool_size=2400M
The Cloudron CLI
Install Cloudron CLI on your PC
https://cloudron.io/blog/2017-03-03-cli-part1.html
https://git.cloudron.io/cloudron/cloudron-cli
Enter a container with Cloudron CLI
$ cloudron list $ cloudron exec --app <app_name>
Helpful commands
Enter a container from within the host
Docker images and containers are not named in a descriptive manner. You can list the Container ID(s?) running at mail.example.net like this
# docker ps --filter "label=location=mail"
Now, with the Container ID we can enter the container
# docker exec -it <container_id> bash
Create a script called /usr/local/sbin/container (chmod +x)
#!/bin/bash output=`docker ps --filter "label=location=$1" | sed -n 2p` if [ -n "$output" ]; then array=(${output// / }) docker exec -it "${array[0]}" bash fi
Now we can enter the container of app running under the subdomain 'chat' with the command
# container chat
MySQL
You can access the mysql command line on the mysql container from within an app container.
$ cloudron list $ cloudron exec --app <app_name> $ mysql --user=${MYSQL_USERNAME} --password=${MYSQL_PASSWORD} --host=${MYSQL_HOST} ${MYSQL_DATABASE}
Integrations
PiWik
with the PiWik admin account I created a website tracking id for WordPress and for Rocket.chat and configured the settings of the PiWik plugin inside these both apps. That should them start being tracked.
ToDo Action Items
Some of the things that we need to do before going into production:
- configure SMTP relay
- set up a registration form and workflow to allow people to become users of the platform Issue 60
- set up a Minio backup server on some other server and configure backups (possibly vice versa)
- make sure users can choose to use the native email account or an external one Issue 60, Issue 267
- MediaWiki: by default Cloudron configures MW as a private wiki, but we want it to be public, i.e. accessible by all.
DONE:
- updated to new version from host machine with $ sudo npm install -g cloudron@0.42.1
- stopped minio app with CLI: $ cloudron stop --app minio
Authentication
LDAP
The LDAP service is not a fully fledged LDAP server like OpenLDAP.
A node.js script that 'emulates' what we would call a traditional LDAP server, listens on port 3002 (not 389 or 636). User credentials are kept in a mariadb database on the same host.
LDAP code: https://git.cloudron.io/cloudron/box/blob/master/src/ldap.js Library http://ldapjs.org/
Cloudron starts the node app on boot here:
/etc/systemd/system/box.service
OAuth 2.0
The same node.js that runs LDAP interface also exposes port 3000 and 3001. (3001 is used for admin user tasks). This nodejs script exposes an API. https://cloudron.io/references/api.html#profile
Nginx handles requests on port 443 coming from containers and forwards it to 3000.
/etc/nginx/applications/default.conf
Backup and Restore
Using minio (no cloudron)
https://docs.minio.io/docs/minio-client-complete-guide
Configure Cloudron backup
https://cloudron.io/documentation/backups/#backing-up-to-minio
Endpoint: https://min.my-domain.com
Create a *new* cloudron app and import backup
Our cloudron server is configured to save backups on a minio server so, Cloudron will pull the backups from that minio server. If, for example, the cloudron server is not configued to use a remote backup server, the backup data need to be copied to thje local cloudron server.
app_id: The long string of chars does not help to know what id (backup file you need to use). :(
From my PC with the cloudron CLI installed..
$ cloudron install --backup app_75b3615a-cbb8-4e80-bb12-b6e59e9983de --appstore-id org.wordpress.cloudronapp
This will fail. If we look at the nginx logs on the minio server we'll see
"HEAD /femprocomuns/test-server/app_75b3615a-cbb8-4e80-bb12-b6e59e9983de.tar.gz HTTP/1.1" 404 0 "-" "aws-sdk-nodejs/2.98.0 linux/v6.11.2 callback"
Where: femprocomuns is a Minio bucket and test-server is the bucket prefix.
Let's help nginx find the file
cd /opt/minio/data/femprocomuncs/test-server ln -s 2017-09-09-220000-031/app_75b3615a-cbb8-4e80-bb12-b6e59e9983de_2017-09-09-220003-079_v0.9.1.tar.gz ./app_75b3615a-cbb8-4e80-bb12-b6e59e9983de.tar.gz
Note: The cloudron app version is inclueded in the backup filename. v0.9.1
Now the cloudron install command will work. We can also define the app version we want to use.
$ cloudron install --backup app_75b3615a-cbb8-4e80-bb12-b6e59e9983de --appstore-id org.wordpress.cloudronapp@0.9.1
Restore an app on a Cloudron virtualized (virtualbox) at locahost
Min 2G ram, Min 20G disk (5.8GB used after fresh cloudron installation)
Local virtualbox machine for testing. Install ubuntu server 16.04. Install Cloudron
DNS concerns
If you don't have a dns server set up somewhere then
On the virtual ubuntu cloudron machine
/etc/hostname localcloudron.net /etc/hosts 127.0.0.1 localcloudron.net my.localcloudron.net www.localcloudron.net
On your PC edit /etc/hosts
192.168.1.44 localcloudron.net my.localcloudron.net www.localcloudron.net
If you don't include the ".net" (or "com", etc) we'll get invalid email errors later when we try to boot apps.
On you PC connect to the new installation
$ cloudron login --allow-selfsigned Cloudron Hostname: localcloudron.net
Restore an app on a local Cloudron instance running on Virtualbox.
This does not work
$ cloudron install --backup app_83069568-d742-4017-a09f-7aa6d857b9a1 --appstore-id org.wordpress.cloudronapp@1.0.0 Location (subdomain): www ERROR Failed to install app. Billing Required (402)
No billing info so we need to pull the image and install it manually. https://cloudron.io/documentation/cli/#installing-an-app-from-source-code-using-cloudron-cli
On your PC with Cloudron CLI installed
cd /tmp git clone https://git.cloudron.io/cloudron/wordpress-app.git cd wordpress-app cloudron build cloudron install
Now, 1) Copy your backup to /var/backups, 2) From the Cloudron web interface initiate a backup of the newly installed app 3) Look at /var/backups and see what was created 4) Replace the newly created backup with the backup file that contains your precious information 5) From the web interface Restore.
App restored!
Orquestration
- https://kubernetes.io/ open-source system for automating deployment, scaling, and management of containerized applications.
- https://www.kontena.io developer friendly container platform with all batteries included that is extremely easy to use and works for everybody on any cloud.