Integration of External Services#
A typical example of external service integration is a cluster interacting with a database instance hosted by a third-party service provider. To deploy in Carbonio CE situations like this one, Carbonio Mesh is used.
Scenario and Requirements#
Our sample scenario consists of a Carbonio CE Multi-Server installation which includes:
One or more Carbonio Files Nodes
One node in the cluster (possibly different from the Carbonio Files Nodes) elected as terminating gateway
-
A PostgreSQL database, which is used by Carbonio Files, which is either
A server outside the Carbonio CE infrastructure
Hosted remotely by a third-party provider
Note
We will refer to this node as database node in the remainder of this guide.
The setup requires to access the command line on the terminating gateway to configure it, because the process requires manual file editing and running commands, although some commands towards the end of the procedure requires to access the database node.
Hint
It is highly suggest to use the Carbonio Mesh Administration Interface to better keep track of the configuration and changes. Please check Carbonio Mesh Administration Interface for directions on how to configure it and reach it.
Finally, keep the cluster credential password at hand, because it is required for token generation.
Let’s now start with the procedure, in which we first set up Carbonio Mesh, then install Carbonio Files.
Security and Setup#
The initial setup requires to complete a few steps.
Note
All commands must be executed on the node elected as terminating gateway, unless stated differently.
-
Create a dedicated user
# groupadd -r 'carbonio-gateway' # useradd -r -M -g 'carbonio-gateway' -s /sbin/nologin 'carbonio-gateway'
-
Define policies. It is necessary to make Carbonio Mesh aware of the services to be routed, which in our scenario is the database for Carbonio Files, carbonio-files-db.
First, create a directory that will store all the configuration.
# mkdir -p /etc/carbonio/gateway/service-discover/
Then edit file
/etc/carbonio/gateway/service-discover/policies.json
and paste in it this content.{ "key_prefix": [ { "carbonio-gateway/": { "policy": "read" } } ], "node_prefix": [ { "": { "policy": "read" } } ], "service": [ { "carbonio-gateway": { "policy": "write" }, "carbonio-files-db": { "policy": "write" } } ] }
Finally, let
consul
pick up the new policy.# consul acl policy create -name "carbonio-gateway-policy" -description "Policy for carbonio-gateway" -rules @/etc/carbonio/gateway/service-discover/policies.json
-
Export a new bootstrap token, which is the one that allows to execute
consul
commands and access its APIs. To extract the bootstrap token, execute the following command and then type the cluster credential password.# export CONSUL_HTTP_TOKEN=$(service-discover bootstrap-token --setup)
-
Generate a new token, which is associated to the policy and will be the only one needed to communicate with the external database.
# consul acl token create -format json -policy-name carbonio-gateway-policy -description "Token for carbonio-gateway" | jq -r '.SecretID' > /etc/carbonio/gateway/service-discover/token # chown carbonio-gateway:carbonio-gateway -R /etc/carbonio/gateway
Definition of the External Service#
To operate properly, the terminating gateway must be aware of the exact location of the remote service, therefore we define both the external service and how the terminating gateway can reach it and allow Carbonio Files nodes access to it.
There is yet no CLI command for this, but we can use the APIs for this
purpose. Create file
/etc/carbonio/gateway/service-discover/carbonio-files-db-external.json
with content
{
"Address": "external-database.example.com",
"Node": "external-files-db-node",
"NodeMeta": {
"external-node": "true",
"external-probe": "true"
},
"Service": {
"ID": "carbonio-gateway",
"Port": 5432,
"Service": "carbonio-files-db"
}
}
Note
Replace the value of Address with the actual URL of the external service.
Then, execute a curl
request to register the external service.
# curl --request PUT --header "X-Consul-Token: ${CONSUL_HTTP_TOKEN}" --data @carbonio-files-db-external.json http://localhost:8500/v1/catalog/register
Services Routing#
Now that the terminating gateway and the service have been defined and registered, it is time to let Carbonio Mesh know the list of the services that can use the gateway.
To do so, place in file
/etc/carbonio/gateway/service-discover/gateway-config.hcl
the
following content, which defines a carbonio-gateway as a terminating
gateway for the carbonio-files-db
service.
Kind = "terminating-gateway"
Name = "carbonio-gateway"
Services = [
{
Name = "carbonio-files-db"
#CAFile = "/etc/carbonio/external-db-ca.pem"
#SNI = "external-db.local"
}
]
There are two commented entries in the above file: they are optional and may not be specified at all in the configuration.
- CAFile
-
A specific SSL certificate for the service. This is usually not necessary, unless some very specific and complex scenario is set up. Indeed, it is
consul
that take charge of encrypting all the traffic among the nodes and with the external resources: services and clients contactconsul
on localhost, so it is safe that they talk in plain text with it. Data received fromconsul
on localhost are immediately SSL-encrypted, before leaving the node. - SNI
-
The Server Name Indication is an additional layer of security on top of TLS, used to prevent name mismatch. In the common case that a single web server hosts many domains each with its own SSL certificate, whenever a client request is received, it may not be yet known by the web server which is the exact domain the client is trying to access, because the HTTPS TSL/SSL handshake takes place before the client send the actual HTTP request for the domain. This may cause the client to receive the wrong certificate and possibly terminate the secure connection. Using a SNI avoids this problem, because it allows to send the domain name right in the SSL/TSL handshake.
Make sure to write the configuration, by issuing the following command.
# consul config write /etc/carbonio/gateway/service-discover/gateway-config.hcl
At this point, we are almost done: configuration of Carbonio Mesh has now been completed. Let’s now go through the last few tasks.
Systemd Service#
Now, create a systemd
unit to control whether the carbonio gateway
is enabled or not and therefore whether access to the external DB is
allowed. Create file
/lib/systemd/system/carbonio-gateway.service
and configure it
with these content.
[Unit]
Description=Carbonio gateway for external services
Documentation=https://docs.zextras.com/
Requires=network-online.target
After=network-online.target
[Service]
Type=simple
ExecStart=/usr/bin/consul connect envoy \
-token-file /etc/carbonio/gateway/service-discover/token \
-admin-bind localhost:0 \
-gateway=terminating \
-register -service carbonio-gateway
Restart=on-failure
RestartSec=15
User=carbonio-gateway
KillMode=process
KillSignal=SIGKILL
LimitNOFILE=65536
TimeoutSec=120
TimeoutStopSec=120
[Install]
WantedBy=multi-user.target
Hint
You can modify the ExecStart
option by adding -- -l
debug
at the end to produce more verbose logs. The option should
then look like:
ExecStart=/usr/bin/consul connect envoy \
-token-file /etc/carbonio/gateway/service-discover/token \
-admin-bind localhost:0 \
-gateway=terminating \
-register -service carbonio-gateway -- -l debug
Once saved the file, reload systemd
to make it aware of the new unit file, then
enable the new carbonio-gateway
service.
# systemctl daemon-reload
# systemctl enable carbonio-gateway
Configuration of carbonio-files-db
#
Note
This step only applies when the external resource is a database, like in our scenario.
The configuration of the database, which includes transferring the DB credentials to Carbonio Mesh and create the DB’s, is usually done by the carbonio-files-db-bootstrap script. However, since the carbonio-files-db package is not installed, this task must be done manually using these commands on the terminating gateway.
-
configure database name
# consul kv put carbonio-files/db-name <database-name>
-
configure username
# consul kv put carbonio-files/db-username <username>
-
configure password
# consul kv put carbonio-files/db-password <password>
Now, let’s log in to the database node, where it is necessary to
create a postgres
superuser. In this example, we assign password
ScrtPsw987^2 to the user. Make sure to use a strong password of
your choice.
First, become the postgres
user and start a direct access to the
database using the CLI client.
# sudo -u postgres psql
Then issue the following commands to create the user.
# CREATE ROLE "carbonio-files-adm" WITH LOGIN SUPERUSER encrypted password 'ScrtPsw987^2'; CREATE DATABASE "carbonio-files-adm" owner "carbonio-files-adm";
Once done, exit the client.
# \q
Carbonio Files Nodes Installation#
The installation of Carbonio Files is slightly different from the standard one in a Multi-Server. In particular, make sure that after the installation, the package carbonio-files-db is not installed on any node. In our scenario, indeed, the database functionalities are not provided by that package, but by the external service. Hence, to avoid conflicts, you need to uninstall it.
-
Install package
carbonio-files-ui
on each Proxy Node.# apt install carbonio-files-ui
# dnf install carbonio-files-ui
-
Install these packages on both Nodes on which Carbonio Files should run. We suggest to install them on the two Stores Nodes.
# apt install carbonio-storages-ce carbonio-files-ce carbonio-user-management
# dnf install carbonio-storages-ce carbonio-files-ce carbonio-user-management
The installation will end with message:
====================================================== Carbonio Files installed successfully! You must run pending-setups to configure it correctly. ======================================================
Hence, execute pending-setups
# pending-setups -a
Remove Services From Catalog#
When the external resource is not needed anymore, for example because the database is brought in the company’s data center, it is straightforward to remove the configuration of the services.
-
Stop the systemd unit service and delete the configuration file
# systemd stop carbonio-gateway # systemd disable carbonio-gateway # rm /lib/systemd/system/carbonio-gateway.service
-
Remove the gateway configuration.
# consul config delete -kind terminating-gateway -name carbonio-gateway # curl --request PUT --header "X-Consul-Token: ${CONSUL_HTTP_TOKEN}" http://localhost:8500/v1/agent/service/deregister/carbonio-gateway # curl --request PUT --header "X-Consul-Token: ${CONSUL_HTTP_TOKEN}" http://localhost:8500/v1/agent/service/deregister/carbonio-files-db
Now you can install the carbonio-files-db package on any node and it will be immediately available to the Carbonio Files nodes.