Ctrl+K
What do you think about our new Documentation Layout? Take the survey
Back to Documentation Hub

Integration of External Services

Integration of External Services#

A typical example of external service integration is a cluster interacting with a database instance hosted by a third-party service provider. To deploy in Carbonio situations like this one, Carbonio Mesh is used.

Scenario and Requirements#

Our sample scenario consists of a Carbonio Multi-Server installation which includes:

  • One or more Carbonio Files Nodes

  • One node in the cluster (possibly different from the Carbonio Files Nodes) elected as terminating gateway

  • A PostgreSQL database, which is used by Carbonio Files, which is either

    • A server outside the Carbonio infrastructure

    • Hosted remotely by a third-party provider

    Note

    We will refer to this node as database node in the remainder of this guide.

../../_images/external.png

Fig. 12 The sample scenario used, with two Carbonio Files nodes and a database hosted remotely.#

The setup requires to access the command line on the terminating gateway to configure it, because the process requires manual file editing and running commands, although some commands towards the end of the procedure requires to access the database node.

Hint

It is highly suggest to use the Carbonio Mesh Administration Interface to better keep track of the configuration and changes. Please check Carbonio Mesh Administration Interface for directions on how to configure it and reach it.

Finally, keep the cluster credential password at hand, because it is required for token generation.

Let’s now start with the procedure, in which we first set up Carbonio Mesh, then install Carbonio Files.

Security and Setup#

The initial setup requires to complete a few steps.

Note

All commands must be executed on the node elected as terminating gateway, unless stated differently.

  1. Create a dedicated user

    # groupadd -r 'carbonio-gateway'
# useradd -r -M -g 'carbonio-gateway' -s /sbin/nologin 'carbonio-gateway'

  2. Define policies. It is necessary to make Carbonio Mesh aware of the services to be routed, which in our scenario is the database for Carbonio Files, carbonio-files-db.

    First, create a directory that will store all the configuration.

    # mkdir -p /etc/carbonio/gateway/service-discover/

    Then edit file /etc/carbonio/gateway/service-discover/policies.json and paste in it this content.

    {
  "key_prefix": [
    {
      "carbonio-gateway/": {
        "policy": "read"
      }
    }
  ],
  "node_prefix": [
    {
      "": {
        "policy": "read"
      }
    }
  ],
  "service": [
    {
      "carbonio-gateway": {
        "policy": "write"
      },
      "carbonio-files-db": {
        "policy": "write"
      }
    }
  ]
}

    Finally, let consul pick up the new policy.

    # consul acl policy create -name "carbonio-gateway-policy" -description "Policy for carbonio-gateway" -rules  @/etc/carbonio/gateway/service-discover/policies.json

  3. Export a new bootstrap token, which is the one that allows to execute consul commands and access its APIs. To extract the bootstrap token, execute the following command and then type the cluster credential password.

    # export CONSUL_HTTP_TOKEN=$(service-discover bootstrap-token --setup)

  4. Generate a new token, which is associated to the policy and will be the only one needed to communicate with the external database.

    # consul acl token create -format json -policy-name carbonio-gateway-policy -description "Token for carbonio-gateway" | jq -r '.SecretID' > /etc/carbonio/gateway/service-discover/token

# chown carbonio-gateway:carbonio-gateway -R /etc/carbonio/gateway

Definition of the External Service#

To operate properly, the terminating gateway must be aware of the exact location of the remote service, therefore we define both the external service and how the terminating gateway can reach it and allow Carbonio Files nodes access to it.

There is yet no CLI command for this, but we can use the APIs for this purpose. Create file /etc/carbonio/gateway/service-discover/carbonio-files-db-external.json with content

{
  "Address": "external-database.example.com",
  "Node": "external-files-db-node",
  "NodeMeta": {
    "external-node": "true",
    "external-probe": "true"
  },
  "Service": {
    "ID": "carbonio-gateway",
    "Port": 5432,
    "Service": "carbonio-files-db"
  }
}

Note

Replace the value of Address with the actual URL of the external service.

Then, execute a curl request to register the external service.

# curl --request PUT --header "X-Consul-Token: ${CONSUL_HTTP_TOKEN}" --data @carbonio-files-db-external.json http://localhost:8500/v1/catalog/register

Services Routing#

Now that the terminating gateway and the service have been defined and registered, it is time to let Carbonio Mesh know the list of the services that can use the gateway.

To do so, place in file /etc/carbonio/gateway/service-discover/gateway-config.hcl the following content, which defines a carbonio-gateway as a terminating gateway for the carbonio-files-db service.

Kind = "terminating-gateway"
Name = "carbonio-gateway"
Services = [
 {
   Name = "carbonio-files-db"
   #CAFile = "/etc/carbonio/external-db-ca.pem"
   #SNI = "external-db.local"
 }
]

There are two commented entries in the above file: they are optional and may not be specified at all in the configuration.

CAFile

A specific SSL certificate for the service. This is usually not necessary, unless some very specific and complex scenario is set up. Indeed, it is consul that take charge of encrypting all the traffic among the nodes and with the external resources: services and clients contact consul on localhost, so it is safe that they talk in plain text with it. Data received from consul on localhost are immediately SSL-encrypted, before leaving the node.

SNI

The Server Name Indication is an additional layer of security on top of TLS, used to prevent name mismatch. In the common case that a single web server hosts many domains each with its own SSL certificate, whenever a client request is received, it may not be yet known by the web server which is the exact domain the client is trying to access, because the HTTPS TSL/SSL handshake takes place before the client send the actual HTTP request for the domain. This may cause the client to receive the wrong certificate and possibly terminate the secure connection. Using a SNI avoids this problem, because it allows to send the domain name right in the SSL/TSL handshake.

Make sure to write the configuration, by issuing the following command.

# consul config write /etc/carbonio/gateway/service-discover/gateway-config.hcl

At this point, we are almost done: configuration of Carbonio Mesh has now been completed. Let’s now go through the last few tasks.

Systemd Service#

Now, create a systemd unit to control whether the carbonio gateway is enabled or not and therefore whether access to the external DB is allowed. Create file /lib/systemd/system/carbonio-gateway.service and configure it with these content.

[Unit]
Description=Carbonio gateway for external services
Documentation=https://docs.zextras.com/
Requires=network-online.target
After=network-online.target

[Service]
Type=simple
ExecStart=/usr/bin/consul connect envoy \
    -token-file /etc/carbonio/gateway/service-discover/token \
    -admin-bind localhost:0 \
    -gateway=terminating \
    -register -service carbonio-gateway
Restart=on-failure
RestartSec=15
User=carbonio-gateway
KillMode=process
KillSignal=SIGKILL
LimitNOFILE=65536
TimeoutSec=120
TimeoutStopSec=120

[Install]
WantedBy=multi-user.target

Hint

You can modify the ExecStart option by adding -- -l debug at the end to produce more verbose logs. The option should then look like:

ExecStart=/usr/bin/consul connect envoy \
  -token-file /etc/carbonio/gateway/service-discover/token \
  -admin-bind localhost:0 \
  -gateway=terminating \
  -register -service carbonio-gateway -- -l debug

Once saved the file, reload systemd to make it aware of the new unit file, then enable the new carbonio-gateway service.

# systemctl daemon-reload
# systemctl enable carbonio-gateway

Configuration of carbonio-files-db#

Note

This step only applies when the external resource is a database, like in our scenario.

The configuration of the database, which includes transferring the DB credentials to Carbonio Mesh and create the DB’s, is usually done by the carbonio-files-db-bootstrap script. However, since the carbonio-files-db package is not installed, this task must be done manually using these commands on the terminating gateway.

  • configure database name

    # consul kv put carbonio-files/db-name <database-name>

  • configure username

    # consul kv put carbonio-files/db-username <username>

  • configure password

    # consul kv put carbonio-files/db-password <password>

Now, let’s log in to the database node, where it is necessary to create a postgres superuser. In this example, we assign password ScrtPsw987^2 to the user. Make sure to use a strong password of your choice.

First, become the postgres user and start a direct access to the database using the CLI client.

# sudo -u postgres psql

Then issue the following commands to create the user.

# CREATE ROLE "carbonio-files-adm" WITH LOGIN SUPERUSER encrypted password 'ScrtPsw987^2'; CREATE DATABASE "carbonio-files-adm" owner "carbonio-files-adm";

Once done, exit the client.

# \q

Carbonio Files Nodes Installation#

The installation of Carbonio Files is slightly different from the standard one in a Multi-Server. In particular, make sure that after the installation, the package carbonio-files-db is not installed on any node. In our scenario, indeed, the database functionalities are not provided by that package, but by the external service. Hence, to avoid conflicts, you need to uninstall it.

  • Install package carbonio-files-ui on each Proxy Node.

    # apt install carbonio-files-ui

    # dnf install carbonio-files-ui

  • Install these packages on both Nodes on which Carbonio Files should run. We suggest to install them on the two Stores Nodes.

    # apt install carbonio-storages-ce carbonio-files-ce carbonio-user-management

    # dnf install carbonio-storages-ce carbonio-files-ce carbonio-user-management

    The installation will end with message:

    ======================================================
Carbonio Files installed successfully!
You must run pending-setups to configure it correctly.
======================================================

    Hence, execute pending-setups

    # pending-setups -a

Remove Services From Catalog#

When the external resource is not needed anymore, for example because the database is brought in the company’s data center, it is straightforward to remove the configuration of the services.

  • Stop the systemd unit service and delete the configuration file

    # systemd stop carbonio-gateway
# systemd disable carbonio-gateway
# rm /lib/systemd/system/carbonio-gateway.service

  • Remove the gateway configuration.

    # consul config delete -kind terminating-gateway -name carbonio-gateway
# curl --request PUT --header "X-Consul-Token: ${CONSUL_HTTP_TOKEN}" http://localhost:8500/v1/agent/service/deregister/carbonio-gateway
# curl --request PUT --header "X-Consul-Token: ${CONSUL_HTTP_TOKEN}" http://localhost:8500/v1/agent/service/deregister/carbonio-files-db

Now you can install the carbonio-files-db package on any node and it will be immediately available to the Carbonio Files nodes.

Contents