Skip to content

I've been trying to get out into more Open Source communities of late. A good number of them are using Matrix as the chat platform for interacting with the community. I used to have a Matrix account, but it is lost to the sands of time. So a couple days ago I went to create a new one, but sadly the username I wanted was taken. What was I to do!?

Oh, and this is not really a how-to guide and there is a decent amount of info left out of this, so don't expect to go from 0 to Matrix using this blog post.

What I Did

Cool thing about Matrix. Decentralized and what not, AND it has a couple selfhostable options for standing up a server! So I did just that.

I fired up a terminal, got into one of my docker hosts, and went to work. I was mostly flying by the seat of other peoples pants (blog posts).

First Attempt

I loosely followed a guide on this lovely blog found while DuckDuckGo-ing terms like 'selfhosted matrix'

I really just nabbed the docker-compose.yml example, but the info there was also handy for how to get my username @my.domain and not @subdomain.my.domain

---
version: '3.3' # (1)

services:
  app:
    image: matrixdotorg/synapse:latest # (2)
    restart: always
    ports:
      - my-tailnet-ip:8008:8008 # (3)
    volumes:
      - ./data:/data # (4)

  1. 🤷‍♂️ I think in newer versions of docker-compose this is not needed. I seem to remember being yelled at about that...but I'm too lazy to look it up right now.

  2. I struggled to determine if there is an LTS or a 'stable' branch to point this at...so I just went latest

  3. I planned to reverse proxy this out-the-box so I locked down the port on the host to only listen on my tailnet interface.

  4. I still don't fully understand the difference between a docker volume and a bind mount, but I always just set this to a folder in the same directory as my docker-compose.yml

For the reverse proxy, I already have an instance of NPM running on the same docker host, so I just carved out a new config using that.

npm_host_config
NPM Config for matrix.my.domain
npm_host_config
SSL all the things!

This was all well and good, but it did not have two things going for it that I needed.

  1. It used the default Sqlite3 database, which is only for testing according to their docs...
  2. Federation did not work

Moving to Postgres

Fair warning...I've slept since I did this. I don't remember all the details. Your mileage may vary.

Synapse comes with a tool called synapse_port_db, however using that in this deployment was a bit complicated since it is all inside Docker containers.

What I ended up doing is first stopping all the services via docker-compose down, then creating a copy of data/homeserver.db to data/homeserver.db.snapshot.

Next, I copied the data/homeserver.yml to data/homeserver-postgres.yml, and edited data/homeserver-postgres.yml to include the database settings.

database:
  name: psycopg2
  args:
    user: synapse
    password: SomeSooperSecretPassword
    dbname: synapse
    host: db # (1)
    cp_min: 5
    cp_max: 10
  1. This is the name of the service as defined in docker-compose.yml for the database.

Then I updated my docker-compose.yml to include the new database container service.

services:
  app:
    image: matrixdotorg/synapse
    restart: always
    ports:
      - my-tailnet-ip:8008:8008
    volumes:
      - ./data:/data

  db:
    image: docker.io/postgres:12-alpine
    environment:
      - POSTGRES_USER=synapse
      - POSTGRES_PASSWORD=SooperSecretPassword
      - POSTGRES_INITDB_ARGS=--encoding=UTF-8 --lc-collate=C --lc-ctype=C # (1)
    volumes:
      - ./db:/var/lib/postgresql/data # (2)
  1. This is taken from the synapse docs for setting up the postgres database.
  2. I again wanted a folder in the same location of my docker-compose.yml

With those tweaks made, I stood up the services again via docker-compose up. You will have the sqlite version running still but now will have a postgresql database server running and ready to get the import.

Once that was running, I found the container ID for the synapse server in order to get a shell within it. You can find it easily with docker-compose ps and then use that ID in the command docker exec -it ID_YOU_FOUND bash.

Inside that shell, you should have access to the tool synapse_port_db which you can now use to port the database into the Postgresql server.

synapse_port_db --sqlite-database /data/homeserver.db.snapshot --postgres-config /data/homeserver.yml

Now the rub here is the import happens when your synapse server is already up. I was not sure how else to get access to the synapse_port_db command without the synapse service actually running in the container. More on that later.

Once the synapse_port_db command completes successfully, I shut down the services again.

Next I copied the data/homeserver.yml to data/homeserver-sqlite.yml, then copied data/homeserver-postgres.yml to data/homeserver.yml.

After starting the services again via docker-compose I was greeted with a nasty error message when the synase server tried to start, BUT the devs must have thought about this because the error message included the SQL I needed to run to fix it!

Using the same method of getting a shell inside a container, I was able to use psql -U synapse to run the suggested SQL on the database. Restarted services and BAM! Seemingly functional Synapse server 🎉

Federation

In order to get this to work AND in order for my Matrix username to be name@my.domain I needed to server up some well_known information at my.domain

There are a few ways to do this, but since I already had this blog sitting at my.domain AND have a host configured in NPM for it, I just added the following configs to the Advanced config section of my.domain.

        location /.well-known/matrix/client {
                return 200 '{"m.server": {"base_url": "my.domain:443"}}';
                default_type application/json;
                add_header Access-Control-Allow-Origin *;
        }

        location /.well-known/matrix/server {
                default_type application/json;
                add_header Access-Control-Allow-Origin *;
                return 200 '{"m.server":"my.domain:443"}';
        }
npm_adv_1
NPM Advanced Config for my.domain

Once I had that in place, I checked all was well using The Matrix Federation Tester

Final Thoughts

What the hell is wrong with me?! All of this AND some I didn't mention (like having to migrate the whole thing off my Linode to an on-prem server because the Linode couldn't handle it) just to get the username I wanted...

Comments