Matrix.org hit some IO problems earlier this week, while largely a problem with our hosting provider, we’re spending a bit of time to make Synapse more resilient if the same thing were to happen again. This will mean the ability to shard the DB (by table) and spread the load so we are not so dependent on high performance from a given db box. Outside of that we’ve been working on the final polishing of the room directory and getting the sqlite -> Postgres port script into better shape.
Not much for Dendrite this week as anoa is off at Ubucon 2019. But we had a few valuable bugs reported by the community, and a pressing reminder to get Dendrite's Monolith mode in as part of its CI.
I've not got much for this week, but the IRC bridge has been undergoing some serious refactors and changes for a larger release. Should be quite a big one when it lands :)
A PR for riot-web has emerged that adds support for search in E2E encrypted rooms. The PR is utilizing Seshat to perform event indexing and search on riot-desktop. While the PR is missing any sort of UI, it is in a usable state.
RiotX: We have fixed quite a lot of issues during the stabilization sprint. We are now working on Sprint 4: read marker, report content, mark all room read, etc. François is changing the media/file picker and we will also be able to share elements from other apps to RiotX. We will schedule a release soon (tm) (should have happen this week, but has been delayed due to stabilization)
Polyjuice Client, a Matrix client library for Elixir, has a new release. There is now a short tutorial that will teach you how to make a simple echo bot with it.
Then:
Igor, a bot framework for Elixir, has had its first release.
Last week the spec core team said they'd start focusing on 3 MSCs per week. Those were MSC2290, MSC2176 and MSC1219. The first two have entered FCP, and MSC1219 will roll over into this week.
Thus, the 3 MSCs the spec core team will be focusing on this week will be MSC2199, MSC1219 and an up and coming security-related MSC. Join us in #matrix-spec:matrix.org for related spec discussions :)
There is also a new MSC, MSC2312 describing the proposed Matrix URI scheme. This is a remake of a much older proposal. The general idea is to make a standard for Matrix URIs: matrix:.
Coming up, improved room directory perf, ironing out wrinkles in the room upgrade UX as well as a major reliability boost in the sqlite -> Postgres db porting script.
Dendrite's latest hiatus has come to a close after the privacy work had taken so much of my time. Thankfully although PR review is blocked on the dendrite team, the community have continued to submit PRs and even review each other's PRs (thanks cnly !).
Fixes this week have mostly focused around the CI. We finally got Dendrite's CI unborked (t'was borked in a half-way transition between CircleCI and Buildkite), but it's now working and faster than ever. We're also working to add the Sytest test failure results to the top of the CI window such that people can see which tests failed and the associated Dendrite logs.
Progress of dendrite is tracked in Dendrite's milestones. We're currently aiming for #1 (Client-Side) Bot Hosting.
CI not running for community PRs - Buildkite hosts currently run multiple PRs from many different projects, so we can't trust arbitrary code to run on them quite yet. There's a project in progress to run the code in a sandboxed environment (think VM or container) to lift this restriction :)
As a proof of concept, I have created a little Matrix client with Flutter for Web, named FluffyWeb. This client has only basic features but it shows the possibilities of Flutter for Web and it seems to work fine so far. The client has a responsive design and should work on mobile fine too. Check it out at: https://christianpauly.gitlab.io/fluffyweb/
(Not working in Internet Explorer - I recommend the AOL Messenger in this case)
Just released version 1.4.0 of the Ruby SDK, the main change in this release is the addition of a naïve set of methods to replace the logger implementation. This should allow the gem to slot more easily into projects where existing logging configuration is already in place.
With the release of Synapse 1.4.0, there was a rush to get packages and containers updated, the community are always fast, but we should acknowledge Anance for having his k8s images updated within a few minutes!
As always, referring to the project's changelog before upgrading is a good idea.
KUDOS to this this project! I love that I can git pull, ansible-playbook -i inventory/hosts setup.yml --tags=setup-all without really checking and my toy homeserver is automagically updated.
Let's reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
Following on from last week, the 3 MSCs the spec core team have chosen to focus on this week are: 2290 (3pid binding endpoints), 2176 (redaction rules), and 1219 (key backups), spurred on by finishing off the spec work for the privacy sprint, and cross signing.
MSC Updates:
News from 2019-09-20 09:00:00 until 2019-09-27 20:07:55.
This week we put out a release candidate for 1.4.0 which support a whole host of privacy features including greater control over interactions with identity servers, cleaning up redacted events and user meta data (like IPs and user agents) and warning users when they are using the default trusted key server (matrix.org).
Aside from privacy, the thing that is most exciting is switching on our solution for mitigating forward extremities build up by default. It should make a big difference for the CPU of servers in fragmented rooms.
matrix-corporal v1.6.0 was recently released to address an issue when used in conjunction with Synapse Single-Sign-On login flows (CAS or SAML).
Until now, matrix-corporal used to interfere with /login requests and demand that users always authenticate with a username/password.
Since v1.6.0, you can use matrix-corporal for automatic management of users/rooms/communities, while letting authentication happen through SSO (as provided by Synapse).
I'm also working on a maubot that accepts invites, announces them in a control room and then invites people from that control room when requested. That bot should be ready NWIM.
The slack bridge got an RC3 today and it is now LIVE on matrix.org!! This brings in a whole host of new changes like speedups for message processing, typing notifications and more reliable edits/reactions/replies. Since the 1.0 release required a migration of data files, we have made every effort to migrate integrations across. If you find that your slack bridge (hosted on matrix.org) is no longer working, please contact me
Work on Bifrost has resumed! We're doing our best to refactor and replace bits that were hacked together at the start of the year and really improve on reliability and documentation. The official matrix.org bridge awaiting our work on PostgreSQL before we can move it further, but the project is accelerating :) https://github.com/matrix-org/matrix-bifrost/tree/develop
Hello. I made a thing for using the Vim text editor for sending messages in Riot, and a friend suggested I share it here. Here it is: https://gitlab.com/MRAAGH/riot-vim#riot-vim
I have extracted Igor's guts, and have distilled them into a new client library for Elixir: polyjuice_client. I've also started working on a library of Matrix utility functions for Elixir that would be relevant to multiple components (clients, application services, homeservers, identity servers): polyjuice_util.
RiotX: A big work to stabilize the application and to implement little missing feature has been done. Also Two releases has been done this week. Please refer the the changelog for a (rather) complete list of what has been done (https://github.com/vector-im/riotX-android/blob/develop/CHANGES.md)
Also the read marker feature is in review. There are still remaining bugs which will be fixed before the merge.
I made a docker image for Synapse (again). However, this time it uses PyPy3. It is a drop-in replacement for matrixdotorg/synapse. Anyone is welcomed to test the impact of it on CPU utilization.
The matrix-fly-paper bot has been renamed to matrix-fly-swatter. On reflection, I realized the original goals for the bot were too ambitious and the project has become an exercise in becoming familiar with the client<->server api. The planned features have been cut back to simply automating sending m.room.server_acl events to several rooms at once. At a later date, planned features may be expanded, but there are other projects I'd have more fun working on in the meantime.
It's the section where we reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
The Spec Core Team is going to try and select 3 MSCs to prioritise per week to work on, in order to not stretch resources too thin and to communicate clearly what it is we're actually up to. More details to follow.
Otherwise this week has been continuing to iterate on existing MSCs, as well as updating the MSC template to rename the "Tradeoffs" section to "Alternatives", and to get rid of "Conclusion" which was always just an unnecessary repeat of the introduction.
Construct did a round of performance work and pursued some of the easier opportunities with the most effective returns. At present, the biggest performance bottleneck Construct experiences is with I/O, specifically latency from accessing disks in a random pattern. This week a few open opportunities for prefetch pipelines were taken, allowing the server to stream data directly from the disk to the client with minimal buffering. This was implemented in several places, mostly for the /media/ system, and also for the /members lists, improving the performance of requesting the members of a room like #whalepool:ericmartindale.com .
There is one area though that requires a special mention for being an order of magnitude more impactful than any other area where a pipelined stream was placed. We have analyzed the way Riot conducts a room change, and by the simple placement of a prefetch operation when the server receives an im.vector.breadcrumb_rooms, which comes early during the room change, Construct is able to populate the room's timeline significantly faster. That has a compounding effect, as the browser initiates several requests based on (and after) the received timeline data- meaning those responses come significantly faster.
Synapse
This week, lots of hacking but not much in the way of news. We’ve been focusing on more privacy improvements as well as polishing up the SAML integration.
Next week, expect a wodge of privacy related features to land.
The Matrix Slack bridge has had another release candidate this week. 1.0 RC2 is out now. We've rewritten the codebase in Typescript and made many changes, and need your feedback before release so please do not hold back on testing it! The 1.0 release is slated to arrive sometime next week :)
Presenting famedly-email-bridge, a new email bridge! It is still early in development but can already send and receive emails.
Features so far:
Receive emails by a prefix (e.g. matrix_sorunome@myserver.com, matrix_otheruser@myserver.com)
Send emails with said prefix
Receiving emails parses plain body, html body and attachments
Sending emails collects all messages for a given amount of time. After that it creates an email based on that! Supports edits, redactions and files (image, video, audio, file)!
Things planned for the future:
Route emails to channels based on threading (reply-chain)
Send emails by making a new channel and inviting an email ghost and just sending messages!
My other news that I forgot to mention is that I have a branch of the IRC bridge that runs off postgres, rather than flat file storage. It's much less memory/cpu intensive and might even offer a bit of a speedup :)
For ref: The current room/user store files for the freenode bridge take up 250MB when stored to disk. The existing storage system works on the principle of loading the whole datastore into memory and periodically saving to disk. Hopefully by not requiring the bridge to load all the state into memory, there are savings to be made.
mautrix-telegram v0.6.1 has been released, no changes since the release candidates two weeks ago. mautrix-telegram v0.7.0 is scheduled to be released within 6 weeks.
Also, there's a WhatsApp business API bridge using Twilio in development, though I'm guessing most people won't have any use for that.
Riot Web worked on the last remaining bits of the current privacy sprint. We are planning to do a release soon that includes this privacy work from the last few months. We are also working on a few improvements for first time users around the create room dialog. We also shipped several patch releases (1.3.5 and 1.3.6) to fix high priority bugs.
Riot-iOS has made some progress on the privacy sprint. We fixed some major issues with VoiceOver. Those fixes will be released soon in a hot fix release: 0.9.5.
A release has been done (0.5.0) with login with SSO support. There is also a new "no network" indicator, which is a bit bugged, we will fix that.
Stabilization sprint is still ongoing. We already have fixed quite a lot of issues.
François is still working on the read marker and is also fixing issues related to the permalinks.
It might be interesting to some of you that the F-Droid repo for RiotX is finally working again.
There was an issue with buildkite which in the end led to the situation that the built apk had a lower versionCode as the apps already in the repo. The repo was then "intelligent enough" to no offer it as update.
benoit was able to resolve that so the repos are updating again.
New!! A new section where we will reveal, rank, and applaud the homeservers with the lowest ping, as measured by pingbot, a maubot that you can host on your own server. Join #ping:maunium.net to experience the fun live, and to find out how to add YOUR server to the game.
In fact they're having a standoff between Matrix, Mattermost, Rocket.chat and Slack.
We're eager to see the results of this trial, the feedback will be very valuable for everyone regardless of the final outcome. (Though of course, we are optimistic..!)
This week sees us complete the last items on the privacy sprint and we expect a release to land early next week. Aside from that OpenTracing is now live, the configuration tool is ready to go (modulo the new Synapse release) and we’ve been trialling out room directory improvements.
Next week expect Synapse 1.4 to land which will contain privacy sprint support as well as a host of bug fixes and perf improvements.
Hello Twimians! After many weeks of creativity, pain, and slack, the bridge team are proud to present our first release candidate for the slack bridge. This release brings in a ton of new changes that should massively improve the slack<->matrix experience. The headline features are Postgres support, Puppeting, RTM (websocket) support as well as an entirely refactored codebase written in Typescript.
This release has made heavy changes to the schema of the slack bridge and as such we are not recommending that anyone upgrade their existing slack bridge instances to 1.0.0-rc1. A migration script is currently being worked on, but for the time being existing installs should not be upgraded to 1.0.0 release candidates.
Hi all! I haven't been around much because our homeserver chugs a little when federation is enabled, but I've got a few updates to share about Scylla, the Elm-based web client for Matrix:
There have been a few optimization changes. The naive Elm approach leads to recomputing room names and usernames on every keystroke, which causes huge performance issues in rooms with many users (like #freenode_#haskell!). The new versions of Scylla avoid recomputing room names and message HTML, leading to much better performance in large rooms and rooms where a lot of history is loaded.
The recommended fixes in the spec regarding flickering have been implemented, and messages should no longer flicker.
"m.notice" and "m.emote" events are now supported
There has been a minor visual update to make the interface look a little more consistent and professional.
We have released FluffyChat 11.16 with minor bug fixes and updated translations. A nice new feature: Messages which contains emojis only are now displayed with a much bigger font. 💕
Continuum desktop client written in pure Kotlin, version 0.9.23:
Include Elliptic Curve module to be able to connect to HTTPS servers using ECDSA
Removed dependency on kotlin-reflect and tornadofx completely and make packaged size about 5 MB smaller, which is a 10% to 20% reduction, depending on the target platform and package format
In addition to automatic scaling when HiDPI is detected, font size scaling is added. Press Ctrl and + or - to increase or decrease the font size. Font size scaling is not limited to texts, components with their size defined relative to the font size also gets scaled.
When a image message is clicked, a viewer is opened to provide a closer look. Now the updated viewer is opened in new windows, multiple images can be viewed at once, and interaction with the main window is not blocked. Images also gets scaled when the viewer window is resized.
Removed borders and backgrounds from 3 buttons and give them a flat appearance.
I've been working occasionally for the past months on a basic matrix web client called Brawl, focusing on performance, offline usage and working on my old phone. I recently got it in still limited, but somewhat useful state. Unsure how much time I'll have to keep working on this regularly, but wanted to share it here anyways. Check it out on https://github.com/bwindels/brawl-chat, there's a GIF of it in action 🙂
We’ve made good progress on our first FTUE (First Time User Experience) project: “Improve add & create room”, with only the makeover of the create room dialog remaining. Next week more FTUE projects, and we hope to get some progress on cross-signing, barring any other distractions.
Two minor releases have been made (0.9.5 and 0.9.6), to allow auto configuration of the homeserver in the login screen with a configuration link, and to fix an issue with SSO using Google account.
Had some free time and ideas this week, so I've improved my release tracking bot to now correctly handle repos mixing full releases as well as tags (lightweight and/or annotated). GitHub's API really sucks in the tags department, so this required a rework to GraphQL queries instead, and it's looking good on that so far.
The bot currently still only handles tracking all starred repos of a given account on GitHub, but I'm working on GitLab support, as well as support for tracking arbitrary repos. And a provisioning API is also somewhere on the roadmap,
This week privacy items have dominated, but we’ve also been edging towards getting our room directory improvements out, shipping a new configuration creator and making further progress on the federation side bus project to improve performance of smaller instances.
We also deployed open tracing on matrix.org, which will help a lot in tracking down all sorts of strange behaviour in the wild and fixing some bugs in the sqlite -> postgres migrator.
Next week we’ll land the room directory improvements, much of the privacy work, and have the configuration creator ready for testing.
Construct added user room listing support so that you can find out what public rooms a user is a member of should they choose to not be invisible. To find out the list of rooms for a user, simply open the room directory in Riot and type a user mxid starting with @...
Construct is implementing the concept of invisibility for users:
The exact mechanism for invisibility is not completely finished but there are obviously some limitations for this feature:
Only users who are local to the server can set any kind of invisibility. Remote users are always fully listed. One day when the spec has "user's rooms" we would be glad to adhere to an m.user.invisible-etc configuration state event in such a room
Since there is no federation support for user's rooms list publication, the rooms list for a user is limited to what the server knows (rooms it has seen the user in).
Construct is eager to implement any MSC's regarding user room and group membership listing over the federation.
Construct is written for maximum performance in C++, and is developing rapidly - join #construct:maunium.net for more info.
🔗matrix-media-repo updates for quarantining and purging media
matrix-media-repo has received a bunch of updates regarding quarantining and purging media - see the admin docs for more information. There's also an s3 upload bug fix if you're using that.
lorico has been rewriting my old GitLab bot to python/maubot: https://github.com/maubot/gitlab/pull/23. Progress on that is looking very good and it'll probably be ready within a week or two.
I decided to make a minor mautrix-telegram release with some small fixes, since v0.7.0 will take a while still. v0.6.1-rc2 is out now, the full release will probably happen next week if nobody finds any problems. Main changes:
Invalid reply fallbacks sent by Riot web in edits are now ignored
User and portal sync errors are now caught so that the sync command wouldn't fail if one telegram chat disappears.
This week we were doing the last bits of the privacy work, and fixing bugs and adding a format bar in the new composer (still behind labs flag).
We’ve also been planning the First Time User Experience work, which will start at full speed next week. This is an effort to address some issues that new users hit in the first days of using Riot. See https://github.com/vector-im/riot-web/projects/16#card-25137120 and Matthew talking over the details at https://youtu.be/JYsiHSL0lEA?t=710. Letterboxes in the left panel will likely only be removed as part of the new community project though, as we need parts of the new community UX to keep things usable for power users.
Also, if you're using https://riot.im/develop please enable the new composer in Labs, it works well and deserves testing.
Continuum desktop client written in pure Kotlin, version 0.9.20:
Updated login page appearance
Added validation to user ID and server address fields
The server address gets automatically filled with https:// followed by server name when user ID is being typed
After signing in, the token and server address are automatically saved
Added progress indicator when being signed in. It doesn't block user from controlling Continuum, pressing Esc will cancel the signing-in process.
Speed up application start-up by drawing the window and opening the database in parallel. Now the window typically appears within half a second with its size and position on screen restored to be the same as the last time it was opened. The most recently used accounts get loaded and automatically filled within a second.
Added drop shadow effect to scroll-to-latest-messages button to increase contrast
Fix: Always retry synchronization after getting a network error. Previous versions didn't retrieve new messages in a timely manner when the network is unreliable. A delay is added if the error is not a time-out to avoid stressing the homeserver.
Fix suspend/hibernate detection. Synchronization is restarted automatically after the computer resumes from sleeping. The reason is that any previously opened connection is unlikely to be still alive, opening new connections reduces delay.
Fix incorrect interface declaration that caused login to fail
Please let me know what you think of the separation of personal and public chats! Currently it's implemented using a bottom navigation bar, but tabs could also be an option (which is closer to what WhatsApp does)
t2bot.io has surpassed 300k monthly active users (about 35k daily) 🎉
roughly 250k are (bridged) from Telegram, the rest is from Discord. (there's some very small numbers which include the bots, webhooks, and other tiny bridges, but together they form less than 100 users)
Inspired by a question in the Matrix HQ channel, I've built a (rudimentary) room list viewer at https://matrix-rooms.cryto.net/ - you can just enter a homeserver (hostname) and see all the public rooms on it, without needing to log into anything. This might be useful for showing other people what kinds of rooms exist on Matrix!
OI Chat has been updated and now connects blockchain users on Blockstack and EOS (and the rest of the matrix universe): https://www.producthunt.com/posts/oi-chat-2
The sever has no passwords or requires permissions by other social account providers, it verifies the authenticity of the user using properties of the blockchain IDs, it names registered on the blockchain.
matrix-fly-paper is a new bot project intended to help automate server and room moderation. It's currently not ready for any kind of use, as I'm trying out serious TDD and attempting a functional approach to break from my OO training. Feedback is appreciated, stop by #matrix-fly-paper:lost-angles.im to talk shop
About a month ago we started selling merch at shop.matrix.org, and WOWEE ZOMG we sold a lot of stuff! If you are happy with your purchase I'd love to hear from you - share photos of where you put the stickers or you happily wearing the t-shirt in #twim:matrix.org and I'll make a little collage.
If you are not happy with your purchase then please message me at @benpa:matrix.org and I will try to make it right.
I made a few spec proposals: MSC2244: Mass redactions and MSC2246: Asynchronous media uploads. Mass redactions are useful for fighting spam (and bridging telegram deletions) more efficiently. Asynchronous media uploads would make it possible for bridges (like telegram) to handle messages synchronously and thus keep message ordering without blocking all messages behind some media upload.
Construct added support for server notice rooms. As the server administrator I've placed quick access to the server control room for myself in this section.
Construct has also added support for the server-side commands \\read <room-tag>, for example \\read m.low_priority will mark all low-priority tagged rooms as read. The command \\read * also will mark all rooms as read.
Also, we now allow users to configure whether they transmit typing events by setting state events in their home-room. If a user creates an event with the type of ircd.typing.disable.send and the state_key of the room_id, typing events will not be sent to that room for them. Further variations exist, such as ircd.typing.disable.sync to prevent the client from receiving typing events for a particular room or all rooms.
For all those who want a new emailbridge, I have great news for you! The new bridge supports sending and writing emails, using multiple rooms for multiple email-addresses. In addition you can use markup to improve the style of your emailbody, which gets translated to HTML. Here you can find the repository: Matrix-EmailBridge
The Slack bridge has had a whole week of care applied to it. We've switched the bridge entirely over to Typescript, and overhauled large parts of the codebase to make future work easier (for ex, using the Slack node library rather than request calls). If that's not exciting enough, we've also added support for the RTM API, which means the bridge can use websockets when talking to slack. This gives us nice things like typing updates, and the ability to deploy without needing a webserver to listen for events. Release 1.0 is imminent, but before then we are hoping to land Postgresql support so that the bridge can scale up to meet the high demands placed upon it.
A new version has been pushed to F-Droid, Google Play and TestFlight!
⚠️ This update requires a reinstall or clearing of data ⚠️: Because of some database
changes which were not that easy to migrate, you'll have to reinstall the app or clear the data.
Expect the new version to be on most platforms in 2 days, and on Google Play
in a few hours.
Provide builds of self-contained executables that have no external dependency. The Linux version has been done. AppImage is used, producing a binary of about 50 MB.
Adjust appearance. Remove alternating row high-lighting in list views. Remove extra padding between messages.
Thanks to Roland Pallai, Quaternion master branch is now able to use customised fonts and sports a faster timeline, better copying and a few other improvements. Settings dialog window is coming.
Initial code dump of Seshat, a Matrix event indexer/database that should enable search for encrypted rooms in riot-desktop. The dump contains a rust library as well as node bindings. The repo can be found here.
Apparently there are no ARM docker images for synapse, so I built a multi-arch synapse docker image. Supported CPU architectures include armv7, aarch64 and amd64.
There's a useless new maubot plugin that lets you run python and soon shell scripts from Matrix. Not intended as a public bot as it doesn't have any sandboxing, but it does have a "userbot" mode to run it under your own account so that you can use it in any room.
This week we’ve been working on getting our brand new room directory implementation ready, creating a new Synapse installer, and creating a TURN server testing tool. We’ve also been pushing on with the privacy project and fixing up some bugs and perf issues.
Next week it’s all about getting the room directory out of the door, shipping the installer and continuing on with the privacy project. We are in the early stages of a project designed to reduce the load caused by outbound SSL connections, it is looking very promising. We’re also going to give the sqlite->postgres porting script some love as well as working on some message retention ideas, and also ways to reduce the meta data we store.
the adding portal rooms to user-specific communities feature that appeared in mautrix-whatsapp and mautrix-hangouts last week is now also available in mautrix-facebook
The gitter bridge is back to working order again today! We've made changes so that Gitter side admins can now issue a !matrix kick|ban name command to evict users who are causing issues, rather than relying on reports or matrix side admins.
The matrix-appservice-bridge library also saw its 1.10.0 release this week, which has experimental support for bridge errors and also has a bugfix related to room upgrades.
We’ve also been continuing the work on the new message composer implementation. Features are still missing, but this has now landed behind a labs flag on /develop. We’d be especially interested in feedback whether typing with an IME (Chinese, Japanese, …) works better than before.
I've hosted a version of riot with the changes from https://github.com/matrix-org/matrix-react-sdk/pull/3251 (for math support) built in so that people can test/use it if they want. It can be found at https://www.pigeon.digital — please do let me know about any bugs you may find!
there is now a room specifically for the LaTeX fork hosted on www.pigeon.digital where you can leave any feedback, report any bugs, or ask for any help: #pigeon.digital:matrix.org
If you would like to support me, you can do so via Liberapay and Patreon!
🔗'matrixcli', new client enables Matrix as a unix util
Very interesting project: matrixcli is a versatile command line client using the matrix-python-sdk. It's not a conventional IM client, rather it exposes access to Matrix rooms so you can pipe to them as you would other Unix utils.
Neo's refactoring is almost done, with the latest changes being css-modulesifying the old stylesheet
We're also looking for nice domain name suggestions, to house the soon to be redesigned project page
The GSoC is soon coming to the end this year and I'm happy no announce that receiving of the encrypted messages PR is here! https://github.com/quotient-im/libQuotient/pull/346
A significant amount of work still needs to be done (session storage, sending messages, device handling, attachments). And I'm interested to implement it after the summer period since I've already dived into the subject :)
Here's the small demo of the receiving messages process: https://youtu.be/gGykLh4mVDg
Jojii created a new Java-based implementation of the Matrix CS API: Matrix-ClientServer-API-java, they call it "A small and simple java API for the Matrix ClientServer Protocol".
For people looking for a rather flexible ansible role just for synapse, https://gitlab.com/famedly/ansible/synapse got your back. You can use it to deploy synapse with pip or docker, depending on what you prefer for your host. It's an actively maintained fork of https://github.com/matrix-org/matrix-ansible-synapse, which has been archived recently and was unmaintained for a while before that. It's just been updated to v1.3.1 too.
This increases the number of services/protocols that the playbook can setup bridging for to 8: Telegram, Whatsapp, Facebook, Google Hangouts, IRC, Discord, Slack and Email.
Learn more in Bridging other networks.
🔗matrix-docker-ansible-deploy gets support for all Synapse config options
Just built a quick maubot plugin to add a /health endpoint to maubot, so that database failures and other brokenness can be tracked and dealt with appropriately.
privacy; IS and IM permissions and management. Hashed lookup API. Metadata cleanup (GCing redactions; deleting media; history retention); TURN management
aggregations paused
FTUE next up
Cross-signing
Immutable DMs
...then communities
RiotX
0.3 came out last week - handles DMs; lots of polishing; reduces the APK down to ~10MB!
Bridges
moving stuff into proper managed infrastructure
bifrost should be back soon?
gitter kept breaking but we're actually working with their team to fix that
working out how to automate deploying them on modular hosts
Progress for Dendrite this week has been a bit starved due to ongoing privacy work from the backend team. However it continues to move forwards with the excellent help of cnly reviewing and merging PRs.
We had 3 authors have pushed 5 commits to master and 6 commits to all branches. On master, 14 files have changed and there have been 298 additions and 121 deletions.
Also this week was some work from cnly on refactoring gomatrixserverlib and exposing some more of its internal constants to deduplicate code between itself and Dendrite.
Finally, we had some more work on an unmerged PR from cnly, which adds some internal query APIs for which servers are currently joined to a room, for use by other components later down the line.
No new Sytests are passing this week. We’re still currently at 172 tests passing out of 761 tests in total.
Synapse 1.3.0 was released - check out the blog post.
contains performance improvements to reduce disk I/O and reduce RAM usage. We’ve been running it on matrix.org for a week or so and are really pleased with the results.
mautrix-telegram got some fixes to bugs that appeared during the move to mautrix-python. It also got a manhole similar to the one in synapse, which gives admins access to an interactive Python shell inside the bridge while it's running.
The main part of the manhole (server and repl) is in mautrix-python, so you can expect the feature to appear in mautrix-facebook, mautrix-hangouts and possibly maubot in the future.
mautrix-whatsapp and mautrix-hangouts can now automatically add rooms to a user-specific community to help with filtering. mautrix-facebook is also getting that feature in the near future.
mautrix-facebook now has a search command, which means you can now actually initiate conversation with facebook users.
STUN fallback server for assisting with voice & video call negotiation (only used when your homeserver is not configured with it’s own TURN server) changed from Google to Matrix.org server and a prompt has been added to request permission before using the fallback server
Many small steps towards supporting user choice of identity servers and integration managers, including no identity server at all
Users can now download artifacts from my gitlab account since I got an SDK container from CoDerus running, cross compiling to Sailfish-RPMs in the Gitlab-Ci.
But my programming progress on master looks a bit silent because I swap to matrix-nio.
Rewrite the list view partially to handle lists with several types of items more efficiently. The message list has text messages, images, membership updates, etc. Each type needs to be rendered differently, but items of the same type could reuse GUI components when they scroll into and out of view.
Since the coming 1.3.0 release is bringing along a whole bunch of perf improvements, (and I want to test them out on my IO starved setup) I'm building K8s optimized images of the RCs. Only the debian version of the image though.
They expanded:
It's basically the regular docker image, only with some changes to make it fit nicer in Kubernetes
And there's no Helm chart at the moment, was poking at that for a bit but took a break from building one as I - back then - had trouble with the amount of data that had to be generated
You can only have one Synapse pod at any given time, as it's not scalable. Though you can scale it with workers (as I do myself)
Database-wise, I'm personally running three node HA clusters using Stolon
We run a public instance on Permaweb.io for everyone to join. We’re a large community of developers focusing on the decentralized web, with lots of channels dedicated to IPFS. In fact, we even host Riot on IPFS! We make full use of Matrix’s bridging capabilities, to Discord, IRC, Slack, Gitter and Telegram too! Come join us at #general:permaweb.io or register at https://riot.permaweb.io/ (or any other compatible client) and say hi!
Anton (@antonivan:matrix.org) is working at Matrix Towers this summer for work experience. He's been learning about Matrix and matrix-bot-sdk:
This week I made a Notification/Remind-Me-Bot using Travis's bot sdk, it took me 1 and half days, and is able to - set reminder, check reminder, cancel reminder, set interval reminder, cancel interval reminder. It is also able to store multiple reminders and cancel them using the numbers 1 - ∞ . Here is a link to my Github repo - https://github.com/joakimvonanka/Matrix-Remind-me-bot
He's also thinking of writing a guide to creating presentations:
If someone is interested on an efficient way of creating presentation using only open source tools I'm planing to write an article on that.
Small spoiler: matrix is one of them 😉
This week, I have been mostly thinking about... merch. We blew the cache on stickers, so if you are waiting for stickers specifically and have not heard from me, that is probably why. Please contact me with any questions!
Yes, it's true: Official Matrix Merch is available now. There are stickers, t-shirts and hoodies all available now, go and claim yours from https://shop.matrix.org.
I will be doing a talk on Matrix this weekend (tomorrow) at the annual Free Open Source Conference (FrOSCon) in Sankt Augustin (Germany).
If you are around - come by and get some awesome [matrix] stickers! =)
This week’s stats for Dendrite are in! 4 authors have pushed 11 commits to master and 11 commits to all branches. On master, 48 files have changed and there have been 676 additions and 146 deletions.
Very old PRs from CromFR have been updated and finally merged! These lay the groundwork for providing filters across various API endpoints (used in search, /sync, etc)
We also have a number of PRs that were moved forward but not merged. Add auth fallback endpoint (our oldest PR at the moment!) which we mentioned last week is so very close, but not quite there in time for this TWIM.
In terms of passing Sytests, Dendrite now passes 172 tests, up from 167 last week. This is out of 761 tests in total. So just a small bump (0.6%), but note that things that are crucial for Dendrite development at the moment, such as federating, only cover a small subset of tests, so each passing one can bring us much closer to getting Dendrite usable day-to-day.
The big news this week we’ve landed some massive DB improvements (https://github.com/matrix-org/synapse/pull/5706) which should make message sending on matrix.org (and any other server) feel noticeably snappier. :) It'll be in the next release.
Work continues on our installer to make it easier to configure Synapse and we’ll be looking for some community feedback rsn. The room directory revamp is now very close and we hope to have something live on matrix.org in about a week.
Finally we’re working on improving the efficiency of smaller instances sending messages into large rooms, we’re still at the design stage, but it will make a huge difference for anyone self hosting.
matrix-room-directory-server ( #matrix-room-directory:t2bot.io ) is less experimental than the key server but is still very early days. Currently it only offers the ability to manipulate the federated public room directory for your server, but in future it is planned to be its own standalone directory server (room aliases without having to run a whole homeserver). Check it out by searching the t2bot.io room directory from your client.
I love getting updates from yuforia - they've been consistently working on Continuum for some time, and by increments are making a great client.
Reuse the ListView of messages across different rooms to reduce memory usage
(Experimental) Remember and refocus the last read message, making it easier to go through all unread messages while switching chat rooms freely.
Renaming of QMatrixClient to Quotient has been finally merged to the master branch; Quaternion master uses it from now, too. Packagers are welcome to make test builds and report bugs in #quotient:matrix.org. libQuotient 0.6 beta is coming close now!
🔗mautrix-facebook, mautrix-telegram and the tulirverse
tulir has been making big updates to two of his bridges:
mautrix-facebook can now bridge formatting, mentions, replies and reactions in both directions.
There is also a logout command now.
mautrix-telegram's switch to mautrix-python is nearly finished (i.e. it didn't cause any errors for the past few days when testing in production). The main reason for the switch is using one Matrix library for all my python bridges. It also means the bridging code like double puppeting and command handling I shared between mautrix-facebook and mautrix-hangouts is now also used in mautrix-telegram.
Visible changes directly caused by using mautrix-python:
Logs are now colorful.
Python 3.5 is no longer supported.
The bridge will refuse to start without access to the base config file.
Other changes that happened during the switch:
Telegram "Saved Messages" can now be bridged even when using double puppeting.
Mentions on Telegram are marked as read when using double puppeting (messages were already being marked as read, but mentions weren't).
Also, this actually happened last week and the week before that, but anyway: I've moved the CI and docker registry of my active projects to a self-hosted GitLab at mau.dev.
Specifically, the CI/docker registry for all four mautrix bridges and maubot and automatic builds for gomuks have been moved. My maubot plugins also have automatic .mbp builds in the CI. For the docker registry, prepending dock.mau.dev/ to the existing image names will work.
The old places (docker hub, dl.maunium.net) won't get new builds anymore. The repos on GitHub are still the "canonical" repos, but they're mirrored more or less instantly with maumirror.
He adds:
mautrix-telegram will probably get some sort of history filling in the near-ish future
Also,
I'm going to add some way to put bridged rooms into personal communities for filtering purposes. Not yet sure if it'll be fully built into the bridges or some kind of an external script
The configuration process (users, rooms, mappings) is quite manual, but it's simple and has worked well for me for the past year and a half.
I've only just polished it up a bit, released and integrated with the Ansible playbook, so others could benefit from it too.
I started a "matrix-xmpp-filter" project. It's like matrix-ircd, but with xmpp instead of irc. It can also filter messages by weekday, time and regular expressions. Target audience is Sailfish phone users, but it could be used with other xmpp clients too (no xmpp MUC support required). https://k2c42.dy.fi/matrix-xmpp-filter.git/#matrix-xmpp-filter:ellipsis.fi.
Currently working on a Puppet module to manage Matrix Synapse installs (both directly and through docker), not quite ready for prime-time yet but watch this spot.
I develop matrix Bot, which converting voice messages to text. It use Yandex Speech API and Yandex API cloud (some as aws) for temporary store voice-data before converting. https://github.com/progserega/voice2textMatrix
Bot have such logic:
user add bot to room (for example room with whatsapp bridge users - now bridge support adding bot to such rooms)
bot listen room, and when get voice-message - send it to Yandex-cloud for translate (now support only Russian language).
When translating is success - bot get result text and show it in room as notice, such: "Username said: text"
Bot also receive some command, which allow disable it in this room, or disable translating for user, which send command..
Seeing this, I wondered, "y tho?", luckily progserega was able to explain with a graphic! They say a picture paints a thousand words, and it's clear from the conversation below why he'd want to have speech-to-text capabilities.
Category Atom Feed
