Create a link to a remote server and access data using CONNECT SE

In this blog we are going to see how to create a link to a remote server and use it to access multiple tables at once.
In the previous blog we have seen how to establish a remote connection between Docker containers.
The way we did it was to specify the connection string to reference a single table only.
But what if we need more tables, what if need a whole database?

The solution is to link to a remote database with the CREATE SERVER statement.
A link obtained this way can be passed to the CREATE TABLE statement of a storage engine (SE) to make a connection where using the table discovery feature SE will find out about the table fields and create the table.

MariaDB in 2021

At Vettabase we support various databases,and in particular we are MariaDB enthusiasts. Even more, we’re proud to be MariaDB Foundation Technology Partners. So I’m happy to write this small wrap-up about the MariaDB Foundation public activities in 2021! Just a disclaimer before starting: this is not an official retrospective, it’s a subjective, opinionated one. My opinions represent myself alone. So here is my favourite 2021 news in the MariaDB world…

MariaDB 10.6 went Generally Available in November (I wrote a summary on Vettabase website). A couple of unmaintained storage engines were removed, as well as 23 InnoDB variables.

10.8 preview feature: IN, OUT, INOUT, IN OUT parameters in CREATE FUNCTION

As an Oracle compatibility syntax extension, the IN, OUT, and INOUT / IN OUT quantifiers of stored function parameters was requested in MDEV-10654. It was noted that Postgres and DB2 also support this syntax. A few years later thanks to our community contributor Manohar KB of Rakuten Securities Japan, this functionality was created. Alexander Barkov from the MariaDB Corporation provided reviews and extended this PL/SQL syntax to be available in the non SQL_MODE=ORACLE mode of SQL/PSM.

Having OUT parameters on functions enables the nesting of functions will multiple return values where previously there was only the single return value possible.

MariaDB 10.8.0 preview releases now available

Following up on our announcement of the new release model, first used in 10.7.0, we are pleased to announce that the MariaDB Server 10.8.0 preview releases are now available.

Maturing MariaDB more quickly

One of the purposes of the new model is to get features into the hands of users more quickly, in a form which supports stability by allowing the new feature to be more easily tested, separate from other new features being developed at the same time.

Candidate features for MariaDB 10.8

The features available in the preview releases, and therefore under consideration for release in 10.8.1, are:

  1. MDEV-4989 mysqlbinlog GTID support
  2. MDEV-10654 Stored Procedures INOUT Parameters
  3. MDEV-11675 Lag free ALTER TABLE in replication
  4. MDEV-13756 Descending indexes
  5. MDEV-14425 InnoDB redo log improvements
  6. MDEV-17554 Auto create partition
  7. MDEV-26519 JSON Histograms
  8. MDEV-26713 Windows – Improved i18n support
  9. MDEV-27106 Spider Storage Engine Improvements
  10. MDEV-27265 Misc.

Tricky Problems? MariaDB debug container

MariaDB does have bugs. Users see them sometimes. Sometimes developers look for a long time at bug reports and code and still cannot see how the situation occurred. Developers during their analysis ask questions like:

  • I wonder if this was already fixed in {not released version}? But how can I ask a user to test that?
  • Can I get the user to get good stack trace that would help understand this better? But users sometimes find this hard.
  • What exact hardware and kernel configuration is this bug occurring on? And how would I reproduce this?

New Service – quay.io/mariadb-foundation/mariadb-devel

During the development of MariaDB, a lot of things are tested. However the most important workload to be tested is the one we don’t have access to, and that is your workload.

As many of you run your own CI, we’d like you to invite you to join the testing of MariaDB. quay.io/mariadb-foundation/mariadb-devel is a container repository using the latest from our main stable branches. By the time any code gets into these branches it has been reviewed and passed our tests. The developers of the change consider it finished, so this is the perfect time to take this code and test it on your workload.

MariaDB Announces New Innovation Release Model

Quarterly release structure delivers new features to the community faster

REDWOOD CITY, Calif. and HELSINKI – December 9, 2021 – MariaDB Corporation Ab and MariaDB Foundation today announced a new release model for MariaDB Server, by MariaDB Corporation referred to as MariaDB Community Server, centered on the goal of delivering new features and innovation faster to the millions of MariaDB users around the world. Under the new model, new server release series, such as MariaDB 10.7 or 10.8, occur once a quarter, rather than once a year, bringing more opportunities for new features and contributions to be delivered to the MariaDB user base.

Using CONNECT SE to access remote JSON data with demo examples in Docker

CONNECT is a storage engine (SE) plugin used to access external, local or remote data. In this blog we are going to show how to install the CONNECT storage engine in a Docker container and how to share JSON data between containers.

Enabling CONNECT SE plugin in Docker

CONNECT SE needs to be installed within the container in order to use it. To see how to do that please check Installing plugins in the MariaDB Docker Library Container.

Create JSON data on remote server

The most important feature of CONNECT SE to MariaDB is the flexibility to create tables from various data sources, like the same database and other DMBS’s tables or files with different formats.