Saturday, 19 November 2022
vkrauseDoing HTTP operations with Qt is relatively straightforward, but there are also a few pitfalls, unexpected default settings and low-hanging performance improvements around it worth keeping in mind.
Delete replies
The most common and most easily to make mistake when working with QNetworkAccessManager is probably
missing to delete finished replies, and thus leaking reply objects over time. Make sure to call deleteLater()
on the QNetworkReply instances in response to their finished() signal.
QNetworkReply *reply = ...
connect(reply, &QNetworkReply::finished, this, [reply]()) {
reply->deleteLater();
...
});Use transport security
That is, use URLs starting with https: rather than http:. That might sound obvious, but the little s is
easy to miss.
So, pay extra attention to hardcoded URLs and think about how to deal with URLs taken from user input. In the best
case the https: scheme can just be enforced unconditionally, but that might not be viable everywhere.
Minimize QNetworkAccessManager instances
You don’t need a QNetworkAccessManager per request, in theory one is enough. In practice you might end up with one per thread
(e.g. in case of QML), but more than that should have a good justification.
There’s two reasons for this:
QNetworkAccessManagercontains logic for request queuing and network connection reuse, which is bypassed by using multiple instances, so you are missing out on useful optimizations.- More instances increase the risk of missing important setup and configuration on one of them (see below), centralizing instance creation in one location is therefore usually a good idea
This also implies that you generally want to prefer QNetworkReply signals over QNetworkAccessManager signals for handling
results or errors. This avoids interference when a QNetworkAccessManager is used by other components as well. It’s also worth
checking whether components which do HTTP requests internally can use an externally provided QNetworkAccessManager instance.
The QML engine is a common example (see QQmlNetworkAccessManagerFactory).
Redirection
Looking at properly setting up a QNetworkAccessManager instance, the most common issue is probably the redirection behavior,
something that has caused us quite some operational headaches in the past. Rather unintuitively, in Qt 5 redirection is disabled by default.
auto nam = new QNetworkAccessManager(this);
nam->setRedirectPolicy(QNetworkRequest::NoLessSafeRedirectPolicy);Starting with Qt 6, the redirection behavior NoLessSafeRedirectPolicy is the default.
HTTP Strict Transport Security (HSTS)
Another thing you probably want to enable in practically all cases is HTTP Strict Transport Security (HSTS). This involves managing persistent state, so this not just needs to be enabled, but also needs a storage location.
auto nam = new QNetworkAccessManager(this);
nam->setStrictTransportSecurityEnabled(true);
nam->enableStrictTransportSecurityStore(true, QStandardPaths::writableLocation(QStandardPaths::CacheLocation) + QLatin1String("/hsts/"));For applications QStandardPaths::CacheLocation is a good default, for shared components/libraries
QStandardPaths::GenericCacheLocation might be more appropriate so the HSTS state is shared among all users.
SSL error handling
Transport security errors are fatal by default, and that is usually what you want. One exception from this are self-signed server certificates, but thanks to Let’s Encrypt that has become increasingly uncommon as well.
If self-signed certificated need to be supported, QNetworkAccessManager unfortunately makes it very easy
to just ignore all possible SSL errors, rather than just accept the unknown server certificate signature.
KIO::SslUI has methods to help with that, including asking for user-confirmation and persisting choices.
QNetworkReply *reply = ...
connect(reply, &QNetworkReply::sslErrors, this, [reply](const QList<QSslError> &errors) {
KSslErrorUiData errorData(reply, errors);
if (KIO::SslUi::askIgnoreSslErrors(errorData)) {
reply->ignoreSslErrors();
}
});Disk cache
By default QNetworkAccessManager doesn’t do any caching, every reply comes from a full request to the server.
It’s however possible to enable the use of HTTP caching and have a persistent on disk cache for this.
Whether or not that makes sense needs to be looked at on a case-by-case basis though. Using real-time data or API (e.g. KPublicTransport) or having higher-level caching (e.g. KWeatherCore, KOSMIndoorMap) might not benefit from that, read-only assets used over a longer period of time on the other hand are ideal (e.g. avatar images in Tokodon).
auto nam = new QNetworkAccessManager(this);
auto diskCache = new QNetworkDiskCache(nam);
diskCache->setCacheDirectory(QStandardPaths::writableLocation(QStandardPaths::CacheLocation) + QLatin1String("/http/"));
nam->setCache(diskCache);The same considerations for the storage locations as for the HSTS state apply here as well.
Does this matter?
For the security-related aspects I hopefully don’t have to argue why we should care, so let’s just look at the impact of disk caches. Here are some numbers:
- Caching the conference list for Kongress cut down the transfer volume per application start by about 20%.
- Adding a disk cache to Tokodon reduced transfer volume on a “warm” start by up to 80%.
How much can be saved varies greatly depending on the specific application, but it’s clearly worth looking into.
