High performance *

Hello!

This post is about writing a fast enumerator in C#:

foreach (var i in 1..10)

We will figure out whether it's possible to make it as fast as for, and what can we do to improve our performance.

A lot of benchmarks and sharplab coming. Let's rock!

• We’ll find out what is an in-memory database, or IMDB. When and how are they better than disk solutions?
• Then, we’ll consider their architecture. What about their efficiency, reliability, and scaling?
• Then, we’ll delve into technical details. Data types, iterators, indexes, transactions, programming languages, replication, and connectors.

If you use Zabbix to monitor your infrastructure objects but have not previously thought about collecting and storing logs from these objects then this article is for you.

RAM disk, this is a disk based on RAM memory chips. This kind of disk is not able to retain data after the power is turned off (unless a supporting battery is used), but has an exceptionally high read/write speed (especially for random access) and an unlimited lifespan. It is important in tasks that need a lot of cycles to write over information, even professional SSD drives don’t live long. To the operating system the RAM disk is indistinguishable from an SSD or HDD disk and no special drivers or setup is required. Unlike a disk that is virtually located in the computer’s RAM memory, where the maximum memory capacity is limited to 128-256 GB in the best consumer motherboards, a RAM disk for a PCIe slot, in general, has no volume limits and can work in any MB with a PCIe slot.

On implementing streaming algorithms, counting of events often occurs, where an event means something like a packet arrival or a connection establishment. Since the number of events is large, the available memory can become a bottleneck: an ordinary -bit counter allows to take into account no more than events.
One way to handle a larger range of values using the same amount of memory would be approximate counting. This article provides an overview of the well-known Morris algorithm and some generalizations of it.

Another way to reduce the number of bits required for counting mass events is to use decay. We discuss such an approach here [3], and we are going to publish another blog post on this particular topic shortly.

In the beginning of this article, we analyse one straightforward probabilistic calculation algorithm and highlight its shortcomings (Section 2). Then (Section 3), we describe the algorithm proposed by Robert Morris in 1978 and indicate its most essential properties and advantages. For most non-trivial formulas and statements, the text contains our proofs, the demanding reader can find them in the inserts. In the following three sections, we outline valuable extensions of the classic algorithm: you can learn what Morris's counters and exponential decay have in common, how to improve the accuracy by sacrificing the maximum value, and how to handle weighted events efficiently.

One thing about WebSockets is that you need a lot of resources on the client's side to generate high enough load for the server to actually eat up all the CPU resources.

There are several challenges you have to overcome because the WebSockets protocol is more CPU demanding on the client's side than on the server's side. At the same time you need a lot of RAM to store information about open connections if you have millions of them.

I've been lucky enough to get a couple of new servers for a limited period of time at my disposal for the hardware "burnout" tests. So I decided to use my Lua Application Server — LAppS to do both jobs: test the hardware and perform the LAppS high load tests.

1. Bad release: database overload