Queries in PostgreSQL. Sequential scan

In previous articles we discussed how the system plans a query execution and how it collects statistics to select the best plan. The following articles, starting with this one, will focus on what a plan actually is, what it consists of, and how it is executed.

In this article, I will demonstrate how the planner calculates execution costs. I will also discuss access methods and how they affect these costs, and use the sequential scan method as an illustration. Lastly, I will talk about parallel execution in PostgreSQL, how it works, and when to use it.

I will use several seemingly complicated math formulas later in the article. You don't have to memorize any of them to get to the bottom of how the planner works; they are merely there to show where I get my numbers from.

• The method of phase-magnitude interpolation (PMI)

• Accurate measure of frequency, magnitude and phase of signal harmonics

• Detection of resonances

The Fast Fourier Transform (FFT) algorithm is an important tool for analyzing and processing signals of various nature.

It allows to reconstruct magnitude and phase spectrum of a signal into the frequency domain by magnitude sample into the time domain, while the method is computationally optimized with modest memory consumption.

Although there is not losing of any information about the signal during the conversion process (calculations are reversible up to rounding), the algorithm has some peculiarities, which hinder high-precision analysis and fine processing of results further.

The article presents an effective way to overcome such "inconvenient" features of the algorithm.

Read in Russian

In Rust strings can be represented in two ways:

a)       String type

b)      String slice

String type:

String type is defined as a struct of the following structure:

Depending on arch (in my case x86 64bit it is 24byte)

{

   pointer to the address where string characters are stored (8b)

   capacity (8b)

   length (8b)

}

In this article I have described problems I have faced during integration Apache Camel with Spring Boot Actuator Prometheus for collecting metrics and my solution to solve this problem (which I haven't found over the internet).

In this article I'd like to describe different traffic mirroring options for HPE Virtual Connect modules. I try to answer two questions: "What options work and why?" and "How different traffic mirroring cases can be implemented?"

This article describes how to develop a real-time weather prediction app in Flutter using the Tomorrow.io Weather API. The app consists of a single screen that displays the current weather and a 4-day daily forecast for a specific location. We’re creating this prediction app on Flutter because it is an open-source UI development kit, which means it can be integrated beautifully into desktop and mobile apps across various platforms, making it scalable. 

Once a upon a time I moved to brand new project. And without much thought, I decided to take DevOps responsibilities (after a long period of Frontend). Huge mono-repository (Angular and Node.js) gives rise to many specific problems. And this project was no exception. At the very beginning CI/CD duration was about 1.5h. And that was the biggest problem to take care of.

But at first, I want to talk about "Bourne again shell", cause CI/CD automation almost entirely was implemented by means of shell scripts (Bash). In context of huge mono-repo even regular build becomes a piece of odd stuff. That`s why a huge amount of scripts with complicated logic inside (build, test, deploy, generation of release notes, collection of logs and metrics, ...) was another significant problem.

Regardless of lang, quality of complex logic should be under control. It is no secret that key aspect of code quality control - is tests. For example, to refactor safely major logic should be covered with tests. We decided to keep shell scripts untouched and cover the major ones with tests before any refactoring. Of course, it is possible to use Jest (or Mocha) with a bunch of awful utils to test shell scripts. This approach is a bit wordy and has no value if scripts under test is written in Bash. Also I have no idea how to mock external shell commands (such as curl, ls, touch, npm, ...) with Jest or similar framework.

After Epic released the UE5 technology demo at the beginning of 2021, the discussion about UE5 has never stopped. Related technical discussions mainly centered on two new features: global illumination technology Lumen and extremely high model detail technology Nanite. There have been some articles [1 ][2] analyzing Nanite technology in more detail. This article mainly starts from the RenderDoc analysis and source code of UE5, combined with some existing technical data, aims to provide an intuitive and overview understanding of Nanite, and clarify its algorithm principles and design ideas, without involving too many source code level Implementation details.

Overview

Sometimes all of us need to make a graphical installer for one's own linux distro. It goes without saying that you are able to use a distro-specific installer like Anaconda for RedHat-based or DebianInstaller for debian-based. On the other hand Calamares is a graphical installer which is not aligned with only one package manager.

I want to share my experience how to make a universal install solution with GUI. I did not find any complete article about it, hence, I reinvented the wheel.

Lumen is UE5’s GI system, it is different from the traditional real-time GI which only includes the contribution of indirect diffuse reflection. It also includes indirect diffuse reflection and indirect highlight, providing a new set of complete indirect lighting. Lumen supports both hardware-based RTX and software-based Trace algorithms. The starting point of this article is that Lumen GI uses the process, algorithm, and data structure analysis of indirect diffuse reflection part based on software Trace to understand the basic principle and operation mechanism of Lumen from a macro perspective.

The core of Lumen includes the following parts:

Somehow I saw code in the project of a neighboring team that generated a string with URL parameters for subsequent insertion into the iframesrc attribute.

This article may seem superfluous, obvious or too simple, but since this occurs in wildlife, you should not be silent about it, but rather share best-practices.

Today, we will share some knowledge points related to resource memory leak. A memory leak is the most common issue that we continuously see and also are afraid of. What is the reason behind it? Because we can’t predict the extent of the leak before we locate the leak bottleneck, we had no idea whether it will burst out at a certain moment on the line. We have received feedback from developers that their players had no problem playing for half an hour, but they would get more and more stuck after 3 to 4 hours of playing, which they never expected before. How can it be solved? Today’s sharing will answer such questions.

UWA’s GOT Online-Assets report has a resource occupancy trend chart. If there is a rising trend like the one below, you must pay special attention.

The Real-Time Clock (RTC) Calendar Registers in STM32 microcontrollers implemented in Binary Code Decimal format (BCD) i. e., every two digits are represented by one byte (low digit in 0-3 bits and high digit 4-7 bits). At least there are 5 bytes required to store date and time data in a such format. There are cases when memory allocation for time stamp might become critical, e. g., the events log keeping in an extern non-volatile memory IC. Here the memory value for a single event (event serial number, timestamp, event parameters, and its CRC) is fixed and can compose up to 16 bytes. The maximum quantity of the events increasing, and time spent on a single operation reducing (for reserved power sources as supercapacitors and electrolytic capacitors is critical) can be achieved with less memory amount required for the event field.

The spread of cryptocurrencies contributed to the development of many solutions based on a distributed ledger technology (blockchain). Although the scope of these solutions varies, most of them have the same basic set of security services, i.e. confidentiality, authenticity, and integrity. These features are granted by the practical applications of public key cryptography, in particular,  digital signature (DS). But unlike many other applications of public key cryptography, cryptocurrency networks do not use public key certificates. This is the aspect that we would like to talk about.

In the last article we reviewed the stages of query execution. Before we move on to plan node operations (data access and join methods), let's discuss the bread and butter of the cost optimizer: statistics.

Dive in to learn what types of statistics PostgreSQL collects when planning queries, and how they improve query cost assessment and execution times.

This is the first in series of articles dedicated to detailed overview of processes and components of enterprise identity and access management (IAM). These articles are designed to help technical architects and system designers to better understand what is involved in complex business processes which drive identity management and provide detailed review of all critical components.

Solid understanding of identity management business processes before implementation of a new IAM system very often makes a difference between success and failure and it must be worked out and agreed to by all key participant before implementation can start.

This review does not address specific software configurations and settings required to support these processes and workflow, but any modern identity management software will be able to support them.

Diagram below documents workflow for an employee on-boarding which is one of the core identity management processes to support new employee who is joining the company. The flow is designed to be integrated into existing hiring process and provide logical and physical access to an employee by day one of the employment.

If you regularly use a static code analyzer, you can save time on guessing why the new code doesn't work as planned. Let's look at another interesting error — the function broke during refactoring, and no one noticed that. No one — except for PVS-Studio that can automatically scan the project and email the report to us.

In the green energy field, the solar panels composed of bendable solar cells tend to significantly increase in popularity last time. The difficulty during the designing of the different devices (such as solar chargers, MPPT-controllers) emerges because of the strong lack of technical information about this kind of solar cell.  

In this article, the equivalent scheme of the bendable solar produced by the Sunpower company - Maxeon gen 5 and its coarse simulation model is proposed.

You decided to make an app that works with SVG. Encouraged by the enthusiasm, you collected libraries and successfully made the application. But suddenly you find that the app is sending strange network requests. And data is leaking from the host-machine. How so?

Let's continue with a series of small notes illustrating the PVS-Studio's ability to quickly find new errors in the code. If the analyzer is regularly used, of course :). Today we have another bug in the Blender project.