PopSci

I've long wanted to create educational materials on the topic of Information Theory + Machine Learning. I found some old drafts and decided to polish them up here, on Habr.

Information Theory and Machine Learning seem to me like an interesting pair of fields, the deep connection between which is often unknown to ML engineers, and whose synergy has not yet been fully revealed.

Let's start with basic concepts like Entropy, Information in a message, Mutual Information, and channel capacity. Next, there will be materials on the similarity between tasks of maximizing Mutual Information and minimizing Loss in regression problems. Then there will be a section on Information Geometry: Fisher metric, geodesics, gradient methods, and their connection to Gaussian processes (moving along the gradient using SGD is moving along the geodesic with noise).

It's also necessary to touch upon AIC, Information Bottleneck, and discuss how information flows in neural networks – Mutual Information between layers (Information Theory of Deep Learning, Naftali Tishby), and much more. It's not certain that I'll be able to cover everything listed, but I'll try to get started.

Ссылка на русскую версию / link to Russian version

FIFO is a key concept in hardware design. Understanding of FIFO is necessary for understanding the valid/ready protocol, which in turn is necessary for organisation of flow-control within a design.

Unfortunately, there are very few books on this topic, and to be fair, microarchitectural concepts are quite difficult to master from books, since understanding of these concepts are coming with practice. In other words it is more about developing hardware intuition.

The idea of the HDL training tool is that it can help develop a hardware intuition, providing the opportunity to explore ready-made scenarios in a step-by-step interactive way. The tool also provides detailed visualization of a simulated scenario.

Since the tool is a front-end for the HDL simulator, the real, synthesized SystemVerilog is executed on the simulator itself, which can be viewed and even modified.

So, the video of exploring FIFO on the training tool is here:

Brainstorming is a popular working method which is commonly used by UX design teams. It involves a group of designers meeting (whether offline or via video call) and generating as many ideas as possible to find the best solution to a specific problem or come up with creative design ideas. Brainstorming sessions are usually held at the start of a UX project so that designers could use the ideas they think are the best later in the process of product creation. These sessions can vary in duration and form depending on which problems need to be solved, how many people participate and how many ideas need to be generated. 

 Python, Java, C++, Delphi, PHP—these programming languages were used create a virtual crypto ATM machine to be tested by the participants of the $NATCH contest at Positive Hack Days 12. The entire code was written by ChatGPT and proved to be exceptionally good. This time, we had reviewed the contest concept and decided to use a report system. In addition to standard tasks (kiosk bypass, privilege escalation, and AppLocker bypass), this year's participants faced new unusual tasks. Read on below to find out which ones.

The general trend of technology development is characterized by surges and declines. Consider, for instance, the mass movement of human bodies. Initially, horses and wagons were used, which gradually evolved into a distinct industry. Then trains appeared abruptly. Horses were quickly forgotten, and the focus shifted to a new avenue. Steam became an object of study and evolved into a complex science. Diesel and electricity developed concurrently. At a certain point, steam engines became obsolete, and everyone transitioned to diesel and electricity. Similarly, we are now transitioning to electric cars that require significantly fewer fluids.

Technologies evolve and function until new technologies completely replace them. I believe we are entering an era where framework and Electron technologies may be eclipsed by generative AI. Let's examine some examples.

Currently I am investigating firmware development for STM32 microcontrollers and I would like to share with you my experience for doing it in Qt Creator IDE.

There are a lot of IDEs, which are used for firmware development of STM32. Some of them, being quite comfortable, have restrictions for trial license. For example, the one of the most known IDE, IAR Embedded, suggests either a limited amount of product usage time (30 days) or the limited firmware size of 32 MB, which is not too much.

Within this scope of the publication, we investigate the method of setting up an environment that allows one to develop the full value of the STM32 firmware in Qt Creator.

ChatGPT was not used in writing this article. The animated image uses the webp file format instead of gif.

In this article, you will find a complete description of how to make a Wi-Fi internet radio receiver from a router that can play mp3 streams from internet radio stations. It is also possible to switch between two internet radio stations. We will use OpenWRT firmware installed on the router to create a Wi-Fi internet radio. It is possible to complete this project without using a soldering iron. All the components can be placed inside the router to create a finished device — a Wi-Fi internet radio.

• Any router with a USB port that can be installed with OpenWRT, a Linux-based operating system designed for embedded systems. You can check if your router is compatible with OpenWRT on this page: https://openwrt.org/toh/start.
• A USB sound card that costs $1-$2.
• Any passive (or active) speakers.
• Two patch cords — twisted pair crimped with 8P8C connectors (also known as RJ-45).
• A personal computer.
• Ethernet and Wi-Fi internet connection (optional).

Referential transparency, a key concept in functional programming, is often associated with more reliable, easier to test, and safer software. This term refers to a principle in which a function, given the same input, will always produce the same output without producing any side effects.

In the real world of software development, side effects are inevitable. Programs are rarely useful unless they interact with the outside world. This interaction could be reading from or writing to the console, making network requests, querying a database, or modifying a variable.

Yet, despite the necessity of side effects, they introduce risks and complexities. Programs with side effects are harder to test, harder to reason about, and more prone to bugs. They can also make the system as a whole more difficult to understand and maintain, due to hidden dependencies between components.

Enter referential transparency - a concept that means a function, given the same input, will always provide the same output, without creating any side effects. A function that adheres to this principle doesn't read any global state or change any state outside of its scope. The result is code that is more predictable and easier to reason about.

In terms of software safety and reliability, the absence of side effects is not enough. Programs should also be free from external influence - their results should only depend on their arguments. That is, programs should not read data from the console, a file, network, database, or even system variables.

In the rapidly evolving landscape of digital entertainment, the year 2023 has witnessed a profound transformation in Video On Demand (VOD) Over-The-Top (OTT) platforms. These platforms have redefined how audiences access and consume content, introducing cutting-edge technologies and innovative features that have revolutionized the entertainment industry.

Embracing Augmented Reality (AR) and Virtual Reality (VR)

In 2023, VOD OTT platforms have embraced AR and VR technologies to provide viewers with immersive and interactive experiences. Users can now step into virtual worlds, interact with characters, and actively participate in the content they are watching, transcending the boundaries of traditional passive viewing.

Imagine exploring an ancient civilization through the eyes of a protagonist, feeling the thrill of an adrenaline-pumping car chase, or even interacting with virtual avatars of celebrities and influencers. The integration of AR and VR into VOD OTT platforms has ushered in a new era of interactive and experiential entertainment.

Personalization: The Heart of the VOD Experience

Powered by advanced artificial intelligence (AI), VOD OTT platforms have taken personalization to new heights. By analyzing user behavior, preferences, and viewing habits, these platforms deliver tailor-made content recommendations, ensuring that viewers find exactly what they love without extensive searching.

This level of personalization ensures that users are presented with content that resonates with their tastes, creating a seamless and enjoyable content discovery experience. By removing the need for extensive searching, VOD OTT platforms have become the go-to destination for viewers seeking highly relevant and engaging content.

Let’s imagine you need access to the real-time data of some smart contracts on Ethereum (or Polygon, BSC, etc.) like Uniswap or even PEPE coin to analyze its data using the standard data scientist/analyst tools: Python, Pandas, Matplotlib, etc. In this tutorial, I’ll show you more sophisticated data access tools that are more like a surgical scalpel (The Graph subgraphs) than a well-known Swiss knife (RPC node access) or hammer (ready-to-use APIs). I hope my metaphors don’t scare you 😅.

Against the backdrop of a sharp decline in venture investments in various traditional industries (this is clearly related to the global economic situation), but against the backdrop of certain upheavals in the crypto industry, the question arises of a driver that can become one of the «pillars» of the development of a decentralized industry.

Today, it becomes quite obvious that the situation with web3 technology is becoming similar to the situation with programmatic advertising 10-12 years ago. When programmatic-ads just started, the major market players were not very optimistic - it was much easier for them to follow the well-known paths with chains of publishers and agencies. However, once programmatic got rid of the «childhood diseases» and became simple in use, effective in attracting customers, it has taken a dominant position in the modern advertising market - providing precise targeting and cost reduction. The situation with web3 is similar - all publishers ideally want to have systems for lifetime tracking of customer achievements (that is, Soul Bound tokens in web3), theoretically stable decentralized data storage systems that cannot be broken through a simple server attack, real users’ ownership for his content, as well as simple cross-border payment systems that have built-in algorithms for calculating all duties and taxes. Sounds like a perfect world, but even Amazon doesn't do that yet. Why?

The problem is the web3 entry point, which is difficult for users. As soon as they start talking about opening a crypto wallet, about 12 phrases as a backup, most users interrupt the onboarding process and immediately leave the site. This is obvious - over the decades of the development of the digital market, users have become accustomed to facilitating and simplifying systems for interacting with the site. One-click registration, effective convenient UI, visibility and the «metro principle», where the client is led by the hand – this is modern digital progress. And here some 12 phrases are offered, incomprehensible tokens - and, most importantly, why? So, it turns out that a self-sufficient audience of geeks or really advanced users revolves around web3, which is growing weakly.

I develop robots, and I'm often asked, "How to make a robot?" and "Where do you find information and what resources do you use?"

If you don't know where to start and want to create your own robot, this article is for you. In it, I will try to explain the process and also share the first steps you should take.

Generative AI is creating waves in the way we work, significantly revolutionizing the software development process. AI tools are appearing in various phases of software development, such as design, development, and testing. However, there aren't many tools specifically focused on software business analysis tasks.

But with a little creative thinking, we can put "one-size-fits-all" applications like ChatGPT to good use. It can definitely speed up execution of many typical tasks and free up analysts to focus on the more challenging, strategic aspects of the job.

I believe that every programmer has at least once heard about ChatGPT and its marvelous abilities to process, calculate and create huge amounts of data; if not, go check out this Wikipedia article - https://en.wikipedia.org/wiki/ChatGPT.

Can you imagine that some 50 years ago people could not even believe that there may be something artificial surpassing humans in so many areas? Nowadays, we have this marvel at the distance of a few tabs on a phone screen or a keyboard; however, there is still a sadly large number of people who do not fully—if at all— utilize all the perks of ChatGPT in their lines of work. This is mostly related either to people's reluctance to learn new technologies or the fear of losing coding skills they have previously gained—which is not the case with using ChatGPT properly.

In this article I want to give you some of the most useful uses of ChatGPT for your coding work. Remember, there is nothing shameful in using the AI, since this the development and further implementation of it in our day-to-day life is inevitable, so we should start adapting to it as early as we can to take the full advantage of this "magical" technology. Let's get started.

Hi, my name is Mikhail Emelyanov, I’m a Python programmer and I would like to show you my pet project — Flywheel, a micro-platform for learning foreign languages, a mixture of Duolingo and Anki, an application that can teach you to properly write in Spanish (or any other language you’re studying). Flywheel’s source code is available on GitHub.

As you may know, generalized knowledge of a foreign language can be broken down into four relatively independent components: reading, writing, listening, and speaking. Unfortunately, training one of these abilities has no direct effect on the other components, so, for example, by developing our reading skills, the effect on our writing skills is quite indirect. Flywheel is a ‘sharpener’ specifically for written Spanish.

If you’ve ever used Duolingo, you should have some idea of the format in which you’ll be studying. The formula is simple: here’s a phrase, translate it into the other language; the app will remember the last time you translated a phrase and how successful you were at it; and depending on the accuracy of your answer, it will determine when you should do the same phrase again. In my opinion, Duolingo and its approach are brilliant. However… There are certain aspects that somewhat spoil the learning experience, and Flywheel was specifically designed to address them.

Recently, The BCI, one of the leading institutes working in the field of organizational resilience and business continuity, issued its regular report BCI Operational Resilience Report 2023 in collaboration with Riskonnect, who work with risk management solutions.

One of the questions they asked the respondents was if there was a difference between organizational resilience and operational resilience. As the answers demonstrated, for most respondents (and in most companies) these terms were used as synonyms. Having studied the report, the colleagues brought up another matter – The BCI introduced the new term of "organizational resilience" in addition to "business continuity" and "operational resilience".

If we search Habr for "Business Continuity", "DRP", "BCP", or "BIA", we’ll find quite enough posts by my colleagues (I’ve met some of them face to face and worked with the others) about data system recovery, data system testing, fault-tolerant infrastructure, and some other things. Yet, hardly any of them explain where all of it has come from, how it is changing, where it is heading – and why.

I thought the time has come to change the situation for the better and answer some of the questions like where business continuity provisions and operational resilience has come from, how they are changing, and where this trend is heading and why. To share my thoughts about development of the industry and its current de-facto state in case of a mature (or not too mature) introduction level – some things I’ve stated for my own use.

3.What's an ugly smiling face?

It's the cat from curved space.

V. Komen, I. Tikhonenkov

Here we are discussing in general the majority of metrics for a stationary gravitation fields in one dimension. The only accepted approach so far to apply the equations of field (A. Einstein):

Today I would like to discuss the games Chess and Go, the world's champions, algorithms and Al.

In 1997, a computer program developed by IBM Deep Blue defeated the world Chess champion Garry Kasparov. Go remained the last board game in which humans were still better than machines.

Why is that?

Chess is primarily distinguished from Go by the number of variations for each move. Chess, the game is more predictable with more structured rules: we have value for each figure (e.g bishop = 3 pawns, rook = 5 pawns -> rook > bishop), some kind of openings and strategies. Go, in turn, has incredibly simple rules, which creates the complexity of the game for the machine. Go is one of the oldest board games. Until recently, it was assumed that a machine was not capable of playing on an equal footing with a professional player due to the high level of abstraction and the inability to sort through all possible scenarios - exactly as many valid combinations in a game on a standard 19×19 go-ban are 10¹⁸⁰ (greater than the number of atoms in the visible universe).

However, almost 20 years later, in 2015, there was a breakthrough. Google's Deep Mind company enhanced AlphaGo, which was the last step for the computer to defeat the world champions in board games. The AlphaGo program defeated the European champion and then, in March 2016 demonstrated a high level of play by defeating Lee Sedol, one of the strongest go players in the world, with a score of 4:1 in favour of the machine. A year later, Google introduced to the world a new version of AlphaGo - AlphaGoZero.

V. Komen, I. Tikhonenkov

In the previous post we've considered a model example of a motion of a free particle within a uniform gravitation field where a coupling to the field is defined by an observed inertion mass (see eq. (2) in https://habr.com/en/articles/739714/). The equation of motion was: