With the rise of lazy.nvim, Neovim plugin management has shifted toward a more declarative, performance-focused approach. Plugins load only when they’re needed, configurations become easier to reason about, and startup time stays fast even as your setup becomes more powerful. When combined with nvim-lspconfig, this approach results in an LSP configuration that is both clean and flexible.

In this post, we’ll walk through a modern Neovim LSP setup using nvim-lspconfig and lazy.nvim. You’ll see how to structure your configuration, load LSP support at the right moment, and avoid common pitfalls—ending up with a maintainable setup that scales as your editor and workflow evolve.

As we saw in the previous post, we are working with the following folder structure:

This folder structure reflects a clean and modular approach to configuring Neovim, using lazy.nvim as the plugin manager and nvim-lspconfig for LSP support. The init.lua file is Neovim’s main entry point and typically only contains minimal bootstrap logic, delegating most of the setup to Lua modules. The lazy-lock.json file is automatically generated by lazy.nvim and ensures reproducible plugin versions by locking dependencies to specific commits. Inside the lua/ directory, configuration is split by responsibility: config/lazy.lua is dedicated to initializing and configuring lazy.nvim itself (including plugin loading options), while the plugins/ directory contains individual plugin specifications. In this case, plugins/lsp.lua encapsulates all LSP-related configuration, such as server setup.

Let’s start with the configuration for the LSP client in the core config nvim/lua/config/lazy.lua file. Depending on the project size and its dependencies, the LSP server can take a noticeable amount of time. In these situations, having visual feedback during initialization is especially helpful, as it lets you know that the LSP is working and gives you an idea of how long it will take to become ready. Neovim’s built-in LSP client supports this by exposing a function that can be integrated into your statusline, allowing you to see the current LSP progress and initialization status at a glance.

----------------------------------------------------------------------
-- LSP progress
----------------------------------------------------------------------
vim.api.nvim_create_autocmd("LspProgress", {
  group = vim.api.nvim_create_augroup("LspProgressStatusline", { clear = true }),
  callback = function()
    vim.cmd("redrawstatus")
  end,
})

vim.opt.statusline:append("%{v:lua.vim.lsp.status()}")

We can now test our configuration by closing Neovim and opening it again with the file lazy.lua and check how the LSP initialization status gets reported on the status line. Once the loading process finishes, the LSP is ready to show diagnostics, if any.

One of the key capabilities of an LSP client is its support for real-time diagnostics, such as warnings, errors, and hints displayed directly in the editor. This effectively acts as a first line of defense when working with compiled languages, surfacing issues as you type rather than waiting for a full build to fail. By highlighting problems inline and providing precise messages, the LSP offers immediate feedback on mistakes, questionable patterns, or potential bugs, allowing you to correct them early and maintain a faster, more confident development flow.

----------------------------------------------------------------------
-- Diagnostics (INLINE)
----------------------------------------------------------------------
vim.diagnostic.config({
  virtual_text = {
    spacing = 4,
    prefix = "●", -- could be "■", "▎", ""
  },
  signs = true,
  underline = true,
  update_in_insert = false,
  severity_sort = true,
})

This configuration customizes how Neovim displays LSP diagnostics, focusing on clear and non-intrusive inline feedback. The virtual_text option enables inline diagnostic messages and fine-tunes their appearance: spacing = 4 adds padding between the code and the message for better readability, while the prefix = "●" inserts a small visual marker to quickly draw attention to problematic lines without overwhelming the code. Enabling signs places icons in the sign column, providing an at-a-glance indication of errors or warnings, and underline = true visually emphasizes the exact range of code associated with the issue. Setting update_in_insert = false prevents diagnostics from updating while you are typing, reducing visual noise and distractions during insertion. Finally, severity_sort = true ensures that when multiple diagnostics exist on the same line, more severe issues (such as errors) take priority over warnings or hints, helping you focus on what matters most first.

Now that the basic LSP setup is in place, we can move on to configuring a language server for a specific language and see the LSP client in action. We’ll do this in the nvim/lua/plugins/lsp.lua file, which contains the specification for the nvim-lspconfig plugin. Each plugin spec is made up of several configurable parts; here, we’ll focus on defining when the plugin should be loaded and setting its core configuration options. We are going to configure the Lua language server. That way, it can help you out from now on to spot any issues on the configuration files for other plugins. And you can explore more easily the APIs provided by Neovim to implement extra functionality.

return {
  "neovim/nvim-lspconfig",
  event = { "BufReadPre", "BufNewFile" },
  config = function()

    ----------------------------------------------------------------------
    -- Lua LSP configuration
    ----------------------------------------------------------------------
    vim.lsp.config("lua_ls", {
      settings = {
        Lua = {
          runtime = { version = "LuaJIT" },
          diagnostics = {
            globals = { "vim" },
          },
          workspace = {
            checkThirdParty = false,
            library = vim.api.nvim_get_runtime_file("", true),
          },
          telemetry = { enable = false },
        },
      },
    })

    -- Enable Lua LSP
    vim.lsp.enable("lua_ls")

This plugin specification defines how nvim-lspconfig is loaded and how the Lua language server is configured in a lazy.nvim setup. The plugin is lazy-loaded on the BufReadPre and BufNewFile events, meaning it is only initialized when you open an existing file or create a new one, which helps keep Neovim’s startup time fast. Inside the config function, the Lua language server (lua_ls) is configured using vim.lsp.config, with settings tailored specifically for Neovim development. The runtime is set to LuaJIT, matching the Lua version embedded in Neovim, and the diagnostics are instructed to recognize vim as a global variable to avoid false warnings. The workspace configuration disables third-party checks and adds Neovim’s runtime files to the server’s library, enabling better completion and navigation for Neovim APIs. Telemetry is explicitly disabled to avoid sending usage data. Finally, vim.lsp.enable("lua_ls") activates the server, ensuring the configuration is applied and the Lua LSP starts automatically when editing Lua files. You can test its functionality with the default keybinding ‘K’ that shows the hover information of a symbol.

To ease navigation over autocompletion options, we will define a couple of keybindings. These keybindings enhance the LSP auto-completion workflow by making <Tab> and <S-Tab> context-aware in insert mode. When the completion pop-up menu is visible (pumvisible()), <Tab> and <S-Tab> are repurposed to navigate forward and backward through the list of completion candidates using <C-n> and <C-p>, allowing you to quickly browse suggestions without leaving the home row. When no completion menu is shown, pressing <Tab> explicitly triggers omni-completion (<C-x><C-o>), which asks the active LSP server for context-aware suggestions such as symbols, methods, or types. This dual behavior keeps completion both discoverable and fluid: you can trigger LSP completion on demand and seamlessly navigate results with familiar keys, reducing friction and making auto-completion feel like a natural extension of typing rather than a separate action.

-- Omni complete
vim.keymap.set("i", "<Tab>", function()
  if vim.fn.pumvisible() == 1 then
    return "<C-n>"
  end
  return "<C-x><C-o>"
end, { expr = true })

vim.keymap.set("i", "<S-Tab>", function()
  if vim.fn.pumvisible() == 1 then
    return "<C-p>"
  end
  return "<S-Tab>"
end, { expr = true })

Wrapping things up, configuring nvim-lspconfig with lazy.nvim gives you a clean, modular, and performant way to manage LSP support in Neovim. By letting lazy.nvim handle loading and dependencies, you keep your configuration declarative and easy to reason about, while nvim-lspconfig focuses on what it does best: connecting language servers to your editor. This setup scales naturally as you add more servers, tools, or customizations, and it encourages a workflow where your editor grows with your needs instead of fighting them. With this foundation in place, you’re well-equipped to fine-tune diagnostics, keymaps, and capabilities—and make Neovim feel truly tailored to your development style.

Enter lazy.nvim. Designed from the ground up with modern Neovim workflows in mind, lazy.nvim goes beyond simply installing plugins. It introduces automatic lazy loading by default, event-based loading, and a highly optimized startup path, which often results in faster launch times with less manual configuration. While packer required you to explicitly define when and how plugins should be loaded, lazy.nvim encourages a declarative style that scales better as your configuration grows. In this post, we’ll walk through how to install and configure lazy.nvim.

First things first. You can install Neovim using the following commands for Windows or macOS. For Linux distributions, please refer to the neovim documentation, where you can find the specific command line for the package manager corresponding to most Linux distributions.

• Windows

    winget install Neovim.Neovim
  

• macOS

    brew install neovim
  

Once the installation completes successfully, you can start Neovim from your terminal by running the following command:

nvim

This will launch Neovim and display its default welcome screen, confirming that the editor is installed correctly and ready to be configured. From here, we can begin customizing Neovim and setting up our development environment:

The next step is to create the root nvim directory that will contain your Neovim configuration files. The exact location of this directory depends on the operating system, but on Windows and macOS, it can be found at the following path:

• Windows:

    mkdir ~\AppData\Local\nvim
  

• macOS:

    mkdir ~/.config/nvim
  

There are many ways to structure Neovim configuration files, but a clean and common approach is to use a bootstrap file called init.lua, along with a lua/config directory for core Neovim settings. In addition, a lua/plugins directory can be used to store one specification file per plugin you want to configure:

The init.lua file serves as a bootstrap that triggers lazy.nvim, which in turn will load all the plugins you have configured. To set this up, create the init.lua file and add the following content:

-- nvim/init.lua
require("config.lazy")

In a Neovim (Lua) configuration, require is how you load and use Lua modules. It’s equivalent to “importing” code so you can reuse it across files. In this case, it loads the module that´s located at nvim/lua/config/lazy.lua. Therefore, we need to create that file:

-- Bootstrap lazy.nvim
local lazypath = vim.fn.stdpath("data") .. "/lazy/lazy.nvim"
if not (vim.uv or vim.loop).fs_stat(lazypath) then
  local lazyrepo = "https://github.com/folke/lazy.nvim.git"
  local out = vim.fn.system({ "git", "clone", "--filter=blob:none", "--branch=stable", lazyrepo, lazypath })
  if vim.v.shell_error ~= 0 then
    vim.api.nvim_echo({
      { "Failed to clone lazy.nvim:\n", "ErrorMsg" },
      { out, "WarningMsg" },
      { "\nPress any key to exit..." },
    }, true, {})
    vim.fn.getchar()
    os.exit(1)
  end
end
vim.opt.rtp:prepend(lazypath)

-- Setup lazy.nvim
require("lazy").setup({
  spec = {
    -- import your plugins
    { import = "plugins" },
  },
  -- automatically check for plugin updates
  checker = { enabled = true },
})

This file does three critical things:

1. Installs lazy.nvim automatically if it’s missing

2. Adds it to Neovim’s runtime

3. Configures it to load plugins from lua/plugins/

We’ve configured the plugins directory as the location where Neovim should look for plugin specifications. Since no plugins are defined yet, starting Neovim at this stage would result in an error complaining about missing specs. To prevent this, we’ll create our first plugin specification. A good candidate is nvim-lspconfig, one of the most important plugins, as it enables support for Neovim’s built-in LSP client. To do this, we’ll add a simple placeholder for nvim-lspconfig in the file nvim/lua/plugins/lsp.lua:

-- nvim/lua/plugins/lsp.lua placeholder
return {
}

At this point, we can start Neovim again and run the :Lazy command. This opens the interface of lazy.nvim, where you should see the plugin listed, confirming that lazy.nvim has been set up correctly and is managing your plugins as expected. From this window, you can also inspect plugin status, trigger installations, and manage updates, giving you a clear overview of your Neovim plugin ecosystem:

And that’s all for the initial setup of lazy.nvim. With the plugin manager now in place, we’re ready to move on to configuring actual plugins. In the next post, we’ll take the first step in that direction by setting up nvim-lspconfig and enabling Neovim’s built-in Language Server Protocol support. See you in the next one!

Nushell, however, takes a different—and more radical—approach. Instead of treating everything as plain text flowing through pipes, Nushell works with structured data. Commands output tables, records, and lists rather than raw strings, making it possible to filter, sort, and transform data in a way that feels closer to working with a programming language than a traditional shell. This model will feel familiar to anyone who has used PowerShell, which pioneered the idea of passing objects instead of text between commands.

Where Nushell stands apart is in how it applies this idea. PowerShell is deeply tied to the .NET ecosystem and primarily centered around Windows, with objects that map directly to .NET types. Nushell, on the other hand, is built from the ground up as a modern, cross-platform shell, designed to run consistently on Linux, macOS, and Windows. Its data model is shell-native rather than framework-bound, making it feel lighter, more predictable, and better suited to everyday CLI workflows.

In this post, I’ll walk through the configuration choices that took me from a fresh Nushell install to a genuinely productive setup: keybindings, prompts, defaults, and small tweaks that reduce friction and make Nushell feel like home—especially if you’re coming from zsh, fish, or even PowerShell.

Installation

Installing Nushell is straightforward on all major platforms, and the project provides first-class support for macOS, Windows, and Linux. You can choose between system package managers, prebuilt binaries, or building from source, depending on how much control you want.

macOS

On macOS, the recommended approach is using Homebrew:

brew install nushell

This installs Nushell system-wide and keeps it up to date with regular Homebrew upgrades. Once installed, you can start it by running:

nu

If you manage multiple shells, you can also add Nushell to your list of available login shells later, but it works perfectly well as a regular interactive shell without replacing your default one.

Windows

On Windows, Nushell integrates well with modern package managers.

Using winget (available by default on recent Windows versions):

winget install Nushell.Nushell

Alternatively, if you use Chocolatey:

choco install nushell

After installation, you can launch Nushell from Windows Terminal, which provides the best experience thanks to proper font rendering, colors, and Unicode support. Nushell runs natively on Windows and does not require WSL.

Linux

On Linux, Nushell is available in most major distributions’ package repositories.

Examples:

• Arch Linux

    pacman -S nushell
  

• Ubuntu / Debian

    apt install nushell
  

• Fedora

    dnf install nushell
  

If your distribution ships an older version, you can always fall back to installing a prebuilt binary or building from source.

Cross-platform alternatives

If you want the same setup everywhere, Nushell also provides precompiled binaries for all platforms. You can download the appropriate archive, extract it, and place the nu binary somewhere in your PATH.

Another option is installing via Cargo:

cargo install nu

This works on all platforms supported by Rust and is useful if you already have a Rust toolchain installed.

Regardless of the platform, once nu is available in your PATH, the next step is the same everywhere: start Nushell, generate the initial configuration files, and begin customizing it to fit your workflow.

Configuration files

Nushell keeps its configuration split into two main files, each with a clear and well-defined role. Understanding how to edit them—and when to use each one—is key to building a clean and maintainable setup.

The first file is config.nu. This is where most of the user-facing behavior of Nushell is defined: keybindings, menus, prompt configuration, environment behavior, and general shell options. If something affects how Nushell feels while you’re using it, it probably belongs here. Changes to config.nu are applied the next time a Nushell session starts, making it the central place for shaping your interactive experience.

The second file is env.nu. This file is evaluated very early during shell startup and is specifically meant for environment variables. Anything that needs to exist before commands run—such as PATH modifications, EDITOR, locale settings, or tool-specific environment variables—should go here. Keeping environment-related logic in env.nu avoids subtle issues and makes your configuration easier to reason about.

To edit these files, Nushell provides built-in commands that respect your configured editor. Running config nu opens config.nu, and config env opens env.nu. This approach avoids hardcoding paths and works consistently across platforms. Once you save your changes and restart Nushell (or reload the configuration), the updates take effect.

By separating configuration and environment concerns into these two files—and using Nushell’s own commands to edit them—you get a setup that is not only powerful, but also clean, portable, and easy to evolve over time.

Initial configuration

When you first start using Nushell, the initial configuration is intentionally minimal, but two small changes make an immediate difference to the day-to-day experience.

The first step is setting a default editor. Many Nushell commands—such as editing configuration files, opening temporary buffers, or interacting with tools that expect an $EDITOR—rely on this value being defined. Explicitly configuring your editor ensures that Nushell integrates smoothly with the rest of your workflow, whether you prefer a terminal editor like Helix, Neovim, or Vim, or a graphical one. It’s a simple change, but it removes friction right away.

$env.config.buffer_editor = "nvim"

If Neovim is your editor of choice and you’d like a clear, practical introduction to it, I cover that in detail in my book.

The second step is disabling the startup banner. By default, Nushell displays a welcome message every time a new shell starts.

While helpful at first, it quickly becomes visual noise once you’re using Nushell regularly or opening many terminal sessions. Removing the banner results in a cleaner startup and a more focused terminal, especially when working inside tools like tmux or zellij.

$env.config.show_banner = false

These two adjustments don’t change how Nushell works, but they set the tone for the rest of the configuration: a shell that starts cleanly, respects your preferred tools, and stays out of your way unless you need it.

Prompt

This configuration defines a deliberately minimal prompt, with the goal of reducing visual noise and avoiding duplicated information.

By default, Nushell’s prompt includes contextual details such as the current working directory and, on the right side, timing information about the last command. While useful in some scenarios, this can become redundant in a modern terminal setup.

Minimal left prompt

$env.PROMPT_COMMAND = ""

PROMPT_COMMAND controls what Nushell renders as the main (left) prompt. Setting it to an empty string effectively removes all prompt content, leaving a clean, distraction-free command line.

The rationale here is that most modern terminal emulators—such as Alacritty or Windows Terminal—already display the current working directory in the window title or tab name. Repeating the same path information in the prompt adds no new value and consumes horizontal space, especially in deeply nested directories.

By removing it, the focus stays entirely on the command being typed.

Disabling the right prompt

$env.PROMPT_COMMAND_RIGHT = ""

The right prompt is commonly used to show execution time, timestamps, or status information for the last command. While this can be helpful when profiling performance or debugging slow commands, it is rarely useful during normal, day-to-day shell usage.

Unless you are actively measuring how long commands take to run, this timing information becomes background noise—something your eyes learn to ignore. Disabling the right prompt removes that unnecessary cognitive load and results in a calmer, more predictable terminal layout.

History

This history configuration is designed to make Nushell’s command history reliable, searchable, and safe across multiple sessions, while keeping performance predictable.

$env.config.history = {
  file_format: sqlite
  max_size: 5_000_000
  sync_on_enter: true
  isolation: true
}

Each option plays a specific role:

file_format: sqlite

Using SQLite instead of a plain text file gives Nushell a structured, indexed history store. This has several advantages:

• Faster and more reliable history searches, even with large histories

• Reduced risk of corruption compared to appending to a text file

• Better handling of concurrent access when multiple shells are open

In practice, this makes history feel more like a database than a log file—something you can query efficiently rather than just scroll through.

max_size: 5_000_000

This sets a hard limit on the history database size (in bytes). Capping the size prevents unbounded growth over time, which can otherwise lead to slower searches and unnecessary disk usage.

A limit of around 5 MB is large enough to retain a long and useful command history, while still keeping performance snappy and storage under control. Old entries are automatically discarded when the limit is reached, so no manual cleanup is needed.

sync_on_enter: true

With this enabled, each command is written to history as soon as you press Enter, not when the shell exits. This is particularly useful if:

• You run multiple Nushell instances in parallel

• A shell crashes or is force-closed

• You frequently open and close terminals

You don’t lose recent commands, and other running shells can immediately see the updated history.

isolation: true

History isolation ensures that each shell session maintains its own logical history context. Commands executed in one session do not instantly bleed into another, which avoids confusing or irrelevant suggestions when navigating history.

This is especially valuable when working on multiple projects at once or when using tools like tmux or zellij with many panes open. You get history that reflects what you were doing in that session, not noise from elsewhere.

Vi mode

Nushell supports both emacs and vi editing modes for the command line, but enabling vi mode can be significantly more ergonomic for long-term, keyboard-driven workflows.

$env.config.edit_mode = "vi"
$env.config.cursor_shape = {
  vi_insert: line
  vi_normal: block
}
$env.PROMPT_INDICATOR_VI_INSERT = "> "
$env.PROMPT_INDICATOR_VI_NORMAL = "$ "

Why vi mode is more ergonomic

The key ergonomic advantage of vi mode is that it allows you to navigate and edit text without moving your hands away from the home row—the default resting position of your fingers on the keyboard (ASDF / JKL;).

In emacs mode, navigating across words or jumping around a command often involves:

• Modifier-heavy key combinations (Ctrl + …)

• Or reaching for the arrow keys

Both actions break hand positioning and slow down editing. Vi mode, by contrast, uses single-key motions (w, b, e, 0, $, etc.) that keep your hands anchored on the home row, making movement faster and less fatiguing over time.

Clear separation between modes

Vi mode introduces a strict distinction between insert mode (typing text) and normal mode (navigating and editing). While this requires a small mental adjustment at first, it enables far more precise and efficient command-line editing once the muscle memory is built.

To make this distinction obvious, the configuration uses visual cues.

Cursor shape as a mode indicator

$env.config.cursor_shape = {
  vi_insert: line
  vi_normal: block
}

Changing the cursor shape provides immediate, subconscious feedback about the current mode:

• Line cursor → insert mode (typing text)

• Block cursor → normal mode (navigation and editing)

This mirrors the behavior of modal editors like Vim and Helix and helps prevent accidental typing in the wrong mode.

Prompt indicators for extra clarity

$env.PROMPT_INDICATOR_VI_INSERT = "> "
$env.PROMPT_INDICATOR_VI_NORMAL = "$ "

The prompt itself also changes depending on the active mode. This reinforces the cursor shape signal and makes the current state unmistakable, even in terminals where cursor shape changes might not be obvious or supported.

Keybindings

History Hint Complete with Ctrl + Space

$env.config.keybindings ++= [
  {
     name: history_hint_complete
     modifier: control
     keycode: space
     mode: vi_insert
     event: [
       { send: HistoryHintComplete }
    ]
  }
 ]

Nushell’s history suggestions are one of its most powerful features, and this keybinding improves the overall efficiency of using that feature:

• Ergonomically better than default keybindings: By binding the history completion feature to Ctrl + Space, you avoid the need to reach for the arrow keys, which interrupts your typing flow and slows down interaction. Instead, you can use Ctrl + Space as a much more comfortable alternative, keeping your hands anchored on the home row.

  In vi mode, you typically don’t need to move your hands away from the home row for basic operations like navigating the command history, and this keybinding is consistent with that design. Arrow keys require awkward finger stretches, especially when you’re typing fast or working in a split terminal.

• Optimized for frequent use: Since history completion in Nushell is a feature you will use often, being able to trigger it with a single keystroke (Ctrl + Space) is a huge time-saver. It’s faster and feels more natural than reaching for the up or down arrow keys, particularly when navigating through long histories.

  This feature is particularly important in Nushell because its suggestions aren’t just about previous commands, but also about contextual hints—making it more likely you will use the history completion multiple times during your workflow.

System Trash

Nushell provides a convenient way to handle deleted files through a system trash configuration, which can make file deletion safer and more forgiving.

$env.config.rm.always_trash = true

How it works

By default, many shells (including basic rm in bash) permanently delete files, which can be risky—especially if you accidentally remove something important. With this configuration:

• always_trash = true tells Nushell that whenever you use the rm command, files should be sent to the system trash/recycle bin instead of being deleted immediately.

• The exact behavior depends on the operating system:

  • macOS → Files go to the Trash.

  • Windows → Files go to the Recycle Bin.

  • Linux → Files go to the user’s trash directory (e.g., ~/.local/share/Trash), following the FreeDesktop.org specification.

This means you can safely undo deletions, restore files, or review them before permanently removing them.

True color

Enabling true color in Nushell is a small but important configuration, especially when working remotely on a server using a text-mode editor like Helix or Neovim.

# enable true color
$env.COLORTERM = "truecolor"

What it does

• COLORTERM is an environment variable that tells applications the terminal supports 24-bit color, also known as true color.

• Setting it to "truecolor" ensures that both the shell and terminal-based applications can use the full RGB color range, rather than being limited to 16 or 256 colors.

Why it’s important for remote editing

1. Consistent color rendering
   When you SSH into a server, the terminal may not automatically detect true color support. Without this variable, editors like Helix or Neovim might fall back to limited color schemes, making syntax highlighting less informative or harder to read.

2. Better syntax highlighting
   True color allows modern editors to use rich themes accurately, so things like code highlighting, git diff colors, or syntax error markers appear exactly as intended.

3. Improved terminal experience
   Even outside the editor, Nushell itself can render colored outputs (like tables, errors, or commands) with full fidelity, which is particularly useful for scripts, logs, or pipelines with multiple colors.

4. Cross-terminal consistency
   By explicitly setting COLORTERM=truecolor, you ensure that the color behavior is consistent whether you’re on Alacritty, Windows Terminal, iTerm2, or a remote SSH session. This avoids surprises when moving between local and remote environments.

Summary

With these configurations in place, Nushell transforms from a minimal, out-of-the-box shell into a powerful, ergonomic, and modern command-line environment. We’ve set up a minimal, distraction-free prompt, enabled vi-mode with clear visual cues, optimized keybindings for efficiency, configured a reliable and fast history system, ensured safe file deletion with system trash, and enabled true color for a rich, consistent visual experience—even on remote servers. Each tweak focuses on reducing friction, improving productivity, and letting the shell work for you rather than demanding constant attention. The result is a Nushell setup that is not only functional and safe, but also comfortable and enjoyable to use day after day, whether you’re navigating files, writing scripts, or managing multiple projects across platforms.

# minimal prompt
$env.PROMPT_COMMAND = ""

# disable right prompt
$env.PROMPT_COMMAND_RIGHT = ""

# setup default editor
$env.config.buffer_editor = "nvim"

# disable banner
$env.config.show_banner = false

# enable system trash
$env.config.rm.always_trash = true

# setup history
$env.config.history = {
  file_format: sqlite
  max_size: 5_000_000
  sync_on_enter: true
  isolation: true
}

# enable vi mode
$env.config.edit_mode = "vi"
$env.config.cursor_shape = {
  vi_insert: line
  vi_normal: block
}
$env.PROMPT_INDICATOR_VI_INSERT = "> "
$env.PROMPT_INDICATOR_VI_NORMAL = "$ "

# key bindings
$env.config.keybindings ++= [
  {
     name: history_hint_complete
     modifier: control
     keycode: space
     mode: vi_insert
     event: [
       { send: HistoryHintComplete }
    ]
  }
 ]

# enable true color
$env.COLORTERM = "truecolor"

However, Nushell stands out as a highly configurable shell, allowing you to tailor almost every aspect of its behavior, appearance, and interaction to your workflow. From prompts and keybindings to history, editor integration, and system behavior, almost every part of the shell can be adjusted to suit your preferences.

If you want to explore all the available configuration options, Nushell provides a built-in documentation command:

config nu --doc | nu-highlight

Thank you for taking the time to read this post all the way through! I hope you found the tips and configurations useful for making Nushell a more productive and enjoyable shell. Be sure to come back for future posts, where I’ll share more insights, tricks, and practical setups to help you get even more out of your command-line workflow.

Let me walk you through my evaluation process and explain why Codeium is the best AI plugin for Neovim compared to other popular options like the ChatGPT plugin and GitHub Copilot.

Evaluating AI Plugins for Neovim

When evaluating AI plugins for Neovim, I focused on several key factors:

• Cost: Is the plugin free, or does it require a subscription?

• Ease of use: How smoothly does the plugin integrate with Neovim's workflow?

• Flexibility: Can you iterate through different suggestions?

• Compatibility: Does the plugin play nicely with other Neovim features, like the built-in Language Server Protocol (LSP) autocompletion?

The Contenders: ChatGPT, GitHub Copilot, and Codeium

1. ChatGPT Plugin

The ChatGPT plugin is an interesting option that integrates OpenAI's powerful language model directly into Neovim. However, there is a significant downside: it requires an OpenAI API key, which isn't free. The cost can quickly add up, especially if you're a heavy user. Additionally, while the ChatGPT plugin offers impressive suggestions, it lacks some of the seamless integration that other plugins provide, especially when it comes to iterating through multiple suggestions in real-time.

2. GitHub Copilot

GitHub Copilot is another popular choice that uses OpenAI's Codex model to provide code suggestions directly in your editor. However, Copilot also comes with a cost. It requires a subscription, and while it offers powerful autocompletion, there are a few drawbacks:

• Lack of Iteration Control: Copilot's completion system does not provide an easy way to iterate over different suggestions. You get one suggestion at a time, and if you don't like it, you must hit undo or try triggering it again.

• Proprietary Nature: Copilot's integration is somewhat closed, and you have less flexibility in terms of configuration and customization.

Overall, while Copilot is a strong tool, its subscription requirement and limited control over suggestions make it less appealing for Neovim users who want a free, flexible solution.

The Winner: Codeium

After testing the options, I found that Codeium was the clear winner among AI plugins for Neovim. Here's why:

1. It's Free!

First and foremost, Codeium is completely free. You don't need to buy an API key or pay for a subscription to use it. This is a huge advantage over both the ChatGPT plugin and GitHub Copilot, making it a much more accessible option for everyone, from hobbyists to professional developers.

2. Flexible Suggestions

Codeium allows you to iterate over multiple suggestions seamlessly. Unlike GitHub Copilot, which only shows one suggestion at a time, Codeium provides a list of suggestions that you can quickly navigate through. This makes it much easier to find the best suggestion for your code without disrupting your flow.

3. Integration with Neovim's Ghost Text

Codeium makes use of Neovim's ghost text feature, which allows code suggestions to appear in a subtle, non-intrusive manner. This is a big deal because it means that Codeium's suggestions do not collide with the LSP (Language Server Protocol) autocompletion menu. Instead, they work harmoniously together, allowing you to enjoy both Codeium's AI-powered suggestions and Neovim's native LSP completions without any conflicts.

4. Easy to Install and Configure

Codeium is also straightforward to install and configure in Neovim. Whether you're using packer.nvim, vim-plug, or another plugin manager, the setup process is simple, and the configuration options are flexible enough to suit different workflows.

How to Get Started with Codeium in Neovim

If you're ready to try Codeium, here's a quick guide to get you started:

1. Install Codeium: Add the following line to your plugin manager configuration. For example, if you use lazy.nvim, create a plugin file for Codeium (e.g.: lua/plugins/codeium.lua):

    local Plugin = {'Exafunction/codeium.vim'}
    -- Call :Codeium Auth after installation to get Token ID
   

2. Enable the plugin on demand: I follow a minimalistic approach when it comes to Neovim plugins, as you can see in my book about Neovim, to avoid cluttering Neovim with too many plugins.

   

   That's why I've defined a key mapping to load the Codeium plugin only when you toggle it:

    Plugin.cmd = {'CodeiumToggle'}
   
    -- Toggle Codeium
    vim.keymap.set('n', '<leader><CR>', ':CodeiumToggle<CR>')
   
    function Plugin.config()
      -- Disabled by default
      vim.g.codeium_enabled = false
   

3. Setup Codeium Chat: Codeium also provides a prompt through their web site, that you can use similarly to ChatGPT for elaborated questions, like creating a unit test for a function.

   

   As Codeium Chat is exposed by a function in this plugin, I prefer to create a user command to call it, so that it is easier to start the chat:

    function Plugin.config()
      -- 
   
      -- Codeium Chat
      vim.api.nvim_create_user_command('CodeiumChat', function(opts)
        vim.api.nvim_call_function("codeium#Chat", {})
      end,
      {})
   

4. Key mappings: the main actions that you need when you are interacting with Codeium are iterating through suggestions, accept a suggestion and clean the suggestions. I use three key mappings that allow you to access these functionalities in a comfortable way while you are typing. But you could choose whatever key mappings that suits you better:

      -- Key bindings
      vim.g.codeium_no_map_tab = true
      vim.keymap.set('i', '<C-l>', function () return vim.fn['codeium#Accept']() end, { expr = true, silent = true })
      vim.keymap.set('i', '<C-j>', function() return vim.fn['codeium#CycleCompletions'](1) end, { expr = true, silent = true })
      vim.keymap.set('i', '<C-k>', function() return vim.fn['codeium#CycleCompletions'](-1) end, { expr = true, silent = true })
      vim.keymap.set('i', '<C-d>', function() return vim.fn['codeium#Clear']() end, { expr = true, silent = true })
   

5. Status line: One of the things that I like the most about the Codeium plugin is its ability to report information on the status line. That way you can see whether the Codeium plugin is activated or not. The total number of suggestions provided by Codeium. And the current suggestion that is being displayed.

    -- Function to wrap the codeium#GetStatusString Vimscript function
    function get_codeium_status()
      return vim.fn['codeium#GetStatusString']()
    end
   
    function Plugin.config()
      -- 
      -- Add Codeium status to the statusline
      vim.o.statusline = table.concat({
        "%f",                       -- Full file path
        " %h",                      -- Help flag
        " %m",                      -- Modified flag
        " %r",                      -- Readonly flag
        "%=",                       -- Right aligned
        " %y ",                     -- File type
        "%{&ff} ",                  -- File format
        " %p%%",                    -- File position percentage
        " %l:%c ",                  -- Line and column number
        " [Codeium:%{v:lua.get_codeium_status()}]", -- Codeium status
      })
    end
   

6. Complete configuration: putting all these pieces together we end up having this configuration for the Codeium plugin:

    local Plugin = {'Exafunction/codeium.vim'}
    -- Call :Codeium Auth after installation to get Token ID
   
    Plugin.cmd = {'CodeiumToggle'}
   
    -- Toggle Codeium
    vim.keymap.set('n', '<leader><CR>', ':CodeiumToggle<CR>')
   
    -- Function to wrap the codeium#GetStatusString Vimscript function
    function get_codeium_status()
      return vim.fn['codeium#GetStatusString']()
    end
   
    function Plugin.config()
      -- Disabled by default
      vim.g.codeium_enabled = false
   
      -- Codeium Chat
      vim.api.nvim_create_user_command('CodeiumChat', function(opts)
        vim.api.nvim_call_function("codeium#Chat", {})
      end,
      {})
   
      -- Key bindings
      vim.g.codeium_no_map_tab = true
      vim.keymap.set('i', '<C-l>', function () return vim.fn['codeium#Accept']() end, { expr = true, silent = true })
      vim.keymap.set('i', '<C-j>', function() return vim.fn['codeium#CycleCompletions'](1) end, { expr = true, silent = true })
      vim.keymap.set('i', '<C-k>', function() return vim.fn['codeium#CycleCompletions'](-1) end, { expr = true, silent = true })
      vim.keymap.set('i', '<C-d>', function() return vim.fn['codeium#Clear']() end, { expr = true, silent = true })
   
      -- Add Codeium status to the statusline
      vim.o.statusline = table.concat({
        "%f",                       -- Full file path
        " %h",                      -- Help flag
        " %m",                      -- Modified flag
        " %r",                      -- Readonly flag
        "%=",                       -- Right aligned
        " %y ",                     -- File type
        "%{&ff} ",                  -- File format
        " %p%%",                    -- File position percentage
        " %l:%c ",                  -- Line and column number
        " [Codeium:%{v:lua.get_codeium_status()}]", -- Codeium status
      })
    end
   
    return Plugin
   

Conclusion

While there are several AI plugins available for Neovim, Codeium stands out as the best option. It's free, flexible, and integrates perfectly with Neovim's existing autocompletion features. Unlike other alternatives, such as the ChatGPT plugin or GitHub Copilot, Codeium doesn't require a subscription or API key, and it gives you full control over the suggestions it provides.

So, if you're looking for an AI-powered code assistant for Neovim, give Codeium a try—you won't be disappointed!

Happy coding!

Whoa! It's been a long road to get here. But that's just an evidence about the complexities which lie behind this technique. In this post we are going to glue all the pieces that we have been developing through the previous posts to get a basic implementation of async/await. Although it is just a toy experiment to let you understand how this technique can be implemented, it provides you with enough functionality to analyze different scenarios and check how it behaves. It even allows you to chain async calls, so that you can test nested asynchronous functions.

In this example we will use terms that you can find with another names in other programming languages. For example, the term Coroutine that we will be using would equate to the term Future (used in Dart) or Promise (used in Javascript). And any function receiving a Coroutine parameter, would be the equivalent to a function marked as async.

async function loadJson(url) {
  let response = await fetch(url);
  if (response.status == 200) {
    return response.json();
  } else {
    throw new HttpError(response);
  }
}

The main difference with respect to the previous post, is that we will store coroutines in the Thread Pool queue (work queue in the previous post) and the Event Loop queue (done queue in the previous post). As coroutines are resumable, we don't need to carry a couple of callbacks as we did before.

#ifndef __LOOP__
#define __LOOP__

#include "coroutine.h"

typedef struct EventLoop EventLoop;

EventLoop* new_EventLoop();
void delete_EventLoop(EventLoop* l);

void EventLoop_submit(EventLoop* l, CoroutineFn fn);
void EventLoop_run(EventLoop* l);
void EventLoop_stop(EventLoop* l);

typedef struct Task {
        CoroutineFn fn;
        Coroutine*  parent;
        void*       data;
} Task;
Coroutine* EventLoop_async(EventLoop* l, Task task);
void* EventLoop_await(EventLoop* l, Coroutine* co);

#endif

Once a coroutine is done, we can resume the execution of its parent coroutine. This leads us to the next big difference. But in this case it is related to the first post. In this implementation, we are going to extend the definition of coroutines to keep track of their parent coroutines. This technique will allow us to nest coroutine calls.

#ifndef __COROUTINE__
#define __COROUTINE__

#include <stdbool.h>
#include <ucontext.h>

typedef struct Coroutine Coroutine;
typedef void* (*CoroutineFn)(Coroutine*);

struct Coroutine {
    Coroutine*  parent;      // New field
    CoroutineFn fn;
    ucontext_t  caller_ctx;
    ucontext_t  callee_ctx;
    void*       yield_value;
    bool        finished;
};

Coroutine* new_Coroutine(CoroutineFn fn, Coroutine* parent);
void* Coroutine_resume(Coroutine* c);
void Coroutine_suspend(Coroutine* c, void* value);
void delete_Coroutine(Coroutine* c);

#endif

When we want to submit a new task to the Event Loop, we create a coroutine without a parent, and push it into the Event Loop queue.

void EventLoop_submit(EventLoop* l, CoroutineFn fn) {
        Coroutine* co = new_Coroutine(fn, NULL);
        EventLoop_resume(l, co);
}

That coroutine will be picked up by the Event Loop later on. It will resume the coroutine, and after that it will check whether that coroutine has finished or not. If that is the case and it has a parent, the Event Loop will queue up the parent coroutine to go up the nested calls of coroutines. Otherwise, the Event Loop will delete the coroutine.

void EventLoop_run(EventLoop* l) {
        AtomicBool_set(l->running, true);

        while (AtomicBool_get(l->running)) {
                Coroutine* co = NULL;
                if (SafeQueue_pop(l->queue, &co)) {
                        Coroutine_resume(co);
                        if (co->finished) {
                                if (co->parent != NULL) {
                                        EventLoop_resume(l, co->parent);
                                } else {
                                        delete_Coroutine(co);
                                }
                        }
                }
        }
}

When a coroutine is resumed, we get to execute its code. There, we can delegate the execution of an asynchronous call to the worker threads. To do so, we can call the async function. We have to wrap the function that we want to execute, a reference to the current coroutine as parent and a reference to any input data into a Task object that we use as argument for the async function. Then, it will create a new coroutine with the data provided in the Task object, and push it into the Thread Pool queue.

void* download_images(Coroutine* co) {
        printf("[%lu] download image 1\n", pthread_self());
        string str1;
        Coroutine* c1 = EventLoop_async(loop, (Task){
                        .fn = get_image,
                        .parent = co,
                        .data = str1
                        });
        char* img1 = (char*)EventLoop_await(loop, c1);
        printf("[%lu] %s\n", pthread_self(), img1);

        printf("[%lu] download image 2\n", pthread_self());
        string str2;
        Coroutine* c2 = EventLoop_async(loop, (Task){
                        .fn = get_image,
                        .parent = co,
                        .data = str2
                        });
        char* img2 = (char*)EventLoop_await(loop, c2);
        printf("[%lu] %s\n", pthread_self(), img2);

        return 0;
}

Coroutine* EventLoop_async(EventLoop* l, Task task) {
        Coroutine* c = new_Coroutine(task.fn, task.parent);
        c->yield_value = task.data;
        ThreadPool_submit(l->pool, c);
        return c;
}

The Worker Threads behave similarly to the Event Loop. They pick up a coroutine from the Thread Pool queue and resume it. After that, they will check if that coroutine has finished. If that is the case, they will queue up their parent coroutine into the Event Loop queue.

void* ThreadPool_run(void* arg) {
        ThreadPool* p = (ThreadPool*) arg;

        while (AtomicBool_get(p->running)) {
                Coroutine* co;
                if (SafeQueue_pop(p->queue, &co)) {
                        Coroutine_resume(co);
                        if (co->finished) {
                                EventLoop_resume(p->loop, co->parent);
                        }
                }
        }

        return NULL;
}

After triggering an asynchronous call from a coroutine, we can wait for its execution using the await function. It will suspend the parent coroutine. And when the execution flow comes back again, it saves the yield value of the current coroutine to return it. But before that, it deletes the coroutine.

void* EventLoop_await(EventLoop* l, Coroutine* co) {
        Coroutine_suspend(co->parent, co->parent->yield_value);
        void* result = co->yield_value;
        delete_Coroutine(co);
        return result;
}

Event Loop

Here is the complete code of the final Event Loop:

static
void EventLoop_resume(EventLoop* l, Coroutine* co);

typedef struct ThreadPool {
        SafeQueue*      queue;
        EventLoop*      loop;
        pthread_t*      threads;
        size_t          nthreads;
        AtomicBool*     running;
} ThreadPool;

static
void ThreadPool_submit(ThreadPool* p, Coroutine* co) {
        while(!SafeQueue_push(p->queue, &co));
}

static
void* ThreadPool_run(void* arg) {
        ThreadPool* p = (ThreadPool*) arg;

        while (AtomicBool_get(p->running)) {
                Coroutine* co;
                if (SafeQueue_pop(p->queue, &co)) {
                        Coroutine_resume(co);
                        if (co->finished) {
                                EventLoop_resume(p->loop, co->parent);
                        }
                }
        }

        return NULL;
}

static
void ThreadPool_start(ThreadPool* p) {
        AtomicBool_set(p->running, true);

        for (int i=0; i<p->nthreads; ++i) {
                pthread_create(&p->threads[i], NULL, ThreadPool_run, p);
        }
}

static
ThreadPool* new_ThreadPool(EventLoop* loop) {
        ThreadPool* p = calloc(1, sizeof(ThreadPool));

        p->nthreads = NUM_THREADS;
        p->queue    = new_SafeQueue(sizeof(Coroutine*), QUEUE_SIZE);
        p->threads  = calloc(p->nthreads, sizeof(pthread_t));
        p->loop = loop;
        p->running = new_AtomicBool();

        ThreadPool_start(p);

        return p;
}

static
void delete_ThreadPool(ThreadPool* p) {
        AtomicBool_set(p->running, false);

        for (int i=0; i<p->nthreads; ++i) {
                pthread_join(p->threads[i], NULL);
        }
        free(p->threads);

        delete_SafeQueue(p->queue);
        delete_AtomicBool(p->running);

        free(p);
}

// *******************************************************

struct EventLoop {
        ThreadPool* pool;
        SafeQueue*  queue;
        AtomicBool* running;
};

EventLoop* new_EventLoop() {
        EventLoop* l = calloc(1, sizeof(EventLoop));

        l->pool = new_ThreadPool(l);
        l->queue = new_SafeQueue(sizeof(Coroutine*), QUEUE_SIZE);
        l->running = new_AtomicBool();

        return l;
}

void delete_EventLoop(EventLoop* l) {
        delete_ThreadPool(l->pool);
        delete_SafeQueue(l->queue);
        delete_AtomicBool(l->running);
        free(l);
}

static
void EventLoop_resume(EventLoop* l, Coroutine* co) {
        while (!SafeQueue_push(l->queue, &co));
}

void EventLoop_run(EventLoop* l) {
        AtomicBool_set(l->running, true);

        while (AtomicBool_get(l->running)) {
                Coroutine* co = NULL;
                if (SafeQueue_pop(l->queue, &co)) {
                        Coroutine_resume(co);
                        if (co->finished) {
                                if (co->parent != NULL) {
                                        EventLoop_resume(l, co->parent);
                                } else {
                                        delete_Coroutine(co);
                                }
                        }
                }
        }
}

void EventLoop_stop(EventLoop* l) {
        AtomicBool_set(l->running, false);
}

void EventLoop_submit(EventLoop* l, CoroutineFn fn) {
        Coroutine* co = new_Coroutine(fn, NULL);
        EventLoop_resume(l, co);
}

Coroutine* EventLoop_async(EventLoop* l, Task task) {
        Coroutine* c = new_Coroutine(task.fn, task.parent);
        c->yield_value = task.data;
        ThreadPool_submit(l->pool, c);
        return c;
}

void* EventLoop_await(EventLoop* l, Coroutine* co) {
        Coroutine_suspend(co->parent, co->parent->yield_value);
        void* result = co->yield_value;
        delete_Coroutine(co);
        return result;
}

Example

And here is the code of the example:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <pthread.h>
#include <time.h>

#include "loop.h"
#include "coroutine.h"

#define STR 128
typedef char string[STR];

static
EventLoop* loop;

void* get_image(Coroutine* co) {
        sleep(1);
        int r = rand();

        char buffer[16];
        sprintf(buffer, "%d", r);

        char* result = (char*)co->yield_value;
        strcat(result, "image id: ");
        strcat(result, buffer);

        return result;
}

void* download_images(Coroutine* co) {
        printf("[%lu] download image 1\n", pthread_self());
        string str1;
        Coroutine* c1 = EventLoop_async(loop, (Task){
                        .fn = get_image,
                        .parent = co,
                        .data = str1
                        });
        char* img1 = (char*)EventLoop_await(loop, c1);
        printf("[%lu] %s\n", pthread_self(), img1);

        printf("[%lu] download image 2\n", pthread_self());
        string str2;
        Coroutine* c2 = EventLoop_async(loop, (Task){
                        .fn = get_image,
                        .parent = co,
                        .data = str2
                        });
        char* img2 = (char*)EventLoop_await(loop, c2);
        printf("[%lu] %s\n", pthread_self(), img2);

        return 0;
}

void* say_hello(Coroutine* co) {
        printf("[%lu] Hello\n", pthread_self());
        return NULL;
}

void* run_cli(void* arg) {
        char* line = NULL;
        size_t len = 0;
        ssize_t read;
        while ((read = getline(&line, &len, stdin)) != -1) {
                if (read>2 && strncmp("quit", line, read-1) == 0) {
                        EventLoop_stop(loop);
                        break;
                } else if (read>2 && strncmp("download", line, read-1) == 0) {
                        EventLoop_submit(loop, download_images);
                } else {
                        EventLoop_submit(loop, say_hello);
                }
        }
        if (line != NULL) {
                free(line);
        }

        return NULL;
}

void test_eventLoop() {
        loop = new_EventLoop();

        pthread_t t;
        pthread_create(&t, NULL, run_cli, NULL);

        EventLoop_run(loop);

        delete_EventLoop(loop);

        pthread_join(t, NULL);
}

int main() {
        srand(time(NULL));

        test_eventLoop();

        return 0;
}

And the corresponding output:

[139925417781056] Hello
download
[139925417781056] download image 1
[139925417781056] image id: 648455896
[139925417781056] download image 2
[139925417781056] image id: 39422855

Conclusion

As you can see, the client code is much easier to read using async/await. And it also helps us to manage memory. In this example we are using an static allocation of memory. But we could also use dynamic memory allocation, and it would be easy to match allocation and deallocation of memory, as both of them could happen in the same function.

And that's all. I hope you have enjoyed this journey through the internals of a toy async/await implementation. It definitely helped me understand how its logic works, which is something that is not pretty straightforward the first time you find out about this pattern. Thank you very much if you made it till the end of this series. And I would very much appreciate any comment or suggestion that you may have about it. See you in the next post!

An Event Loop in a UI framework is a critical component responsible for managing and handling various events, such as user input (like mouse clicks or keyboard presses), system notifications, and other asynchronous operations. It ensures that the events are processes sequentially one by one, avoiding any overlapping among them.

In the case of asynchronous programming (i.e.: processing done by other threads), if we send the output generated by the worker threads back to the Event Loop, we are also able to sequence the processing of those outputs. Thus, avoiding any overlapping. The point being is that as an Event Loop is a single thread, we can use it as synchronization point whenever we need to sequence the processing of multiple sources of information.

One of the most famous Event Loops implementations is libuv, which is a library used for implementing an event-driven asynchronous I/O model in Node.js.

As you can see, every task in libuv is linked to a couple of callbacks. The work_cb, which is executed by the worker thread that processes that task. And the after_work_cb, which is executed by the Event Loop thread when it picks up a task from the done queue.

We can follow a similar approach of using a pair of callbacks for each event or task to be executed. But instead of using an input queue and a done queue, we can merge both of them together to simplify the implementation. So the user input events will be mixed with the done tasks in the same queue. This is just a simplification to keep the code shorter and easier to read in this example.

Starting from our basic Thread Pool implementation, we can develop a basic Event Loop on top of that as follows. It will just basically provide a run and stop functions to start and stop the Event Loop. But the key functions are submit and async.

typedef void (*TaskFn)(void*);

typedef struct Task {
        TaskFn cb;
        TaskFn done_cb;
        void*  data;
} Task;

typedef struct EventLoop EventLoop;

EventLoop* new_EventLoop();
void delete_EventLoop(EventLoop* l);

void EventLoop_submit(EventLoop* l, Task task);
void EventLoop_async(EventLoop* l, Task task);
void EventLoop_run(EventLoop* l);
void EventLoop_stop(EventLoop* l);

The submit function will insert a task into the done queue, for the Event Loop thread to process it. Both the user input thread, as well as the worker threads will use this function to insert tasks into that queue.

Whereas the async function will insert a task into the work queue, so that it gets picked up by a worker thread for its processing. This function will be called from the Event Loop thread.

typedef struct ThreadPool {
        SafeQueue*    queue;
        EventLoop*    loop;
        pthread_t*    threads;
        size_t        nthreads;
        AtomicBool*   running;
} ThreadPool;

static
void* ThreadPool_run(void* arg) {
        ThreadPool* p = (ThreadPool*) arg;

        while (AtomicBool_get(p->running)) {
                Task task;
                if (SafeQueue_pop(p->queue, &task)) {
                        task.cb(task.data);
                        EventLoop_submit(p->loop, task);
                }
        }

        return NULL;
}

static
void ThreadPool_start(ThreadPool* p) {
        AtomicBool_set(p->running, true);

        for (int i=0; i<p->nthreads; ++i) {
                pthread_create(&p->threads[i], NULL, ThreadPool_run, p);
        }
}

static
ThreadPool* new_ThreadPool(EventLoop* loop) {
        ThreadPool* p = calloc(1, sizeof(ThreadPool));

        p->nthreads = NUM_THREADS;
        p->queue    = new_SafeQueue(sizeof(Task), QUEUE_SIZE);
        p->threads  = calloc(p->nthreads, sizeof(pthread_t));
        p->loop = loop;
        p->running = new_AtomicBool();

        ThreadPool_start(p);

        return p;
}

static
void delete_ThreadPool(ThreadPool* p) {
        AtomicBool_set(p->running, false);

        for (int i=0; i<p->nthreads; ++i) {
                pthread_join(p->threads[i], NULL);
        }
        free(p->threads);

        delete_SafeQueue(p->queue);
        delete_AtomicBool(p->running);

        free(p);
}

static
void ThreadPool_submit(ThreadPool* p, Task task) {
        while(!SafeQueue_push(p->queue, &task));
}

struct EventLoop {
        ThreadPool* pool;
        SafeQueue*  queue;
        AtomicBool* running;
};

EventLoop* new_EventLoop() {
        EventLoop* l = calloc(1, sizeof(EventLoop));

        l->pool = new_ThreadPool(l);
        l->queue = new_SafeQueue(sizeof(Task), QUEUE_SIZE);
        l->running = new_AtomicBool();

        return l;
}

void delete_EventLoop(EventLoop* l) {
        delete_ThreadPool(l->pool);
        delete_SafeQueue(l->queue);
        delete_AtomicBool(l->running);
        free(l);
}

void EventLoop_run(EventLoop* l) {
        AtomicBool_set(l->running, true);

        while (AtomicBool_get(l->running)) {
                Task task;
                if (SafeQueue_pop(l->queue, &task)) {
                        task.done_cb(task.data);
                }
        }
}

void EventLoop_submit(EventLoop* l, Task task) {
        while (!SafeQueue_push(l->queue, &task));
}

void EventLoop_async(EventLoop* l, Task task) {
        ThreadPool_submit(l->pool, task);
}

void EventLoop_stop(EventLoop* l) {
        AtomicBool_set(l->running, false);
}

We can now code a simple command line example that exercises our implementation. It will accept a "download" command to simulate a long running job that we want to delegate to the worker threads for asynchronous programming. Meanwhile, if the user enters any other command (apart from "quit"), the application will respond with a "hello" message just to confirm that the Event Loop is able to keep processing user input events, while the long running tasks are being processed by the worker threads.

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <pthread.h>
#include <time.h>

#include "loop.h"

#define STR 128

static
EventLoop* loop;

void get_image(void* data) {
        char* result = (char*)data;

        int r = rand();
        char buffer[16];
        sprintf(buffer, "%d", r);
        strcat(result, buffer);
}

void done_get_image(void* arg) {
        printf("[%lu] BEGIN done_get_image ", pthread_self());

        sleep(1);
        char* img = (char*)arg;
        printf("%s ", img);
        free(img);

        printf("[%lu] END done_get_image\n", pthread_self());
}

void download_img(void* arg) {
        char* str = calloc(STR, sizeof(char));
        printf("[%lu] download image\n", pthread_self());
        EventLoop_async(loop, (Task){
                        .cb = get_image,
                        .done_cb = done_get_image,
                        .data = str
                        });
}

void say_hello(void* arg) {
        printf("[%lu] Hello\n", pthread_self());
}

void* run_cli(void* arg) {
        char* line = NULL;
        size_t len = 0;
        ssize_t read;
        while ((read = getline(&line, &len, stdin)) != -1) {
                if (read>2 && strncmp("quit", line, read-1) == 0) {
                        EventLoop_stop(loop);
                        break;
                } else if (read>2 && strncmp("download", line, read-1) == 0) {
                        EventLoop_submit(loop, (Task){
                                        .cb = NULL,
                                        .done_cb = download_img,
                                        .data = NULL
                                        });
                } else {
                        EventLoop_submit(loop, (Task){
                                        .cb = NULL,
                                        .done_cb = say_hello,
                                        .data = NULL
                                        });
                }
        }
        if (line != NULL) {
                free(line);
        }

        return NULL;
}

void test_eventLoop() {
        loop = new_EventLoop();

        pthread_t t;
        pthread_create(&t, NULL, run_cli, NULL);

        EventLoop_run(loop);

        delete_EventLoop(loop);

        pthread_join(t, NULL);
}

int main() {
        srand(time(NULL));

        test_eventLoop();

        return 0;
}

Unlike the output that we got in the previous post, where the text generated by the worker threads could be overlapped. In this case we cannot run into a situation where the output of one task overlaps another one, because all of them are processed by the single Event Loop thread. Thus, we managed to sequence the processing of the tasks outputs.

[139901546895168] Hello

[139901546895168] Hello
do
[139901546895168] download image
do
[139901546895168] BEGIN done_get_image 1087146321 [139901546895168] END done_get_image
[139901546895168] download image
[139901546895168] BEGIN done_get_image 925167204 [139901546895168] END done_get_image

[139901546895168] Hello

However, as you can tell from this ugly code, we have run into something similar to the famous callback hell that we can see in the Javascript world. It is really cumbersome to chain asynchronous calls using this strategy.

void done_get_image_1(void* arg) {
        char* img = (char*)arg;
        printf("[%lu] %s\n", pthread_self(), img);
        free(img);

        // We have to chain another async call from this first callback
        char* str = calloc(STR, sizeof(char));
        printf("[%lu] download image 2\n", pthread_self());
        EventLoop_async(loop, (Task){
                        .cb = get_image,
                        .done_cb = done_get_image_2,
                        .data = str
                        });
}

And it is also very uncomfortable to track the allocation of dynamic memory in languages that do not support garbage collection.

void done_get_image_2(void* arg) {
        char* img = (char*)arg;
        printf("[%lu] %s\n", pthread_self(), img);

        // It is really difficult to track where this memory was allocated
        free(img);
}

Here is where the async/await ergonomics come in handy. Using them we can translate the callbacks chain into sequential code. That way it feels like we were writing synchronous code. Making it much easier for the reader to understand the logic of the program. And it also makes handling the deallocation of dynamic memory easier. This is specially valuable for languages like C or Zig, where there is no garbage collection. But we will see that in the last part of our series, where we will implement the async/await logic using all the pieces that we have been developing so far.

A thread pool is a concurrent programming concept that involves managing a group or pool of pre-initialized threads to efficiently execute tasks. The architecture of a thread pool typically consists of the following components:

• Thread Pool Queue: The queue holds the tasks or jobs that need to be executed by the thread pool. When a task arrives, it is added to the queue, and an available thread from the pool picks it up for execution. This queue helps in managing the order of task execution and prevents overloading the system with too many active threads simultaneously.

• Worker Threads: Worker threads are pre-initialized threads kept in the pool, ready to execute tasks. They continuously check the task queue for new tasks to execute. Once a task is obtained from the queue, a worker thread processes it and becomes available for the next task.

As many threads will be accessing the task queue in parallel for new tasks, we need to implement a mechanism that avoids data races. We will do that by implementing a thread safe queue. But first, we need to start with the implementation of a basic queue.

Queue

A queue is a fundamental data structure in computer science that follows the First-In-First-Out (FIFO) principle. In a queue, elements are added to the rear (enqueue) and removed from the front (dequeue). Imagine it as a line of people waiting for a service—those who arrive first are served first.

A circular queue, also known as a ring buffer, is a variation of the basic queue data structure with a circular arrangement of elements in a fixed-size array. Unlike a traditional queue, where elements are added at one end and removed from the other, a circular queue reuses the space in the array, creating a circular pattern.

For example, if we have a queue of six elements of capacity, we push six items and then we pop out three of them, we'll end up having a queue with the following status:

If after that we push three more items into the queue, its status will be the following:

We will use a basic implementation of a circular queue as baseline for a thread safe queue.

typedef struct Queue {
        size_t item_size;
        size_t front;
        size_t capacity;
        size_t count;
        void*  items;
} Queue;

Queue* new_Queue(size_t item_size, size_t capacity) {
        if (capacity <= 0) {
                return NULL;
        }

        Queue* q = calloc(1, sizeof(Queue));
        q->item_size = item_size;
        q->capacity  = capacity;
        q->items     = calloc(capacity, item_size);

        return q;
}

void delete_Queue(Queue* q) {
        free(q->items);
        free(q);
}

bool Queue_push(Queue* q, void* item) {
        if (q->count == q->capacity) {
                return false;
        }
        char* items = (char*)q->items;
        size_t index = (q->front + q->count) % q->capacity;
        char* dst = &items[index * q->item_size];
        memcpy(dst, item, q->item_size);
        q->count++;
        return true;
}

bool Queue_pop(Queue* q, void* item) {
        if (q->count == 0) {
                return false;
        }
        char* items = (char*)q->items;
        size_t index = q->front;
        char* src = &items[index * q->item_size];
        memcpy(item, src, q->item_size);
        q->count--;
        q->front = (q->front + 1) % q->capacity;
        return true;
}

Thread Safe Queue

A thread-safe queue is a data structure designed to be used in concurrent or multi-threaded environments where multiple threads may simultaneously access and modify the queue. The primary goal of a thread-safe queue is to ensure that operations such as enqueue (insertion) and dequeue (removal) can be performed safely without leading to race conditions, data corruption, or other synchronization issues. In our case, the main thread will be accessing the queue for pushing new tasks. Whereas the worker threads will access the queue for popping queued tasks.

Condition variables are synchronization primitives used in concurrent programming to coordinate the execution of threads. We will use them together with mutexes to implement a thread-safe queue. When the client tries to push a new item into the queue but there is no room available, the client will wait on a condition variable until it gets notified about the availability of a new slot for pushing the item. But to avoid keeping the client stuck in that call in case there is no room available in the queue for a long time, we will use a timed condition variable. So that the client can wake up after each second to allow them to decide if they want to keep waiting or not. The same happens on the other end of the queue. We will use a timed condition variable for popping out items from the queue.

typedef struct SafeQueue {
        pthread_mutex_t  mutex;
        pthread_cond_t   pop_cv;
        pthread_cond_t   push_cv;
        Queue*           queue;
} SafeQueue;

SafeQueue* new_SafeQueue(size_t item_size, size_t capacity) {
        SafeQueue* q = calloc(1, sizeof(SafeQueue));

        pthread_mutex_init(&q->mutex, NULL);
        pthread_cond_init(&q->pop_cv, NULL);
        pthread_cond_init(&q->push_cv, NULL);

        q->queue = new_Queue(item_size, capacity);

        return q;
}

void delete_SafeQueue(SafeQueue* q) {
        delete_Queue(q->queue);

        pthread_cond_destroy(&q->push_cv);
        pthread_cond_destroy(&q->pop_cv);
        pthread_mutex_destroy(&q->mutex);

        free(q);
}

struct timespec get_timeout() {
        struct timespec result;
        clock_gettime(CLOCK_REALTIME, &result);
        result.tv_sec += 1;
        return result;
}

bool SafeQueue_push(SafeQueue* q, void* item) {
        bool result = false;

        pthread_mutex_lock(&q->mutex);
        {
                result = Queue_push(q->queue, item);
                if (!result) {
                        struct timespec timeout = get_timeout();
                        pthread_cond_timedwait(&q->push_cv, &q->mutex, &timeout);
                }
                else {
                        pthread_cond_signal(&q->pop_cv);
                }
        }
        pthread_mutex_unlock(&q->mutex);

        return result;
}

bool SafeQueue_pop(SafeQueue* q, void* item) {
        bool result = false;

        pthread_mutex_lock(&q->mutex);
        {
                result = Queue_pop(q->queue, item);
                if (!result) {
                        struct timespec timeout = get_timeout();
                        pthread_cond_timedwait(&q->pop_cv, &q->mutex, &timeout);
                }
                else {
                        pthread_cond_signal(&q->push_cv);
                }
        }
        pthread_mutex_unlock(&q->mutex);

        return result;
}

Atomic Bool

In order to stop the worker threads from the thread pool, we need a mechanism that allow us to notify them without running into race conditions. We will be using an atomic Boolean variable for that. But as C does not have support for atomic values, we will implement one from scratch.

An atomic bool is a type of variable in concurrent programming that supports atomic (indivisible) operations. We will use an atomic bool to signal to the threads in the pool that they should stop their execution.

typedef struct AtomicBool {
        pthread_mutex_t mutex;
        bool            value;
} AtomicBool;

AtomicBool* new_AtomicBool() {
        AtomicBool* b = calloc(1, sizeof(AtomicBool));
        pthread_mutex_init(&b->mutex, NULL);
        return b;
}

void delete_AtomicBool(AtomicBool* b) {
        pthread_mutex_destroy(&b->mutex);
        free(b);
}

bool AtomicBool_get(AtomicBool* b) {
        bool result = false;
        pthread_mutex_lock(&b->mutex);
        {
                result = b->value;
        }
        pthread_mutex_unlock(&b->mutex);
        return result;
}

void AtomicBool_set(AtomicBool* b, bool value) {
        pthread_mutex_lock(&b->mutex);
        {
                b->value = value;
        }
        pthread_mutex_unlock(&b->mutex);
}

Thread Pool

Now that we have all the components ready, we put them all together to implement a basic version of a thread pool.

static
const int NUM_THREADS = 4;

static
const int QUEUE_SIZE = 10;

typedef struct ThreadPool {
        SafeQueue*    queue;
        pthread_t*    threads;
        size_t        nthreads;
        AtomicBool*   running;
} ThreadPool;

void* ThreadPool_run(void* arg) {
        ThreadPool* p = (ThreadPool*) arg;

        while (AtomicBool_get(p->running)) {
                Task task;
                if (SafeQueue_pop(p->queue, &task)) {
                        task.cb(task.data);
                }
        }

        return NULL;
}

void ThreadPool_start(ThreadPool* p) {
        AtomicBool_set(p->running, true);

        for (int i=0; i<p->nthreads; ++i) {
                pthread_create(&p->threads[i], NULL, ThreadPool_run, p);
        }
}

ThreadPool* new_ThreadPool() {
        ThreadPool* p = calloc(1, sizeof(ThreadPool));

        p->nthreads = NUM_THREADS;
        p->queue    = new_SafeQueue(sizeof(Task), QUEUE_SIZE);
        p->threads  = calloc(p->nthreads, sizeof(pthread_t));
        p->running = new_AtomicBool();

        ThreadPool_start(p);

        return p;
}

void delete_ThreadPool(ThreadPool* p) {
        AtomicBool_set(p->running, false);

        for (int i=0; i<p->nthreads; ++i) {
                pthread_join(p->threads[i], NULL);
        }
        free(p->threads);

        delete_SafeQueue(p->queue);
        delete_AtomicBool(p->running);

        free(p);
}

void ThreadPool_submit(ThreadPool* p, Task task) {
        while(!SafeQueue_push(p->queue, &task));
}

This pattern can be used for handling server requests. As each request is independent, each one of them can generate a response at is own pace. The problem is that in case the requests take up too much time to serve, newer clients may not have a response for a long period of time. That is what can be avoided using a green threads pattern, similar to the one provided by goroutines in Go. In that case, the runtime scheduler can reschedule the execution of goroutines, so none of them keeps waiting too much.

In the case of client code, we can use this pattern for executing tasks asynchronously. However, the limitation of this pattern is that in case several asynchronous tasks generate output for the client, as they are running independently without any synchronization we can end up having overlapping outputs coming out from multiple worker threads.

For example, let's write an example program that uses this thread pool to simulate several asynchronous calls that write their output into the console.

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <pthread.h>
#include <time.h>

#include "pool.h"

void download_img(void* data) {
        printf("[%lu] BEGIN download_img", pthread_self());

        sleep(1);
        int r = rand();
        printf("%d ", r);

        printf("[%lu] END download_img\n", pthread_self());
}

void say_hello() {
        printf("[%lu] Hello\n", pthread_self());
}

int main() {
        srand(time(NULL));

        ThreadPool* pool = new_ThreadPool();
        {
                char* line = NULL;
                size_t len = 0;
                ssize_t read;
                while ((read = getline(&line, &len, stdin)) != -1) {
                        if (read>2 && strncmp("quit", line, read-1) == 0) {
                                break;
                        } else if (read>2 && strncmp("hello", line, read-1) == 0) {
                                say_hello();
                        } else if (read>2 && strncmp("download", line, read-1) == 0) {
                                for (int i=0; i<5; ++i) {
                                        ThreadPool_submit(pool, (Task){
                                                        .cb = download_img,
                                                        .data = NULL
                                                        });
                                }
                        }
                }
                if (line != NULL) {
                        free(line);
                }
        }
        delete_ThreadPool(pool);

        return 0;
}

As there is no synchronization between the worker threads execution, their output can overlap each other. So multiple executions of the download command may have different outputs.

>clang -g -Iinclude cmd/main.c src/** -o bin/main -lpthread
>bin/main
download
[140364231157504] BEGIN download_img[140364247942912] BEGIN download_img[140364239550208] BEGIN download_img[140364256335616] BEGIN download_img1677218124 [140364256335616] END download_img
[140364256335616] BEGIN download_img1290723833 [140364239550208] END download_img
1311847477 [140364247942912] END download_img
1998703620 [140364231157504] END download_img
1256452635 [140364256335616] END download_img

download
[140364256335616] BEGIN download_img[140364239550208] BEGIN download_img[140364247942912] BEGIN download_img[140364231157504] BEGIN download_img649066008 448770898 1852008977 [140364256335616] END download_img[140364256335616] BEGIN download_img[140364247942912] END download_img
501791773 [140364239550208] END download_img
[140364231157504] END download_img
1564908724 [140364256335616] END download_img

In order to avoid race conditions in the output generated by asynchronous tasks, we need to use the next piece of our async/await puzzle, event loops. We'll cover that in the next part of this series.

One of the things that grabbed my attention the most while I was having a first look at the features provided by the Zig language was the async/await keywords (even though they have been finally discarded).

const net = @import("std").net;

pub const io_mode = .evented;

pub fn main() !void {
    const addr = try net.Address.parseIp("127.0.0.1", 7000);

    var sendFrame = async send_message(addr);
    // ... do something else while
    //     the message is being sent ...
    try await sendFrame;
}

// Note how the function definition doesn't require any static
// `async` marking. The compiler can deduce when a function is
// async based on its usage of `await`.
fn send_message(addr: net.Address) !void {
    // We could also delay `await`ing for the connection
    // to be established, if we had something else we
    // wanted to do in the meantime.
    var socket = try net.tcpConnectToAddress(addr);
    defer socket.close();

    // Using both await and async in the same statement
    // is unnecessary and non-idiomatic, but it shows
    // what's happening behind the scenes when `io_mode`
    // is `.evented`.
    _ = try await async socket.write("Hello World!\n");
}

I had already seen them in other languages like C#, Python, Dart, Rust, etc. But as they never felt too appealing to me for writing server side code, I was not paying too much attention to that programming pattern. I prefer the concept of goroutines introduced by Golang much more for handling server requests, agreeing with what Loris Cro says on his awesome article about async/await. However, after digging a bit deeper into this subject, it is clear to me that using this pattern for writing client code is very convenient. You can express the logic of your application in a much clearer way. This is related to the infamous "callback hell" very well known in the javascript world. But once I understood the pros of this feature, I wanted to know how it works under the hood. The first time you approach this programming pattern, it is not completely straightforward to understand how it works, because it involves several complex concepts: coroutines, event loops, thread pools, etc. So I decided to develop a toy implementation of async/await in C. As this language does not provide any of those concepts out of the box, building each one of them from scratch will allow you to gather the knowledge required to truly understand how async/await works.

Coroutines

When you start looking for a definition of coroutines, you usually find resources that define them as "computer program components that generalize subroutines for non-preemptive multitasking by allowing execution to be suspended and resumed. They are also known as cooperative multitasking or cooperative threading." Easy, right? Coroutines are just resumable functions. Period. It just means that the execution of a coroutine can be suspended. At that point, the execution flow will come back to the caller function. And if that function resumes the coroutine, the execution flow will go back to the coroutine code where it was suspended.

How is this possible? When we execute a subroutine, we create a new function stack to store function parameters, local variables and result value. Once the function returns, its function stack is destroyed.

However, in the case of coroutines, as they can be suspended and resumed, we need to keep track of the caller and callee function stacks. That way we can switch between them when we suspend or resume the coroutine.

We can create a simple implementation of coroutines in C using the ucontext (user context) module. It provides you with the functions to create a user context (getcontext and makecontext), which includes the following information:

• the contents of the calling thread's machine registers

• the signal mask

• the current execution stack

And the function to switch contexts (swapcontext). Keeping a reference to the caller user context and the callee user context on the coroutine object, we can switch back and forth between them when we suspend or resume the coroutine.

The suspend function of a coroutine is also referred to as yield, because it can produce an intermediate value that can be received on the caller function. So the coroutine can provide values to the caller function every time it suspends. But I prefer to stick to the suspend name, as it is clearer and it also matches the keyword used by the Zig language.

When we create a coroutine, we allocate the memory required for storing the context of the coroutine function.

The resume function will save the caller function context, and swap it with the context of the coroutine function. The execution flow will continue at the point of the coroutine function where it was suspended. If the coroutine function has not yet been suspended yet, the execution flow will continue at the starting point of the coroutine function.

Similarly, the suspend function will save the callee function (coroutine function) context, and swap it with the context of the caller function. The execution flow will continue at the point of the caller function where it resumed the coroutine function.

Implementation

Let's code a basic implementation of a coroutine that is able to generate integer values.

#include <stdio.h>
#include <stdlib.h>
#include <stdbool.h>
#include <ucontext.h>

typedef struct Coroutine Coroutine;
typedef int (*CoroutineFn)(Coroutine*);

struct Coroutine {
    CoroutineFn fn;
    ucontext_t  caller_ctx;
    ucontext_t  callee_ctx;
    int         yield_value;
    bool        finished;
};

Coroutine* new_Coroutine(CoroutineFn fn);
int Coroutine_resume(Coroutine* c);
void Coroutine_suspend(Coroutine* c, int value);
void delete_Coroutine(Coroutine* c);

static
const int default_stack_size = 4096;

static
void Coroutine_entry_point(Coroutine* c) {
    int result = c->fn(c);
    c->finished = true;
    Coroutine_suspend(c, result);
}

Coroutine* new_Coroutine(CoroutineFn fn) {
    Coroutine* c = (Coroutine*)calloc(1, sizeof(Coroutine));
    c->fn = fn;

    getcontext(&c->callee_ctx);
    c->callee_ctx.uc_stack.ss_sp = calloc(1, default_stack_size);
    c->callee_ctx.uc_stack.ss_size = default_stack_size;
    c->callee_ctx.uc_link = 0;
    makecontext(&c->callee_ctx, (void (*)())Coroutine_entry_point, 1, c);

    return c;
}

int Coroutine_resume(Coroutine* c) {
    if (c->finished) return -1;
    swapcontext(&c->caller_ctx, &c->callee_ctx);
    return c->yield_value;
}

void Coroutine_suspend(Coroutine* c, int value) {
    c->yield_value = value;
    swapcontext(&c->callee_ctx, &c->caller_ctx);
}

void delete_Coroutine(Coroutine* c) {
    free(c->callee_ctx.uc_stack.ss_sp);
    free(c);
}

int giveMeTwo(Coroutine* c) {
        printf("[%s] suspend 1\n", __func__);
        Coroutine_suspend(c, 1);
        printf("[%s] after suspend\n", __func__);
        return 2;
}

int main() {
        Coroutine* c = new_Coroutine(giveMeTwo);
        {
                printf("[%s] resume\n", __func__);
                int a = Coroutine_resume(c);       // a == 1
                printf("[%s] after resume a: %d\n", __func__, a);
                printf("[%s] resume\n", __func__);
                int b = Coroutine_resume(c);       // b == 2
                printf("[%s] after resume b: %d\n", __func__, b);
        }
        delete_Coroutine(c);
        return 0;
}

The output of this example is as follows:

[main] resume
[giveMeTwo] suspend 1
[main] after resume a: 1
[main] resume
[giveMeTwo] after suspend
[main] after resume b: 2

So far so good. But we want to use coroutines for providing asynchronous functionality. This means that we need to relate them to threads somehow. But how can we mix this functionality with threads? We can use suspend and resume calls as synchronization points between multiple threads communicating through safe-thread queues. Not bad, right? We will continue our investigation about how async/await works by implementing a basic thread pool on the Part2 of this series. Stay tunned!

I first learned about Zig just a few months ago. However, with the recent surge of languages attempting to replace C/C++, such as Rust, Carbon, Nim, and others, I initially held a degree of skepticism toward yet another contender. It seemed like another attempt to replace the longstanding C/C++. Nevertheless, as I observed Zig gaining traction in the indie game development scene, my curiosity was piqued, prompting me to explore it further. The more I delve into its features, the more I find myself drawn to it. This growing affinity led me to take the step of setting up a development environment on Windows, eager to give Zig a chance and test how it feels to program in this language.

Zig installation

First, we are going to install manually Zig by downloading a release for Windows from the official website:

1. Go to Download ⚡ Zig Programming Language (ziglang.org)

2. Download the release for Windows (e.g.: zig-windows-x86_64-0.12.0-dev.1819+5c1428ea9.zip)

3. Unzip the file in the folder that you want (e.g.: C:\Users\<username>\zig)

After that, we need to add the path of the unzipped folder (i.e.: C:\Users\<username>\zig\zig-windows-x86_64-0.12.0-dev.1819+5c1428ea9) to the PATH environment variable, so that we can run Zig from any location.

Test installation

Now that we have Zig installed, we can check if it works properly. The zig init command sets up the basic structure and configuration files needed for a Zig project. Open a Terminal and type the following commands:

mkdir zig_hello_world
cd zig_hello_world
zig init

This command creates an example program containing a basic main function that prints a message, and a unit test.

const std = @import("std");

pub fn main() !void {
    // Prints to stderr (it's a shortcut based on `std.io.getStdErr()`)
    std.debug.print("All your {s} are belong to us.\n", .{"codebase"});

    // stdout is for the actual output of your application, for example if you
    // are implementing gzip, then only the compressed bytes should be sent to
    // stdout, not any debugging messages.
    const stdout_file = std.io.getStdOut().writer();
    var bw = std.io.bufferedWriter(stdout_file);
    const stdout = bw.writer();

    try stdout.print("Run `zig build test` to run the tests.\n", .{});

    try bw.flush(); // don't forget to flush!
}

test "simple test" {
    var list = std.ArrayList(i32).init(std.testing.allocator);
    defer list.deinit(); // try commenting this out and see if zig detects the memory leak!
    try list.append(42);
    try std.testing.expectEqual(@as(i32, 42), list.pop());
}

We can execute the main function by running the command zig build run:

zig build run
All your codebase are belong to us.
Run `zig build test` to run the tests.

To run the unit test, we just have to run the command zig build test, which will not print anything. But we can check that it works properly by modifying the value to be expected in the unit test.

zig build test

Neovim

Language Server

To install the Zig Language Server we can download the Windows version from the following website:

• Zig Language Server

After downloading it, we have to copy the zls.exe file to the path where we have installed Zig (i.e.: C:\Users\<username>\zig\zig-windows-x86_64-0.12.0-dev.1819+5c1428ea9).

Then, we have to configure the language server in our Neovim configuration file. I prefer to use a local configuration file (i.e.: .nvim.lua), instead of the global Neovim configuration files as I explain in my book about Neovim.

There I also explain how to configure the main actions of lspconfig in Neovim.

require 'lspconfig'.zls.setup{}

We can check that it works properly by opening the main.zig function, and showing the details of a function.

Debugger

LLVM

To debug Zig code we will use LLDB, which is included in the LLVM installation. We can download the LLVM version for Windows (e.g.: LLVM-16.0.6-win64.exe) from the following website:

• LLVM

Python

Once we have LLVM installed, we need to install the Python version required by that LLVM version. We can check that by running LLDB from a terminal. It will show an error dialog complaining of the missing Python DLL (e.g.: python310.dll not found). For the LLVM-16.0.6 version, we need to install the Python version 3.10.x (e.g.: python-3.10.10-amd64.exe). We can download it from the website:

• Python

Finally, we need to define the following environment variable:

LLDB_USE_NATIVE_PDB_READER="yes"

After that, we can check lldb from a terminal and debug the executable coming out from the example program:

PS C:\Users\<username>\zig_hello_world> lldb.exe .\zig-out\bin\zig_hello_world.exe
(lldb) target create ".\\zig-out\\bin\\hello_world.exe"
Current executable set to 'C:\Users\<username>\zig_hello_world\zig-out\bin\zig_hello_world.exe' (x86_64).
(lldb) (lldb) b main
Breakpoint 1: where = zig_hello_world.exe`main + 26 at main.zig:5, address = 0x00000001400013fa
(lldb) r
(lldb) Process 28096 launched: 'C:\Users\<username>\zig_hello_world\zig-out\bin\zig_hello_world.exe' (x86_64)
Process 28096 stopped
* thread #1, stop reason = breakpoint 1.1
    frame #0: 0x00007ff7d46b13fa zig_hello_world.exe`main at main.zig:5
   2
   3    pub fn main() !void {
   4        // Prints to stderr (it's a shortcut based on `std.io.getStdErr()`)
-> 5        std.debug.print("All your {s} are belong to us.\n", .{"codebase"});
   6
   7        // stdout is for the actual output of your application, for example if you
   8        // are implementing gzip, then only the compressed bytes should be sent to
(lldb)

As Neovim supports the Debug Adapter Protocol (DAP), we can debug directly from Neovim. To configure Zig debugging in Neovim we can use the nvim-dap plugin. The required configuration for Zig that we need to add to our local Neovim configuration files (i.e.: .nvim.lua) is the following:

local dap = require('dap')
dap.adapters.lldb = {
  type = 'executable',
  command = 'C:\\Program Files\\LLVM\\bin\\lldb-vscode.exe', -- adjust as needed, must be absolute path
  name = 'lldb'
}

dap.configurations.zig = {
  {
    name = 'Launch',
    type = 'lldb',
    request = 'launch',
    program = '${workspaceFolder}/zig-out/bin/zig_hello_world.exe',
    cwd = '${workspaceFolder}',
    stopOnEntry = false,
    args = {},
  },
}

Then, you can start a debugging session from Neovim, and move around the source code while inspecting the execution flow of the program.

Bonus: VScode setup

If you don't feel comfortable working with a text editor like Neovim, I'm going to give you the instructions to configure VScode to work with Zig as VScode is one of the most commonly used IDEs nowadays. To add support for Zig, you just need to install the Zig extension from the VScode marketplace.

Now that we have the Zig extension installed, we can open the folder of the example project that we created previously (i.e.: zig_hello_world) from VScode. When we open the main.zig file, VScode will ask us to install Zig and the Zig Language Server. It will install them under the local VScode directory (e.g.: c:\Users\<username>\AppData\Roaming\Code\User\globalStorage\ziglang.vscode-zig\zig_install\zig.exe).

Once we have the Zig Language Server installed, we can test it by opening the main.zig file and placing the cursor over one of the functions (e.g.: std.debug.print). It will pop up a dialog showing details about the selected function.

Lastly, we can configure the debugger for Zig by creating the following launch.json file:

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Debug",
            "type": "cppvsdbg",
            "request": "launch",
            "program": "${workspaceFolder}/zig-out/bin/zig_hello_world.exe",
            "cwd": "${workspaceFolder}",
        },
    ]
}

After that, if we place a breakpoint on one of the lines of the main.zig file and press F5 to start a debugging session, we will see how the program execution stops at the breakpoint:

And that's all. I hope this post helps you set up a development environment for Zig so that you can give this language a try. It is promising, especially for game development, where I've seen several indie games and engines already implemented using Zig. In the case of Linux, the setup is pretty similar to this one. You just need to pick the versions corresponding to Linux for each product, but the steps would be pretty much the same.

See you in the next post!

1. Purity and Immutability:

   • Haskell is a purely functional language, which means that functions in Haskell are referentially transparent and side-effect-free. This purity ensures that the result of a function depends only on its inputs, promoting clarity and reasoning about code.

   • Immutability is enforced by default in Haskell. Once a value is bound to a name, it cannot be changed. This immutability simplifies reasoning about program state and helps avoid bugs related to mutable data.

2. Lazy Evaluation:

   • Haskell employs lazy evaluation, where expressions are not evaluated until their values are actually needed. This enables the creation of infinite data structures and allows for more modular and composable code.

   • Lazy evaluation helps avoid unnecessary computations, leading to potentially more efficient and expressive programs.

3. Type System and Type Inference:

   • Haskell has a strong, static type system that is based on Hindley-Milner type inference. The type system helps catch errors at compile time, providing a high level of safety.

   • Haskell's type system also supports polymorphism, type classes, and algebraic data types, allowing for expressive and concise code.

4. Higher-Order Functions and First-Class Functions:

   • Haskell treats functions as first-class citizens, meaning functions can be passed as arguments to other functions, returned as values, and stored in data structures. This enables the use of higher-order functions, facilitating functional programming patterns.

   • Higher-order functions in Haskell promote code that is concise, modular, and expressive.

5. Pattern Matching and Algebraic Data Types:

   • Pattern matching in Haskell allows for concise and readable code when working with algebraic data types. This feature makes it easy to destructure and process complex data structures.

   • Algebraic data types, including sum types (enums) and product types (structs), provide a powerful mechanism for modeling data in a clear and extensible way.

6. Monads and Monadic I/O:

   • Haskell introduced the concept of monads to handle side effects in a pure functional setting. Monads allow sequencing of computations while maintaining referential transparency.

   • The use of monads in Haskell enables a clean and principled approach to handling input/output operations in a functional language.

7. Expressive Type Classes:

   • Haskell's type classes allow ad-hoc polymorphism, enabling the definition of common operations for various types. This leads to more generic and reusable code.

   • The use of type classes in Haskell is a powerful mechanism for creating abstract interfaces and promoting code that is both expressive and modular.

The combination of these features makes Haskell a paradigmatic functional programming language. It serves as a reference for functional programming principles and has influenced the development of other functional languages. Haskell's design choices encourage developers to adopt a functional programming mindset, leading to code that is often concise, elegant, and maintainable.

Here's an example that demonstrates the principles of recursion, higher-order functions, immutability, and referential transparency:

-- Example of recursion
factorial :: Integer -> Integer
factorial 0 = 1
factorial n = n * factorial (n - 1)

-- Example of a higher-order function (map)
multiplyByTwo :: [Integer] -> [Integer]
multiplyByTwo = map (*2)

-- Example of immutability
originalList :: [Integer]
originalList = [1, 2, 3, 4, 5]

-- Applying a higher-order function to the original list
doubledList :: [Integer]
doubledList = multiplyByTwo originalList

-- Referentially transparent function
addTwo :: Integer -> Integer
addTwo x = x + 2

main :: IO ()
main = do
  -- Example of immutability
  let originalValue = 5
  putStrLn $ "Original value: " ++ show originalValue

  -- Referentially transparent function
  let doubledValue = addTwo originalValue
  putStrLn $ "Doubled value: " ++ show doubledValue

  -- Example of recursion
  let result = factorial 5
  putStrLn $ "Factorial of 5: " ++ show result

  -- Example of a higher-order function (map)
  putStrLn "Original list: " ++ show originalList
  putStrLn $ "Doubled list: " ++ show doubledList

In this Haskell example:

• Recursion: The factorial function calculates the factorial of a number using recursion.

• Higher-order function (map): The multiplyByTwo function is defined using the map higher-order function, which multiplies each element of a list by 2.

• Immutability: In Haskell, variables are immutable. The originalList and doubledList values are bound to names, and once assigned, their values do not change.

• Referential Transparency: The addTwo function is referentially transparent. Given the same input, it will always produce the same output.

Note: Haskell uses a lazy evaluation strategy, which means that values are only computed when needed. This can have a profound impact on how functions are evaluated and how recursion is handled.

In C, achieving full functional programming principles such as immutability and referential transparency can be challenging due to the mutable nature of variables. However, you can still demonstrate functional programming concepts like recursion and higher-order functions. Here's a simple example that showcases recursion and a basic form of a higher-order function in C:

#include <stdio.h>

// Example of a higher-order function
typedef int (*UnaryOperation)(int);

// Higher-order function that applies a unary operation function to each element of an array
void map(int* array, size_t size, UnaryOperation op) {
    for (size_t i = 0; i < size; ++i) {
        array[i] = op(array[i]);
    }
}

// Example of a recursive function
int factorial(int n) {
    if (n == 0 || n == 1) {
        return 1;
    } else {
        return n * factorial(n - 1);
    }
}

int main() {
    // Example of immutability (const) --------------------------------
    const int originalValue = 5;
    printf("Original value: %d\n", originalValue);

    // Example of referential transparency (pure function) ------------
    int doubledValue = originalValue * 2;
    printf("Doubled value: %d\n", doubledValue);

    // Example of recursion -------------------------------------------
    int result = factorial(5);
    printf("Factorial of 5: %d\n", result);

    // Example of a higher-order function (map) -----------------------
    int numbers[] = {1, 2, 3, 4, 5};
    size_t arraySize = sizeof(numbers) / sizeof(numbers[0]);

    // Define a unary operation function (double)
    int doubleOperation(int x) {
        return x * 2;
    }

    // Apply the double operation using the map function
    map(numbers, arraySize, doubleOperation);

    // Print the modified array
    printf("Doubled array: ");
    for (size_t i = 0; i < arraySize; ++i) {
        printf("%d ", numbers[i]);
    }
    printf("\n");

    return 0;
}

In this example:

• Immutability: The const keyword is used to declare a constant (originalValue), demonstrating a form of immutability. However, note that C does not enforce immutability in the same way as functional languages.

• Referential Transparency: The expression originalValue * 2 is referentially transparent, meaning it will always produce the same result for the same inputs.

• Recursion: The factorial function is a recursive function that calculates the factorial of a number.

• Higher-order function (map): The map function takes an array, its size, and a unary operation function (op). It applies the unary operation to each element of the array, demonstrating a basic form of a higher-order function.

While this example incorporates some functional programming principles, it's important to note that C is not a purely functional language, and achieving true immutability and referential transparency might require a different programming paradigm.

When you perform a search using Windows Index Search, the system does not have to scan every file and folder on your computer every time you search. Instead, it refers to the index, which is a database containing information about the files' names, locations, and contents. This significantly reduces the time it takes to find and retrieve search results, especially when dealing with a large number of files.

Key Features and Benefits of Windows Index Search:

• Fast Search Results: With the index pre-built, Windows Search can quickly return search results, making it easier to find the files and documents you need.

• Support for Various File Types: Windows Index Search supports various file types, including documents, images, music, videos, and more.

• Partial and Phrasal Search: It allows partial word matches and supports searching for phrases, making it more flexible and powerful.

• Outlook Integration: The Windows Search index can also be used by Microsoft Outlook to speed up email searches.

• Customization: Users can customize which folders and locations are indexed to include or exclude specific locations based on their preferences.

• Advanced Search Operators: Windows Search supports advanced search operators, such as AND, OR, and NOT, to refine search queries further.

• Real-Time Indexing: As files and folders change or new ones are added, the index is updated in real-time to ensure the search results remain up-to-date.

How to set the index sources

The Windows Index Search can look for text in multiple locations. You can specify the folders that you want to add to the database. To edit the directories included in the index you have to do the following steps:

• Win + S: press the "Windows key" plus the "key s" to open the Windows Search panel

• Type Indexing Options to open the corresponding dialog

• Add the folders that you want to include in the index

Once you have configured the folders to be included in the database, the Index Engine will start indexing all the documents included in those locations. Once it finishes, it will show a message saying that the indexing process is complete. From then on, you can do full-text searches on the specified directories.

How to do full-text searches on the Windows Explorer

The Windows Explorer provides you with a search text box that allows you to look for files or directories. To do a full-text search using Windows Explorer you have to do the following steps:

• Open the Windows Explorer window

• Navigate to the folder where you want to perform the search

• Type in the search text box the term that you are looking for, specifying the content filter (e.g.: content: "design analysis")

As the contents of all the documents contained in that folder have been already indexed by the Windows Search Index, you will see how the results of documents containing that term (if any) start to emerge immediately.

How to do full-text searches on PowerShell

We are now going to see how we can use that functionality from a PowerShell script. It will use the APIs exposed by Windows to have access to the Windows Search Index database.

function SearchIndex {
<#
.PARAMETER Path
Absoloute or relative path. Has to be in the Search Index for results to be presented.
.PARAMETER Pattern
File name or pattern to search for. Defaults to *.*. Aliased to Filter to ergonomically match Get-ChildItem.
.PARAMETER Text
Free text to search for in the files defined by the pattern.
.PARAMETER Recurse
Add the parameter to perform a recursive search. Default is false.
.PARAMETER AsFSInfo
Add the parameter to return System.IO.FileSystemInfo objects instead of String objects.
.SYNOPSIS
Uses the Windows Search index to search for files.
.DESCRIPTION
Uses the Windows Search index to search for files. SQL Syntax documented at https://msdn.microsoft.com/en-us/library/windows/desktop/bb231256(v=vs.85).aspx Based on https://blogs.msdn.microsoft.com/mediaandmicrocode/2008/07/13/microcode-windows-powershell-windows-desktop-search-problem-solving/ 
.OUTPUTS
By default one string per file found with full path.
If the AsFSInfo switch is set, one System.IO.FileSystemInfo object per file found is returned.
#>
    [CmdletBinding()]
    param (  
        [string]$Path = $PWD,
        [string]$Pattern = "*.*",
        [string]$Text = $null,
        [switch]$Recurse = $true,
        [switch]$AsFSInfo = $false
    )

    $Path = (Resolve-Path -Path $Path).Path

    $Pattern = $Pattern -replace "\*", "%"
    $Path = $Path.Replace('\','/')

    if ((Test-Path -Path Variable:fsSearchCon) -eq $false)
    {
        $global:fsSearchCon = New-Object -ComObject ADODB.Connection
        $global:fsSearchRs = New-Object -ComObject ADODB.Recordset
    }

    $fsSearchCon.Open("Provider=Search.CollatorDSO;Extended Properties='Application=Windows';")

    [string]$queryString = "SELECT System.ItemPathDisplay FROM SYSTEMINDEX WHERE System.FileName LIKE '" + $Pattern + "' "
    if ([System.String]::IsNullOrEmpty($Text) -eq $false){
        $queryString += "AND System.Search.Contents='" + $Text + "' "
    }

    if ($Recurse){
        $queryString += "AND SCOPE='file:" + $Path + "' ORDER BY System.ItemPathDisplay "
    }
    else {
        $queryString += "AND DIRECTORY='file:" + $Path + "' ORDER BY System.ItemPathDisplay "
    }

    $fsSearchRs.Open($queryString, $fsSearchCon)

    While(-Not $fsSearchRs.EOF){
        if ($AsFSInfo){
            [System.IO.FileSystemInfo]$(Get-Item -LiteralPath ($fsSearchRs.Fields.Item("System.ItemPathDisplay").Value) -Force)
        }
        else {
            $fsSearchRs.Fields.Item("System.ItemPathDisplay").Value
        }
        $fsSearchRs.MoveNext()
    }
    $fsSearchRs.Close()
    $fsSearchCon.Close()
}

SearchIndex @args

You can copy this script to a file named SearchIndex.ps1, which you can edit to adapt any part that you want. And if Neovim is your editor of choice and you’d like a clear, practical introduction to it, I cover that in detail in my book.

As long as this file is contained in one of the directories defined in the Path environment variable, you will be able to run it from any location on the PowerShell terminal. The script supports several arguments. It looks recursively in the folders contained in the current directory by default. It looks over all kinds of documents by default too. But the argument -Pattern allows you to filter out the results just to the files of the specified extension.

PS C:\>SearchIndex.ps1 -Text 'design analysis' -Pattern '*.pdf'

How to do full-text searches from the Windows run dialog

Windows has a very useful feature that allows you to quickly execute a command. To do so, you just have to press the "Windows key" plus the "key r", and you will see a pop-up dialog called Run. You can type there any program that is available on the Path environment variable, and execute it.

In our case, we want to trigger the execution of a full-text search over any document contained in the Windows Search Index database from that Run dialog. That way, we would just need to type "Win + r" and the search command together with the search term to do a full-text search over all the target documents on our machine. Not bad, right?

The problem is that this Run dialog can only run batch files, not PowerShell scripts. Therefore, to overcome that limitation, we have to create a wrapper batch file that will call our PowerShell script. And add the location of this wrapper batch file to our Path environment variable.

We are going to create a search-text.bat file, that will allow us to perform a full-text search over any pdf file contained in a specific directory, which belongs to the Windows Search Index database.

@echo off

Rem Description: Search text in Windows Search Index pdf files
Rem Usage: search-text.bat {strings}
Rem Example: search-text.bat Seamless

set cmd="SearchIndex.ps1 -Path 'C:\Users\alem\Documents' -Pattern *.pdf -Text '%*' -Recurse"
echo %cmd%

PowerShell -ExecutionPolicy Bypass -Command %cmd%
Pause

Once we have done that, we can use the Run dialog to execute the SearchIndex script to search for all the PDF documents from that directory that have been indexed in the Windows Search Index database.

And that's all. With this utility, you can search for any text at lightning speed on your computer. It is so fast, that it almost feels like you have a Google Search Engine on your local laptop.

We are going to implement a mechanism that allows the server to send messages to all of its clients at the same time. If we wanted to send messages to a specific client, it would just be a matter of keeping track of an ID for each client associated with their corresponding connection. And use that mapping to send the messages to a specific client.

Client connection

We will start by modeling each client connection with a reference to the server, a reference to the actual network connection and a buffered channel of 256 strings. This channel will be used to store responses as a buffer before sending them back to the client. That way, we can detach the process of sending messages back to the clients from the process of handling their requests.

type connection struct {
    s         *server
    conn      net.Conn
    responses chan string
}

func newConnection(s *server, conn net.Conn) *connection {
    var c connection
    c.s = s
    c.conn = conn
    c.responses = make(chan string, 256)
    return &c
}

The client connection processing will be composed of two goroutines. One of them will be in charge of reading requests from the client, and sending them to the server. Each client request will be a text line.

func (c *connection) readConnection() {
    defer c.s.removeConnection(c)

    buf := bufio.NewReader(c.conn)

    for {
        data, err := buf.ReadString('\n')
        if err != nil {
            break
        }
        c.s.submitRequest(c, data)
    }
}

The other goroutine will iterate over the responses channel, and every time a new response is added to the channel, this goroutine will send it to the client through the network connection. It will keep iterating over the responses channel, so in case there are no more responses ready to be sent back to the client, this goroutine will wait until a new one gets to the channel.

func (c *connection) writeConnection() {
    for message := range c.responses {
        c.conn.Write([]byte(message))
    }
}

Server

The server object will keep the client connections in a sync.Map. This is needed because the list of connections can be accessed from different goroutines at the same time: the ones that add and remove references to the map, and the ones that broadcast responses to the clients. Similarly to the client connection, the server will contain a channel of 256 requests. That way, it can buffer the requests coming from the clients without the need to block any of them. It will also contain a handle function that allows the user to specify a function to process the requests.

type server struct {
    connections sync.Map
    handle      handleFn
    requests    chan *request
}

func newServer(handle handleFn) *server {
    var s server
    s.handle = handle
    s.requests = make(chan *request, 256)
    return &s
}

The server processing will be composed of a goroutine that accepts new network connections and stores them in the sync.Map.

func (s *server) serve(network, address string) {
    l, err := net.Listen(network, address)
    if err != nil {
        log.Fatal(err)
    }
    defer l.Close()

    for {
        c, err := l.Accept()
        if err != nil {
            log.Fatal(err)
        }
        connection := newConnection(s, c)
        s.connections.Store(c.RemoteAddr().String(), connection)
        connection.start()

        log.Printf("New connection %s", c.RemoteAddr().String())
    }
}

It will also contain several goroutines that handle client requests. This number of goroutines is the same as the number of CPUs available in the server machine. Each one of these worker goroutines will execute the handle function of the server to process the requests.

func (s *server) start(network, address string) {
    go s.serve(network, address)

    numCpu := runtime.NumCPU()
    for i := 0; i < numCpu; i++ {
        go s.worker()
    }
}

func (s *server) worker() {
    for req := range s.requests {
        s.handle(req.c, req.data)
    }
}

The server will also expose a method to broadcast messages to the clients. This method will iterate over the client connections sync.Map, and will send the message to each client. That message will then be queued in the requests channel of the client connection.

func (s *server) broadcast(message string) {
    s.connections.Range(func(k, v interface{}) bool {
        c := v.(*connection)
        c.send(message)
        return true
    })
}

Complete code

After going over the main parts of the example, here you have the complete code of the server.

package main

import (
    "bufio"
    "fmt"
    "log"
    "net"
    "os"
    "runtime"
    "sync"
)

type connection struct {
    s         *server
    conn      net.Conn
    responses chan string
}

func newConnection(s *server, conn net.Conn) *connection {
    var c connection
    c.s = s
    c.conn = conn
    c.responses = make(chan string, 256)
    return &c
}

func (c *connection) start() {
    go c.readConnection()
    go c.writeConnection()
}

func (c *connection) stop() {
    close(c.responses)
    c.conn.Close()
}

func (c *connection) send(data string) {
    c.responses <- data
}

func (c *connection) readConnection() {
    defer c.s.removeConnection(c)

    buf := bufio.NewReader(c.conn)

    for {
        data, err := buf.ReadString('\n')
        if err != nil {
            break
        }
        c.s.submitRequest(c, data)
    }
}

func (c *connection) writeConnection() {
    for message := range c.responses {
        c.conn.Write([]byte(message))
    }
}

type request struct {
    c    *connection
    data string
}

type handleFn func(*connection, string)

type server struct {
    connections sync.Map
    handle      handleFn
    requests    chan *request
}

func newServer(handle handleFn) *server {
    var s server
    s.handle = handle
    s.requests = make(chan *request, 256)
    return &s
}

func (s *server) submitRequest(c *connection, data string) {
    req := request{c, data}
    s.requests <- &req
}

func (s *server) start(network, address string) {
    go s.serve(network, address)

    numCpu := runtime.NumCPU()
    for i := 0; i < numCpu; i++ {
        go s.worker()
    }
}

func (s *server) worker() {
    for req := range s.requests {
        s.handle(req.c, req.data)
    }
}

func (s *server) serve(network, address string) {
    l, err := net.Listen(network, address)
    if err != nil {
        log.Fatal(err)
    }
    defer l.Close()

    for {
        c, err := l.Accept()
        if err != nil {
            log.Fatal(err)
        }
        connection := newConnection(s, c)
        s.connections.Store(c.RemoteAddr().String(), connection)
        connection.start()

        log.Printf("New connection %s", c.RemoteAddr().String())
    }
}

func (s *server) removeConnection(c *connection) {
    c.stop()
    s.connections.Delete(c.conn.RemoteAddr().String())

    log.Printf("Closed connection %s", c.conn.RemoteAddr().String())
}

func (s *server) broadcast(message string) {
    s.connections.Range(func(k, v interface{}) bool {
        c := v.(*connection)
        c.send(message)
        return true
    })
}

func main() {
    arguments := os.Args
    if len(arguments) != 3 {
        log.Fatal("Usage: server <network> <address>")
    }

    network := arguments[1]
    address := ":" + arguments[2]

    s := newServer(func(c *connection, request string) {
        c.send(request)
    })
    s.start(network, address)

    fmt.Print("Enter message: \n")
    reader := bufio.NewReader(os.Stdin)
    for {
        text, _ := reader.ReadString('\n')
        s.broadcast(text)
    }
}

Testing the example

We can test the server by running it on a terminal specifying the kind of network for the sockets, which can be TCP or UNIX.

TCP sockets example

We start the sever in one terminal, and test it from another terminal using netcat (nc).

# TCP Server
./server tcp <port>

# TCP Client
nc localhost <port>

As soon as we type any line on the netcat terminal, we will see an echo message coming back from the server.

>nc localhost 8080
test1
test1
test2
test2

From the server terminal we can also broadcast messages to the connected clients. If type a line on the server terminal, we will see it appearing on the client terminal.

>./main tcp 8080
Enter message:
2023/08/11 12:32:50 New connection 127.0.0.1:34498
message1

>nc localhost 8080
test1
test1
test2
test2
message1

UNIX Domain Sockets example

For the case of UNIX sockets it is pretty similar. It is just a matter of starting the server in one terminal using the unix as network type.

# UNIX Server
./server unix <socket>
# example
./server unix test

# UNIX Client
nc -U <socket>
# example:
nc -U :test

Conclusion

In this article, we have seen an example of how to code a simple full duplex server that can be used with TCP or UNIX Domain Sockets. Unlike half-duplex connections, where data can only flow in one direction at a time, a full duplex connection allows for seamless and real-time exchange of information in both directions concurrently. This capability enhances the efficiency and speed of communication, making it ideal for applications such as video conferencing, online gaming, and data-intensive tasks.

Understanding TCP Concurrent Servers

A TCP server is a fundamental component of network programming that listens for incoming connections from clients and responds to their requests. It operates on the Transmission Control Protocol (TCP), a reliable and connection-oriented protocol that guarantees the delivery of data packets in the order they were sent. The TCP server binds to a specific IP address and port number, waiting for clients to establish connections. Once a connection is established, the server can exchange data with the client bidirectionally, allowing for real-time communication. TCP servers are widely used in various applications, such as web servers, chat applications, file transfer systems, and more, providing a robust foundation for building scalable and efficient network services.

A concurrent TCP server is an advanced networking application that combines the power of concurrent programming with the capabilities of a TCP server. Unlike traditional TCP servers, a concurrent TCP server can handle multiple client connections concurrently, allowing it to process multiple requests simultaneously without blocking other clients. This is achieved by utilizing concurrency primitives like Goroutines in Go or Threads in other programming languages. By employing parallelism, a concurrent TCP server can efficiently serve a large number of clients, enhancing its responsiveness and scalability. This makes it a preferred choice for high-traffic and real-time applications, such as chat servers, multiplayer games, or any system requiring simultaneous interactions with multiple clients. The use of concurrency in TCP servers significantly improves performance and resource utilization, making it a crucial aspect of modern network programming.

The Power of Goroutines

Goroutines are a key feature of the Go programming language, and they are the building blocks of concurrent programming in Go. A Goroutine is a lightweight, independent execution unit that allows developers to execute functions concurrently without the complexities of traditional threads. Goroutines are highly efficient and can be created and destroyed with minimal overhead. They enable concurrent execution of tasks, allowing multiple operations to run simultaneously without the need for explicit thread management. Due to their lightweight nature, Goroutines make it easy to scale applications and handle a large number of concurrent operations efficiently. With Goroutines, Go programmers can write concurrent code with ease, enabling the creation of highly responsive and efficient applications. They are an essential feature for building highly scalable and responsive servers.

Implementing the TCP Concurrent Server

Let's create a simple TCP server that handles multiple client connections concurrently using Goroutines. It will create a goroutine for each new incoming TCP connection. That goroutine will execute an endless loop, where it just basically returns an echo of the received message to the client. Unless the message is the string "STOP", in which case, the goroutine for that connection will end, and the server will stop serving that client.

package main

import (
    "bufio"
    "log"
    "net"
    "strings"
)

func main() {
    l, err := net.Listen("tcp", ":8080")
    if err != nil {
        log.Fatal(err)
    }
    defer l.Close()

    for {
        c, err := l.Accept()
        if err != nil {
            log.Fatal(err)
        }
        go handleConnection(c)
    }
}

func handleConnection(c net.Conn) {
    defer c.Close()

    connReader := bufio.NewReader(c)

    log.Printf("Serving %s\n", c.RemoteAddr().String())
    for {
        data, err := connReader.ReadString('\n')
        if err != nil {
            log.Println(err)
            break
        }
        request := strings.TrimSpace(string(data))

        log.Printf("Request from %s: %s\n", c.RemoteAddr().String(), request)
        if request == "STOP" {
            break
        }
        c.Write([]byte(data))
    }
    log.Printf("Stop serving %s\n", c.RemoteAddr().String())
}

Once we have our server ready, we can test it using netcat in Linux. In order to do so, we just need to start the server in one terminal. And run the netcat command in another terminal using the address localhost and port 8080 to connect to the server. Then, we can start typing strings, and every time we press Enter we will see an echo of that message received from the server.

>nc localhost 8080
test1
test1
test2
test2

Meanwhile, in the server terminal we will see the messages received from the client. We can connect multiple clients from multiple terminal running the netcat in each one of them to check how the server is able to handle all their requests.

>./main
2023/08/02 10:13:35 Serving 127.0.0.1:58740
2023/08/02 10:13:38 Request from 127.0.0.1:58740: test1
2023/08/02 10:13:38 Request from 127.0.0.1:58740: test2
2023/08/02 10:13:51 Serving 127.0.0.1:58742
2023/08/02 10:13:57 Request from 127.0.0.1:58742: test3
2023/08/02 10:13:58 Request from 127.0.0.1:58742: test4
2023/08/02 10:14:01 Request from 127.0.0.1:58742: STOP
2023/08/02 10:14:01 Stop serving 127.0.0.1:58742
2023/08/02 10:14:04 Request from 127.0.0.1:58740: test5
2023/08/02 10:14:05 Request from 127.0.0.1:58740: test6

Conclusion

In this article, we explored the process of building a TCP concurrent server in Go, utilizing Goroutines to handle multiple client connections simultaneously. By leveraging the power of concurrency through Goroutines, our server can efficiently handle a large number of clients, leading to improved performance and responsiveness. With the knowledge gained from this example, you can further extend the server to incorporate your own business logic and create powerful network applications. Go's built-in concurrency features make it an ideal choice for developing high-performance servers, and mastering the art of concurrent programming opens up a world of possibilities for creating scalable and efficient applications.

Understanding JSON-RPC

JSON-RPC is a simple yet effective protocol for remote procedure calls over HTTP or other transport layers. It allows clients to invoke methods on a server and receive responses in a structured JSON format. The protocol is lightweight, language-agnostic, and easy to implement, making it an excellent choice for web applications.

We are going to use the version JSON-RPC 2.0 which follows this format:

     {"jsonrpc":"2.0","method":"HelloService.Hello","params":{"msg":"John"},"id":0}

Using WebSockets in Go

WebSockets enable bidirectional communication between clients and servers, allowing real-time data transfer. In Go, the standard library provides support for WebSockets through the golang.org/x/net/websocket package. There are other more advanced packages for WebSockets, like the gorilla/websocket package, but I'll stick to the standard one for this example, as it is simpler.

Implementing a JSON-RPC Server using WebSockets in Go

For this example, let's create a simple JSON-RPC server in Go that communicates with clients over WebSockets.

Step 1: Define the service

Make sure you have Go installed and set up a Go workspace. Create a new directory for the project, and inside it, create the Go file named internal/service/hello.go containing the definition of the service that the server is going to expose through a WebSockets interface.

package service

import (
    "log"
)

type HelloService struct{}

type HelloRequest struct {
    Name string
}

type HelloResponse struct {
    Greeting string `json:"string"`
}

func (s *HelloService) Hello(req *HelloRequest, res *HelloResponse) error {
    log.Println("Execute method: HelloService.Hello()")
    res.Greeting = "Hello: " + req.Name
    return nil
}

Step 2: Implementing the JSON-RPC Server

Then we create the file cmd/server/main.go file to set up a WebSocket server and handle incoming JSON-RPC requests.

package main

import (
    "bytes"
    "golang.org/x/net/websocket"
    "io"
    "io/ioutil"
    "log"
    "net/http"
    "net/rpc"
    "net/rpc/jsonrpc"

    "go-websocket-jsonrpc/internal/service"
)

func wsHandleRequest(ws *websocket.Conn) {
    for {
        var req []byte
        err := websocket.Message.Receive(ws, &req)
        if err != nil {
            log.Println("ReadMessage:", err)
            return
        }

        log.Println("ServeRequest...")
        var res bytes.Buffer
        err = rpc.ServeRequest(jsonrpc.NewServerCodec(struct {
            io.ReadCloser
            io.Writer
        }{
            ioutil.NopCloser(bytes.NewReader(req)),
            &res,
        }))
        if err != nil {
            log.Println("ServeRequest:", err)
            return
        }

        err = websocket.Message.Send(ws, res.Bytes())
        if err != nil {
            log.Println("WriteMessage:", err)
            return
        }
    }
}

func main() {
    log.Println("Starting http server")

    rpc.Register(&service.HelloService{})

    http.Handle("/ws", websocket.Handler(wsHandleRequest))
    http.ListenAndServe("localhost:8080", nil)
}

Step 3: Implementing the JSON-RPC Client

Finally, we create the file cmd/client/main.go to implement a Go WebSocket client that reads strings from the command line, sends them to the server, and prints out the responses.

package main

import (
    "bufio"
    "golang.org/x/net/websocket"
    "log"
    "net/rpc"
    "net/rpc/jsonrpc"
    "os"

    "go-websocket-jsonrpc/internal/service"
)

func sayHello(c *rpc.Client, name string) {
    req := service.HelloRequest{Name: name}
    var res service.HelloResponse

    err := c.Call("HelloService.Hello", req, &res)
    if err != nil {
        log.Fatal("error:", err)
    }
    log.Printf("Response: %s", res.Greeting)
}

func main() {
    ws, err := websocket.Dial("ws://localhost:8080/ws", "", "http://localhost/")
    if err != nil {
        log.Fatal(err)
    }
    defer ws.Close()

    c := jsonrpc.NewClient(ws)

    reader := bufio.NewReader(os.Stdin)
    for {
        text, _ := reader.ReadString('\n')
        sayHello(c, text)
    }
}

Step 4: Test the example

Now that we have the server and client ready, we start the server in one terminal, and the client on another one. Every time we type a string in the client terminal and press Enter, the client will send that string to the server, and we will see right away the response from the server.

>./bin/client
./bin/client
john
2023/08/02 05:27:57 Response: Hello: john
lisa
2023/08/02 05:28:07 Response: Hello: lisa
tom
2023/08/02 05:28:08 Response: Hello: tom

Conclusion

WebSockets and JSON-RPC can work harmoniously together, providing an efficient and real-time communication mechanism for web applications. In this article, we explored how to implement a JSON-RPC server using WebSockets in Go. You can build upon this example to create more sophisticated applications with real-time updates and bidirectional data flow. Whether it's real-time chats, live notifications, or dynamic dashboards, the combination of WebSockets and JSON-RPC in Go is a powerful solution for modern web development.

What is gRPC?

gRPC is an open-source high-performance Remote Procedure Call (RPC) framework developed by Google. It enables seamless communication between services running on different platforms and languages. gRPC is based on HTTP/2, which provides multiplexing, stream prioritization, and header compression, leading to faster and more efficient communication compared to REST.

Key Features and Advantages of gRPC

Protocol Buffers (Protobuf): gRPC uses Protocol Buffers as the default data serialization mechanism. Protobuf offers efficient binary serialization, reducing payload size and facilitating faster data transmission.

Bidirectional Streaming: Unlike REST, where the client sends a request and waits for a response, gRPC supports bidirectional streaming. This means both the client and server can send multiple messages asynchronously over a single connection, ideal for real-time applications.

Code Generation: gRPC generates client and server code in various languages (including Go, Java, Python, and more) from a single Protobuf definition. This reduces the manual effort and minimizes the risk of errors when integrating services.

Strong Typing: Protobuf enforces strong typing for message structures, ensuring the consistency and integrity of data exchanged between services.

Unary and Streaming Calls: gRPC supports unary calls (single request and response) as well as streaming calls (continuous stream of requests or responses). This flexibility caters to various communication scenarios.

Practical Example: Building a Greet Service in Go

Let's create a simple gRPC server and client to demonstrate the power of gRPC using Go. They will show an example of a single request and response. As well as an example of a continuous stream from the server to the client.

Install Protobuf Compiler

First, we need to install the protobuf compiler. We must ensure that the version is 3+.

    sudo apt install -y protobuf-compiler

Then, we have to install the Go plugins for the protobuf compiler.

    go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
    go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest

Define the Protobuf Messages

Create a file named idl/service/service.proto and define the Greeter service and message structure using Protocol Buffers.

syntax = "proto3";

option go_package = "internal/service";

package service;

// The greeting service definition.
service Greeter {
  // Sends a greeting
  rpc SayHello (HelloRequest) returns (HelloReply) {}

  // Subscription to receive a continuous stream of notifications
  rpc Subscribe (Subscription) returns (stream Notification) {}
}

// Messages definition

message HelloRequest {
  string name = 1;
}

message HelloReply {
  string message = 1;
}

message Subscription {
    string address = 1;
}

message Notification {
    string message = 1;
}

Generate Go Code from Protobuf

Run the protobuf compiler to compile the idl/service/service.proto file and generate the internal/service/service.pb.go and the internal/service/service_grpc.pb.go files containing the Go implementation.

    protoc --proto_path=idl --go_out=internal --go_opt=paths=source_relative --go-grpc_out=internal --go-grpc_opt=paths=source_relative service/service.proto

Implement the server in Go

Create a Go file named cmd/server/main.go and implement the server.

package main

import (
    "context"
    "flag"
    "log"
    "net"
    "time"

    "google.golang.org/grpc"

    "go-grpc/internal/service"
)

type server struct {
    service.UnimplementedGreeterServer
}

func (s *server) SayHello(ctx context.Context, in *service.HelloRequest) (*service.HelloReply, error) {
    log.Printf("Received: %v", in.GetName())
    return &service.HelloReply{Message: "Hello " + in.GetName()}, nil
}

func (s *server) Subscribe(c *service.Subscription, stream service.Greeter_SubscribeServer) error {
    for {
        stream.Send(&service.Notification{Message: "notification"})
        time.Sleep(time.Second * 3)
    }
    return nil
}

func main() {
    flag.Parse()
    lis, err := net.Listen("tcp", ":50051")
    if err != nil {
        log.Fatalf("failed to listen: %v", err)
    }
    s := grpc.NewServer()
    service.RegisterGreeterServer(s, &server{})
    log.Printf("server listening at %v", lis.Addr())
    if err := s.Serve(lis); err != nil {
        log.Fatalf("failed to serve: %v", err)
    }
}

Now that we have the server implementation ready, we can compile it.

    go build -o bin/server cmd/server/main.go

Implement the client in Go

Create a Go file named cmd/client/main.go and implement the client.

package main

import (
    "bufio"
    "context"
    "io"
    "log"
    "os"
    "time"

    "google.golang.org/grpc"
    "google.golang.org/grpc/credentials/insecure"

    "go-grpc/internal/service"
)

func sayHello(c service.GreeterClient, name string) {
    ctx, cancel := context.WithTimeout(context.Background(), time.Second)
    defer cancel()
    r, err := c.SayHello(ctx, &service.HelloRequest{Name: name})
    if err != nil {
        log.Fatalf("could not greet: %v", err)
    }
    log.Printf("Greeting: %s", r.GetMessage())
}

func subscribe(c service.GreeterClient) {
    ctx := context.Background()
    stream, err := c.Subscribe(ctx, &service.Subscription{Address: "client"})
    if err != nil {
        log.Fatalf("could not subscribe: %v", err)
    }
    for {
        notification, err := stream.Recv()
        if err == io.EOF {
            break
        }
        if err != nil {
            log.Fatalf("%v.Subscribe(_) = _, %v", c, err)
        }
        log.Println(notification)
    }
}

func main() {
    conn, err := grpc.Dial(":50051", grpc.WithTransportCredentials(insecure.NewCredentials()))
    if err != nil {
        log.Fatalf("did not connect: %v", err)
    }
    defer conn.Close()
    c := service.NewGreeterClient(conn)
    go subscribe(c)

    reader := bufio.NewReader(os.Stdin)
    for {
        text, _ := reader.ReadString('\n')

        sayHello(c, text)
    }
}

We now compile the client to generate the corresponding executable.

    go build -o bin/client cmd/client/main.go

Run the example

Once we have the server and client executables ready, we can run them to test the example. We start the server in one terminal.

    >./bin/server
    ./bin/server
    2023/08/01 11:07:51 server listening at [::]:50051

And the client in another one. We'll start seeing messages coming up on the client terminal. And if we type a string on the client terminal, we will see the reply from the server.

    >./bin/client
    ./bin/client
    2023/08/01 11:08:01 message:"notification"
    2023/08/01 11:08:04 message:"notification"
    2023/08/01 11:08:07 message:"notification"
    John
    2023/08/01 11:08:08 Greeting: Hello John
    2023/08/01 11:08:10 message:"notification"

We can also check on the server terminal that the message from the client was received.

    >./bin/server
    ./bin/server
    2023/08/01 11:07:51 server listening at [::]:50051
    2023/08/01 11:08:08 Received: John

Conclusion

gRPC is a game-changer in the world of distributed systems, providing a fast, efficient, and flexible communication framework. In this article, we explored the key features and advantages of gRPC and demonstrated a practical example using Go. As you venture into building distributed systems, consider integrating gRPC to experience the full potential of this cutting-edge technology.

Key Types of Profiling in Go

CPU Profiling: CPU profiling identifies areas of your code that consume the most CPU time. It helps you understand which functions or methods are taking up the majority of the processing power, allowing you to focus on optimizing those areas.

Memory Profiling: Memory profiling reveals memory allocation patterns and potential memory leaks within your application. It provides insights into how your application uses memory and assists in preventing excessive memory consumption.

Goroutine Profiling: Goroutines are a hallmark of Go's concurrency model. Goroutine profiling aids in understanding how goroutines are being created, executed, and blocked, helping you optimize concurrent execution and resource utilization.

Popular Profiling Tools

pprof: The built-in pprof package is a cornerstone of Go's profiling capabilities. It offers both CPU and memory profiling, along with a web-based visualization tool. By simply importing the net/http/pprof package and exposing endpoints, you can access valuable insights about your application's performance.

go-torch: This tool creates flame graphs from pprof profiles, providing a visual representation of the CPU consumption hierarchy. Flame graphs make it easier to identify hotspots and optimize critical sections of your code.

pprof-utils: This collection of utilities extends the pprof toolset by offering additional functionalities, such as memory profiling for long-running applications and custom visualizations.

Heapster: Heapster is designed specifically for memory profiling in Go. It offers memory allocation stack traces, allowing you to identify memory-hungry sections of your code and optimize memory usage effectively.

trace: The trace package in the Go standard library facilitates tracing the execution of programs. It helps you understand the timing and interactions between goroutines, system calls, and application logic, aiding in fine-tuning concurrency.

Using Profiling Tools in Practice

We are going to use a simple example to exercise the basic profiling tools that are part of the Go standard utilities. This sample will have two variants, one of them will allocate an array of bytes in a loop to send them to a receiving goroutine. Meanwhile, the other variant will take advantage of a BytePool to reuse the required array of bytes, instead of allocating new ones in each iteration of the loop.

package main

import (
        "os"
)

const REP int = 10000000

type BytePool struct {
    pool  chan []byte
    width int
}

func New(poolSize int, bufferWidth int) *BytePool {
    return &BytePool{
        pool:  make(chan []byte, poolSize),
        width: bufferWidth,
    }
}

func (bp *BytePool) Get() (b []byte) {
    select {
    case b = <-bp.pool:
    default:
        b = make([]byte, bp.width)
    }
    return
}

func (bp *BytePool) Put(b []byte) {
    select {
    case bp.pool <- b:
    default:
    }
}

func ProcessPool() {
    channel := make(chan []byte)
    go func() {
        pool := New(REP, 5)
        for i := 0; i < REP; i++ {
            data := pool.Get()
            channel <- data
            pool.Put(data)
        }
    }()

    for i := 0; i < REP; i++ {
        _ = <-channel
    }
}

func Process() {
    channel := make(chan []byte)
    go func() {
        for i := 0; i < REP; i++ {
            data := []byte("hello")
            channel <- data
        }
    }()

    for i := 0; i < REP; i++ {
        _ = <-channel
    }
}

func main() {
        arguments := os.Args

        if (arguments[1] == "process") {
                Process()
        } else if (arguments[1] == "processPool") {
                ProcessPool()
        }
}

Garbage Collection

In Go, the GODEBUG environment variable allows you to enable various debugging options and runtime parameters. One of the features you can enable using GODEBUG is the Garbage Collection (GC) tracing, which is referred to as gctrace. Garbage collection is an essential process in Go that manages memory by identifying and reclaiming unused memory occupied by objects that are no longer reachable.

The gctrace option provides insight into the behavior of the Go garbage collector. It outputs information about garbage collection events, pause durations, and memory usage during the garbage collection process. This information can be incredibly useful for understanding how the garbage collector operates, diagnosing potential performance issues, and optimizing your Go programs.

To enable the gctrace option, set the GODEBUG environment variable to include gctrace=1. You can do this before running your Go program from the command line.

Let's run the sample variant without the BytePool to check how many garbage collection cycles it goes through.

GODEBUG=gctrace=1 go run main.go process
gc 1 @0.071s 0%: 0.043+0.31+0.022 ms clock, 0.34+0.073/0.27/0.31+0.18 ms cpu, 3->3->0 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 8 P
gc 2 @0.080s 0%: 0.022+0.68+0.002 ms clock, 0.17+0.38/0.62/0.008+0.022 ms cpu, 3->4->1 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 8 P
gc 3 @0.083s 0%: 0.034+0.66+0.028 ms clock, 0.27+0/0.70/0.37+0.22 ms cpu, 3->3->1 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 8 P
gc 4 @0.089s 0%: 0.010+0.37+0.002 ms clock, 0.081+0.19/0.43/0.22+0.018 ms cpu, 3->4->1 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 8 P
gc 5 @0.091s 0%: 0.11+0.59+0.11 ms clock, 0.88+0.10/0.51/0.14+0.92 ms cpu, 3->4->1 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 8 P
gc 6 @0.093s 0%: 0.008+0.33+0.002 ms clock, 0.070+0.089/0.32/0.34+0.020 ms cpu, 3->3->1 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 8 P# command-line-arguments
gc 1 @0.002s 5%: 0.050+0.63+0.024 ms clock, 0.40+0.16/0.96/0.42+0.19 ms cpu, 3->4->2 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 8 P
gc 2 @0.006s 6%: 0.044+1.2+0.036 ms clock, 0.35+0.49/1.4/0.53+0.29 ms cpu, 6->6->4 MB, 6 MB goal, 0 MB stacks, 0 MB globals, 8 P

# command-line-arguments
gc 1 @0.001s 6%: 0.018+0.54+0.022 ms clock, 0.14+0.27/0.34/0.56+0.17 ms cpu, 4->6->5 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 8 P
gc 2 @0.003s 5%: 0.008+0.97+0.037 ms clock, 0.070+0.051/0.40/0.72+0.29 ms cpu, 9->9->9 MB, 11 MB goal, 0 MB stacks, 0 MB globals, 8 P
gc 1 @0.111s 0%: 0.019+0.23+0.079 ms clock, 0.15+0.052/0.041/0+0.63 ms cpu, 3->3->0 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 8 P
gc 2 @0.227s 0%: 0.14+0.20+0.064 ms clock, 1.1+0.031/0.10/0+0.51 ms cpu, 3->3->0 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 8 P
gc 3 @0.348s 0%: 0.091+0.23+0.022 ms clock, 0.72+0.032/0.085/0.030+0.17 ms cpu, 3->3->0 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 8 P
gc 4 @0.466s 0%: 0.048+0.29+0.040 ms clock, 0.38+0.030/0.044/0.024+0.32 ms cpu, 3->3->0 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 8 P
gc 5 @0.551s 0%: 0.078+0.20+0.032 ms clock, 0.62+0.027/0.074/0.016+0.25 ms cpu, 2->2->0 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 8 P
gc 6 @0.665s 0%: 0.074+0.36+0.082 ms clock, 0.59+0.032/0.10/0+0.66 ms cpu, 3->3->0 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 8 P
gc 7 @0.752s 0%: 0.041+0.36+0.079 ms clock, 0.33+0.028/0.063/0.016+0.63 ms cpu, 2->2->0 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 8 P
gc 8 @0.866s 0%: 0.14+0.50+0.072 ms clock, 1.1+0.062/0.060/0+0.57 ms cpu, 3->3->0 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 8 P
gc 9 @0.953s 0%: 0.066+0.27+0.089 ms clock, 0.52+0.028/0.055/0+0.71 ms cpu, 2->2->0 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 8 P
gc 10 @1.069s 0%: 0.076+0.36+0.023 ms clock, 0.61+0.041/0.049/0+0.18 ms cpu, 3->3->0 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 8 P
gc 11 @1.185s 0%: 0.074+0.30+0.083 ms clock, 0.59+0.036/0.050/0+0.66 ms cpu, 3->3->0 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 8 P
gc 12 @1.301s 0%: 0.077+0.26+0.035 ms clock, 0.61+0.028/0.075/0.040+0.28 ms cpu, 3->3->0 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 8 P
gc 13 @1.386s 0%: 0.087+0.38+0.075 ms clock, 0.69+0.030/0.049/0+0.60 ms cpu, 2->2->0 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 8 P
gc 14 @1.503s 0%: 0.081+0.20+0.002 ms clock, 0.65+0.068/0.045/0+0.016 ms cpu, 3->3->0 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 8 P

Now, we can compare those results with the BytePool variant.

GODEBUG=gctrace=1 go run main.go processPool
gc 1 @0.053s 0%: 0.080+0.24+0.023 ms clock, 0.64+0.039/0.28/0.36+0.19 ms cpu, 3->3->0 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 8 P
gc 2 @0.060s 0%: 0.038+0.74+0.002 ms clock, 0.31+0.38/0.79/0.37+0.022 ms cpu, 3->4->1 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 8 P
gc 3 @0.062s 0%: 0.031+0.49+0.027 ms clock, 0.25+0.15/0.46/0.48+0.21 ms cpu, 3->3->0 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 8 P
gc 4 @0.066s 0%: 0.010+0.35+0.002 ms clock, 0.084+0.040/0.39/0.33+0.023 ms cpu, 3->3->1 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 8 P
gc 5 @0.068s 0%: 0.008+0.35+0.026 ms clock, 0.065+0.052/0.47/0.37+0.21 ms cpu, 3->3->1 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 8 P
gc 6 @0.070s 1%: 0.010+0.46+0.022 ms clock, 0.083+0.040/0.35/0.45+0.18 ms cpu, 3->4->1 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 8 P
# command-line-arguments
gc 1 @0.002s 5%: 0.050+0.63+0.024 ms clock, 0.40+0.16/0.96/0.42+0.19 ms cpu, 3->4->2 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 8 P
gc 2 @0.006s 6%: 0.044+1.2+0.036 ms clock, 0.35+0.49/1.4/0.53+0.29 ms cpu, 6->6->4 MB, 6 MB goal, 0 MB stacks, 0 MB globals, 8 P
# command-line-arguments
gc 1 @0.001s 5%: 0.006+0.45+0.003 ms clock, 0.054+0.25/0.43/0.21+0.024 ms cpu, 4->5->5 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 8 P
gc 2 @0.002s 7%: 0.007+0.76+0.025 ms clock, 0.063+0.048/0.99/0.35+0.20 ms cpu, 9->9->9 MB, 11 MB goal, 0 MB stacks, 0 MB globals, 8 P
gc 1 @0.018s 13%: 0.009+11+0.003 ms clock, 0.075+9.3/21/45+0.030 ms cpu, 229->229->228 MB, 4 MB goal, 0 MB stacks, 0 MB globals, 8 P

Here we can see how the number of garbage collection cycles in the BytePool variant is far lower than in the regular version of the program.

Heap escape analysis

Escape analysis is a critical optimization technique used by the Go compiler to determine the lifetime of variables and whether they should be allocated on the stack or the heap. The main goal of escape analysis is to minimize the allocation and deallocation of memory, resulting in more efficient memory usage and improved performance in Go programs.

In Go, memory allocation on the heap involves more overhead compared to stack allocation. Variables allocated on the stack have faster access times and are automatically deallocated when they go out of scope, whereas heap-allocated variables require the Garbage Collector to manage their lifecycle, which can introduce potential performance overhead.

Escape analysis helps the compiler make informed decisions about whether a variable's reference escapes its immediate scope, thus necessitating heap allocation. The analysis identifies scenarios where a variable's reference is stored outside the current function's scope, such as:

Returning Pointers: If a function returns a pointer to a local variable, the variable must be allocated on the heap to ensure its data remains valid even after the function exits.

Storing Pointers: If a pointer to a local variable is stored in a data structure or globally accessible variable, the variable's data must be kept on the heap.

Goroutine Communication: If a variable's reference is passed to a goroutine (concurrent function), it must be accessible from the heap since the goroutine's lifetime extends beyond the current function.

Function Parameters: If a variable's reference is passed as a parameter to another function and that function retains the reference, the variable's data must reside on the heap.

We can perform a heap escape analysis of a program by using the -gcflags argument with the option '-m'. Let's run the sample version using these options to generate the heap escape analysis of the program.

go run -gcflags '-m' main.go process
# command-line-arguments
./main.go:14:6: can inline New
./main.go:21:6: can inline (*BytePool).Get
./main.go:30:6: can inline (*BytePool).Put
./main.go:39:5: can inline ProcessPool.func1
./main.go:40:14: inlining call to New
./main.go:42:20: inlining call to (*BytePool).Get
./main.go:44:12: inlining call to (*BytePool).Put
./main.go:55:5: can inline Process.func1
./main.go:15:9: &BytePool{...} escapes to heap
./main.go:21:7: bp does not escape
./main.go:25:11: make([]byte, bp.width) escapes to heap
./main.go:30:7: bp does not escape
./main.go:30:25: leaking param: b
./main.go:39:5: func literal escapes to heap
./main.go:40:14: &BytePool{...} does not escape
./main.go:42:20: make([]byte, bp.width) escapes to heap
./main.go:55:5: func literal escapes to heap
./main.go:57:18: ([]byte)("hello") escapes to heap

Memory benchmark

The benchmem option is a feature provided by the built-in testing package (testing) that allows you to measure memory allocation and deallocation behavior during benchmark tests. Benchmark tests in Go are used to evaluate the performance of code snippets, functions, or packages by running them repeatedly and measuring execution time. The benchmem option extends this capability to include memory-related metrics, providing insights into memory usage patterns.

When you run benchmark tests with the benchmem option enabled, the Go testing framework not only measures execution time but also collects information about memory allocations, memory deallocations, and memory allocations per operation.

We need first to code a test for the sample program. It will contain methods for testing and benchmarking both versions of the program.

package main

import "testing"

func TestProcess(t *testing.T){
    Process()
}

func BenchmarkProcess(b *testing.B){
        for n := 0; n < b.N; n++ {
        Process()
        }
}

func TestProcessPool(t *testing.T){
    ProcessPool()
}

func BenchmarkProcessPool(b *testing.B){
        for n := 0; n < b.N; n++ {
        ProcessPool()
        }
}

Now we can exercise the sample program with the benchmem option to compare the performance of both versions of the program.

go test -bench . -benchmem
goos: linux
goarch: amd64
pkg: go-profiling-tools
cpu: 11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz
BenchmarkProcess-8                     1        1585461568 ns/op        53334224 B/op   10000006 allocs/op
BenchmarkProcessPool-8                 1        2078355011 ns/op        240001664 B/op         6 allocs/op
PASS
ok      go-profiling-tools      7.252s

The output includes memory-related information for each benchmark iteration.

• allocs/op: The average number of memory allocations per benchmark iteration.

• alloced/op: The average amount of memory allocated per benchmark iteration.

• bytes/op: The average amount of memory allocated per operation (operation in the benchmark code).

• mallocs/op: The number of heap allocations (mallocs) per benchmark iteration.

• frees/op: The number of heap deallocations (frees) per benchmark iteration.

CPU profiling

The cpuprofile is a feature provided by the built-in pprof package that allows you to profile the CPU usage of your program. Profiling is the process of analyzing a program's runtime behavior to identify performance bottlenecks and areas for optimization. The cpuprofile feature is particularly useful for understanding how much time your program spends on different functions and methods, helping you pinpoint areas that might benefit from optimization.

The pprof package provides HTTP endpoints that allow you to navigate through the resulting html files using a web browser. However, we are going to use the options required for generating static reports, so that we can integrate the CPU profiling as a step of a CI/CD pipeline. To do so, we need to install the graphviz package to be able to generate the resulting SVG graphics.

sudo apt install graphviz

The CPU profile generated using the cpuprofile feature provides insights into the call stack and the amount of time spent in each function or method. This information can help you identify performance hotspots and optimize critical sections of your code.

go test -cpuprofile cpu.out && go tool pprof -svg -output=cpu.out.svg cpu.out
PASS
ok      go-profiling-tools      3.917s
Generating report in cpu.out.svg

It's worth noting that profiling does introduce some overhead, so it's recommended to use it selectively in performance-critical scenarios. Additionally, the profiling endpoints should not be exposed in production environments due to potential security risks.

RAM profiling

The memory profiling feature allows you to monitor and analyze the memory usage and allocation patterns of your program over time. This is accomplished through the generation of memory profiles, which provide insights into how memory is being allocated and deallocated by your Go application. Memory profiling can help you identify memory leaks, inefficient memory usage, and opportunities for optimization.

The Go runtime includes a built-in memory profiler that can be triggered during the execution of your program. This profiler generates memory profiles that you can later analyze using various tools to gain a better understanding of your program's memory behavior.

Similarly to the CPU profiling, we are going to generate a static report that could be integrated in a CI/CD pipeline.

go test -test.memprofilerate=1 -memprofile mem.out && go tool pprof -svg -output=mem.out.svg mem.out
PASS
ok      go-profiling-tools      8.389s
Generating report in mem.out.svg

Coverage profiling

Coverage profiling is not related to performance. However, it is a really useful feature of the Go toolset. The coverprofile feature is used to generate code coverage reports during the execution of your tests. Code coverage measures the extent to which your tests exercise the different parts of your codebase. It helps you understand which portions of your code are being tested and which areas might need additional test coverage.

By using the coverprofile flag during the go test command, you can generate a coverage profile that shows which lines of code were executed by your tests. This profile can then be used to generate detailed coverage reports that indicate which parts of your code were covered and which were not.

go test -coverprofile=coverage.out && go tool cover -html=coverage.out -o coverage.out.html
PASS
coverage: 80.8% of statements
ok      go-profiling-tools      3.948s

Conclusion

Profiling tools are indispensable assets for any Go developer aiming to create performant applications. By harnessing the capabilities of CPU, memory, and goroutine profiling, you can unlock the full potential of Go's efficiency and concurrency model. Whether you're building microservices, web applications, or system-level software, profiling tools provide the roadmap to optimizing performance, minimizing resource consumption, and delivering exceptional user experiences. So, embrace the power of profiling tools and embark on a journey toward crafting high-performance Go applications that stand out in today's competitive software landscape.

Installation

The first step will consist of installing the dependencies required for executing the service. These will be curl for sending the file using FTP, and pure-ftpd as FTP server.

sudo apt install curl
sudo apt install pure-ftpd
sudo apt install inotify-tools

Send service

The send service will consist of a send.sh script, which takes five arguments:

• out_tray: this is the folder where the user has to put the files to be sent to the destination host.

• ftp_user: user used for the FTP connection.

• ftp_password: password for the user of the FTP connection.

• ftp_host: destination host for the FTP transfer.

• ftp_folder: folder within the destination host where the files are going to be copied.

This script monitors the out_tray folder checking for new files using the inotifywait utility. Once a new file is detected, it sends the file to the FTP destination using curl. It also uses the logger utility to print traces about the execution of the script.

#!/bin/bash
if [ $# -ne 5 ]; then
  echo "Usage: send.sh <out_tray> <ftp_user> <ftp_password> <ftp_ip> <ftp_folder>";
  exit;
fi

while file="$(inotifywait -q -e move --format %f $1)";
do
  logger "[`basename "$0"`] Sending file $file"
  curl -T $1/$file ftp://$4/$5/$file --user $2:$3
  rm $1/$file;
  logger "[`basename "$0"`] Sent file $file"
done

The next step is to define a systemd service that will call the send.sh script.

[Unit]
Description=Send Service

[Service]
ExecStart=/usr/local/bin/transferor/send.sh /usr/local/bin/transferor/outTray al al 127.0.0.1 /usr/local/bin/transferor/inTray
StandardOutput=null
Restart=on-failure

[Install]
WantedBy=multi-user.target
Alias=send.service

Receive service

The receiving service will follow a similar approach. It will be composed of the recv.sh script, which takes two arguments:

• in_tray: folder where files sent through FTP from the source host are received.

• script: script file to be executed to process the files received in the in_tray folder.

This script also monitors a folder using the inotifywait utility. Once a new file appears in the in_tray folder, it runs the script specified as an argument to process the received file. As the send.sh did, it uses the logger utility to print traces about the execution of the script.

#!/bin/bash
if [ $# -ne 2 ]; then
  echo "Usage: recv.sh <in_tray> <script>";
  exit;
fi

while file="$(inotifywait -q -e close_write --format %f $1)";
do
  logger "[`basename "$0"`] Received file $file"
  source $2 "$1/$file";
  logger "[`basename "$0"`] Processed file $file"
done

For example, we can define an echo.sh processing script that just echoes the content of the received file.

#!/bin/bash

cat $1

After coding these two scripts, we can define the systemd service that will execute the recv.sh script.

[Unit]
Description=Receive Service

[Service]
ExecStart=/usr/local/bin/transferor/recv.sh /usr/local/bin/transferor/inTray /usr/local/bin/transferor/echo.sh
StandardOutput=null
Restart=on-failure

[Install]
WantedBy=multi-user.target
Alias=recv.service

Deployment

Once we have all the scripts and services defined, we can deploy them to the proper folders and set them up to be run as systemd services.

#!/bin/bash

TARGET=/usr/local/bin/transferor

echo "Creating directory $TARGET"

sudo mkdir $TARGET
sudo mkdir $TARGET/inTray
sudo chown al:al $TARGET/inTray
sudo mkdir $TARGET/outTray
sudo chown al:al $TARGET/outTray

echo "Copying files into $TARGET"
sudo cp app/echo.sh $TARGET
sudo cp recv/recv.sh $TARGET
sudo cp recv/recv.service $TARGET
sudo cp send/send.sh $TARGET
sudo cp send/send.service $TARGET

echo "Installing services"
sudo ln -s $TARGET/recv.service /etc/systemd/system
sudo systemctl enable recv
sudo ln -s $TARGET/send.service /etc/systemd/system
sudo systemctl enable send

echo "Done"

After deploying the scripts and services, we have to reboot the machine. And after that, we can check the status of the defined services.

sudo reboot
sudo systemctl status recv
sudo systemctl status send

Testing the example

At this point, we should have both services up and running. Therefore, we can do a quick test to check that they work as expected. To do so, we just have to copy a test file into the out_tray folder, and we will see the content of that file in the log messages of the logger utility.

echo "test" > test.txt
sudo mv test.txt /usr/local/bin/transferor/outTray

journalctl -f

Conclusion

In this article, we have seen a simple way of defining a systemd service that allows you to easily send files between hosts, just by copying them to the outbound folder. And allows you to run a script to process the received file in the destination host. That way you could automate the distribution and processing of files among multiple machines. Obviously, this example is not suitable for production code. But it allows you to get a grasp of how basic utilities like inotifywait, logger, curl and the definition of systemd services work.