Touring the Zig•EM code-scape

The next few blog posts will explore the Zig•EM programming framework in ever-greater detail – starting with instructions for installing the latest software version, and then moving on to a 10,000' overview that touches upon some core concepts and constructs of Zig•EM.

↶

Updating your installation

The process for (initially) installing and (subsequently) updating your local version of Zig•EM boils down to three basic steps:

install Zig
clone zigem-dev
execute zig build

As part of step , you should have added the zig executable to your path. Invoke the zig version command for confirmation.(1)

Zig•EM currently requires version 0.13.0 of Zig

As for step , an initial git clone and subsequent git pull of the zigem-dev repository will take you to the latest release of Zig•EM. Tags on the main branch (v25.0.1, v25.0.2, ...) enable you to easily move to earlier releases.(1)

If you wish to avoid the git command altogether, just
download the sources for individual releases.

Step will then build the zigem command-line executable, as well as download/install other required artifacts. Invoke zig build verify as a final test.

You may want to consult these Getting started instructions for more detail on first-time installation.

The Zig cache — a cornerstone of zig build

As you might expect, the first invocation of zig build can take a fair amount of time; subsequent invocations of zig build, however, can complete almost instantenously. Not unlike make, the zig build-system strives to perform the minimal number of steps required to complete the task at hand.

Should you choose to learn more(1)about the build-system, you'll quickly come to appreciate the critical role played by the Zig cache – a fascinating foundational element first described here in detail. In simple terms, virtually any artifact touched when invoking zig build will persist in Zig's filesystem cache.

ziglang.org
zig.news
kristoff.it
michelh.com

Strange as it may seem, you won't find anything resembling a zig clean command; all cached artifacts have a unique content-based identifier. But sometimes to alleviate any lingering doubts about cache coherence, I will delete the special .zig-cache/ and zig-out/ folders in the repository's root before invoking zig build.

The artifacts provisioned through zig build include a version of this Zig Language extension for VS Code – specially modified to add awarenesss of the Zig•EM framework. Forked from upstream repositories, this extension currently functions as a "drop-in" replacement.(1)

If you've already installed the stock Zig Language extension,
you'll have to disable it within the zigem-dev workspace.

To install, invoke Extensions: Install from VSIX... from the VS Code Command Palette and then navigate to your zigem-dev/zig-out/tools/ folder. This folder should contain a file named vscode-zigem-VERSION.vsix, which you can now select and Install.(1)

If you find multiple .vsix files in the folder, select the one with the highest VERSION number.

Once complete, invoke Command Palette > Developer: Reload Window to refresh your workspace. If all goes as planned, you'll see a Zig•EM activated popup appear briefly.

Repository organization

With our vscode-zigem extension in place, let's start exploring the zigem-dev repository itself – beginning with its top-level organization as a "typical" Zig project:

It all starts with build.zig

Even if you never plan to follow-up on the zig build references cited earlier, do invest under a minute of your valuable time digesting this material found at zig.guide.

Drilling down into the special workspace/ folder highlighted above, we formally enter the domain of Zig•EM – featuring a number of packages with (unique) names like em.core, which in turn contain (uniquely-named) buckets such as em.utils and em.examples.basic.

A Zig•EM workspace contains a special zigem.ini file at its root, along with a distinguished zigem-package.ini file at the top of each package found therein. We'll visit these as well as other .ini configuration files over the course of time.

Descending one more level in our package > bucket hiearchy, we come to individual compilation units – Zig source files which uses a novel "em" library provided by the Zig•EM framework. A special .em.zig file extension plus an associated icon visually distinguish a Zig•EM unit from "ordinary" Zig sources [src/*.zig] depicted earlier.

We'll soon take a closer look at the BlinkerP.em.zig and FiberP.em.zig source files.

By design, Zig•EM limits its package > bucket > unit hiearchy to just three levels. The names chosen for top-level packages and their constituent buckets will, however, often assume a long.qualified.form – used to ensure a measure of uniqueness and durability.

A hierarchical namespace for buckets and packages

Zig•EM favors a namespace hierarchy when labeling individual buckets and packages, in which the supplier has a globally-unique prefix ["org.<domain>", "git.<userid>", etc ] used in these element names.

Having said that, buckets and packages supplied by The EM Foundation itself ["git.em-foundation"] will often use a shorter (though still unique) moniker such as "em." for their namespace prefix.

Another Zig•EM convention: names ascribed to buckets should themselves remain globally-unique – independent of their containing package(s), which also should have unique names.

While the "em.arch.arm" bucket lives in the "em.core" package, a third-party ["git.biosbob"] could also supply a package (labeled with their own prefix) that contains buckets with the same legacy "em.**" names.

As we'll see shortly, individual Zig•EM source files use a special em.import operator to reference other units found in the workspace. By design, each unit has a canonical name of the form bucket/Unit: em.examples.basic/BlinkerP, ti.distro.cc23xx/BoardC, etc.

While buckets ultimately reside within packages, individual .em.zig source files should never reference the latter by name. Zig•EM units in fact remain oblivious to any package(s) containing their own bucket as well as other bucket/Unit elements they may import.

In essence, packages provide a delivery vehicle for bucket/Unit content. Each package has a manifest (zigem-package.ini) which in general names other packages upon which it depends. Zig•EM topologically sorts the (acyclic) dependency relation amongst all packages in the workspace, yielding a search path used when resolving bucket/Unit references.

Terminology :: Zig•EM = Zig + EM

Mapping the original EM language into Zig can lead to terminology clashes, in which elements like package have distinct meanings in each language domain. EM, for instance, featured a bundle > package > unit hierarchy. For consistency, Zig•EM will favor using native Zig nomenclature as much as possible.

Over time, we'll directly leverage Zig Package Management mechanisms for delivering buckets of Zig•EM content. Said another way, expect each Zig•EM package to carry their own build.zig and build.zig.zon files. We'll also use Zig modules in build.zig to define bucket/Unit names exposed by the package.

The workspace/ folder depicted earlier currently serves as an embryonic prototype – delivered as part of the zigem-dev repository. When we reach Zig•EM v25.1.0, however, users will create their own workspace(s) populated with multiple packages from multiple suppliers – each bundling uniquely-labeled bucket/Unit content.

Source code constructs

Zig•EM units, as noted earlier, reside in .em.zig source files and make use of a special "em" library. In all other respects, these files conform to the syntax and semantics of Zig.

For Zig-lings and Zig-gurus alike....

As you explore the dozens of .em.zig files found in the workspace, we recognize that your knowledge of Zig can vary considerably. We'll do our best to literally and figuratively highlight Zig•EM's unique constructs.

For those seeing the Zig language for the first time, we strongly recommend Learning Zig by Karl Sequin. For those coming from the embedded space with a background in C, you'll find some further motivation behind Zig in this @avestura blog post as well as this Zig in 100 Seconds video.

And for the Zig gurus: we always welcome your insightful comments at ziggit.dev on how to best leverage the underlying language in implementing the core concepts and constructs of the Zig•EM framework.

To begin, let's focus on the em.examples.basic/BlinkerP program which illustrates some rudimentary constructs commonly found in a Zig•EM unit:

em.examples.basic/BlinkerP.em.zig
pub const em = @import("../../zigem/em.zig");
pub const em__U = em.module(@This(), .{});

pub const AppLed = em.import.@"em__distro/BoardC".AppLed;
pub const Common = em.import.@"em.mcu/Common";

pub const EM__TARG = struct {
    pub fn em__run() void {
        AppLed.on();
        for (0..10) |_| {
            Common.BusyWait.wait(500_000);
            AppLed.toggle();
        }
        AppLed.off();
    }
};

Even with no prior knowledge of the Zig language, the behavior of the "main" function between lines 8 and 15 should seem obvious – toggling AppLed ten times every half-second.

Standing back, the first two lines of this file serve as boilerplate which you'll find in every Zig•EM unit: line 1 imports our special "em" library, while line 2 declares this unit's role as an em.module and binds a corresponding object to the em__U framework constant.

language keywords versus framework operators

The EM language introduced the keywords module, interface, composite, and template to declare the role played by a specific unit; the Zig•EM framework uses its operators em.module, em.interface, etc. for a similar purpose. While Zig•EM modules predominate, you will see other sorts of units in due course.

Lines 4–5 use the framework's em.import operator to access em.mcu/Common, as well as the BoardC (composite) unit located in a logical bucket named em__distro.(1) The BoardC composite in turn aggregates a large number of modules featuring hardware-specific implementations [AppLed] of hardware-independent interfaces [em.hal/AppLedI] .

currently bound to ti.distro.cc23xx within our workspace's zigem.ini configuration file

Finally, the EM_TARG structure defined at line 7 introduces a new lexical scope containing other Zig declarations (constants, types, variables, functions). We'll explain the role of this special struct once we examine the lengthier em.examples.basic/FiberP program:

em.examples.basic/FiberP.em.zig
pub const em = @import("../../zigem/em.zig");
pub const em__U = em.module(@This(), .{});
pub const em__C = em__U.config(EM__CONFIG);

pub const EM__CONFIG = struct {
    blinkF: em.Param(FiberMgr.Obj),
};

pub const AppLed = em.import.@"em__distro/BoardC".AppLed;
pub const Common = em.import.@"em.mcu/Common";
pub const FiberMgr = em.import.@"em.utils/FiberMgr";

pub const EM__META = struct {
    //
    pub fn em__constructH() void {
        const blinkF = FiberMgr.createH(em__U.fxn("blinkFB", FiberMgr.BodyArg));
        em__C.blinkF.set(blinkF);
    }
};

pub const EM__TARG = struct {
    //
    const blinkF = em__C.blinkF;

    pub fn em__run() void {
        blinkF.post();
        FiberMgr.run();
    }

    var count: u8 = 5;

    pub fn blinkFB(_: FiberMgr.BodyArg) void {
        em.@"%%[d]"();
        count -= 1;
        if (count == 0) em.halt();
        AppLed.on();
        Common.BusyWait.wait(100_000);
        AppLed.off();
        Common.BusyWait.wait(100_000);
        blinkF.post();
    }
};

This sample program illustrates basic use of em.utils/FiberMgr – a module delivered within the em.core package, and which manages lightweight threads of type FiberMgr.Obj using a factory design pattern applied often within Zig•EM.

Looking first at the features declared and defined within the EM__TARG scope, we find:

line 23 — the constant blinkF, which references a Fiber.Obj created earlier in the program
line 25 — the framework function em__run, which serves as this example's runtime entry-point
line 30 — the variable count, used to track the number of times the program must blink AppLed
line 32 — the function blinkFB, which represents the code executed upon activating the blinkF fiber

We'll fill in some more details after we examine the top-half of em.examples.basic/FiberP.

Complementing EM__TARG, the FiberP program also declares EM__META beginning at line 13. Here too, this special struct can contain declarations and definitions within its scope:

line 15 — the framework function em__constructH, which creates the blinkF fiber used later in the program
line 16 — FiberMgr.createH also binds our (runtime) blinkFB function to the newly-created blinkF fiber

To complete the picture, line 5 of the FiberP program defines a special EM__CONFIG structure – an instance of which the program assigns to the EM__C framework constant using more boilerplate code back at line 3.

VS Code snippets

The vscode.zigem extension includes boilerplate code which you can inject into new units within your workspace. Invoke Command Palette > Snippets: Fill File with Snippet on an empty .em.zig file to experiment.

To appreciate the pivotal role played by the EM__C.blinkF field throughout our FiberP example, we'll need to examine how the Zig•EM framework transforms the FiberP.em.zig source file into an executable program image.

Program compilation

Let's now compile the em.examples.basic/FiberP program, using the framework's zigem command first seen here:

[zigem-dev/workspace]
$ zigem compile -f em.core/em.examples.basic/FiberP.em.zig
compiling META ...
    board: LP_EM_CC2340R5
    setup: ti.cc23xx://default
compiling TARG ...
    image sha: 93aad9c9
    image size: text (1560) + const (0) + data (40) + bss (4)
done in 6.96 seconds

You'll immediately notice that zigem compiles the program twice : once for the META domain, and then again for the TARG domain.

The META compilation stage processes all source code from lines 1–19, effectively merging the EM__META declarations into the top-level file scope.

The TARG compilation stage follows suit, combining common declarations [1–11] with EM__TARG declarations [21–42] as input for the underlying Zig compiler.

While diving into the implementation details of the zigem command lies well beyond the scope of this introductory article, we'll give you a quick peek into the Zig•EM "basement" where most of the magic occurs.

For those in the know, just follow the trail heading out from build.zig and src/main.zig ....

To start, invoke the zigem clean command followed by zigem refresh. Your workspace should now reveal two additional elements generated by the framework:

The .zigem-main.zig file provides entry points for the META and TARG compilatiion stages, which at this point respectively reference empty zigem/meta.zig and zigem/targ.zig files.

You'll find the "interesting" code generated at this stage in files like imports.zig, which reflects an upfront discovery of all package > bucket > unit elements in the workspace.

You'll also find a makefile, whose recipes will eventually migrate into (generated) build.zig files.

By invoking zigem compile with its -m [--meta] option, we can better appreciate the role played by this initial compilation stage.

[zigem-dev/workspace]
$ zigem compile -f em.core/em.examples.basic/FiberP.em.zig -m
compiling META ...
    board: LP_EM_CC2340R5
    setup: ti.cc23xx://default
done in 4.73 seconds

Before compilation begins, the framework populates zigem/meta.zig with code that will execute our new META program – rooted at the em.examples.basic/FiberP unit and incorporating all other units (recursively) reached through em.import.

Next, the framework invokes the Zig compiler using the zig build-exe command and then runs the zigem/out/meta-main executable image output by the compiler. The meta goal inside zigem/makefile currently codifies the steps used in this process.

If not obvious, META programs run on your host PC — and not on resource-constrained target HW !!!

Functions like em__constructH [ defined in the FiberP EM__META scope at 13] enjoy execution in an environment with virtually unlimited memory and processing resources, as well as access to the host's file system or even the internet if necessary.

In general, META programs declare config parameters as fields within per-unit EM__CONFIG structures [5] and modify their values [17] via local definitions of em__C. Ultimately, Zig•EM writes the final state of all program config parameters into an updated zigem/targ.zig file.

In general, META programs declare config parameters as fields within per-unit EM__CONFIG structures [5] and modify their values [17] via local definitions of em__C. Ultimately, Zig•EM updates its zigem/targ.zig file to reflect the final state of all program config parameters.

A config parameter acts like a var in your META program, but like a const in your TARG program.

The downstream compilation of FiberP for the TARG domain uses a public exec function defined at the bottom of zigem/targ.zig as the runtime entry-point. After initializing the target hardware(1)exec will call the em__run function [25] defined in BlinkerP.

via special function like em__reset and em__startup
defined in lower-level program units

To summarize: We have a hosted upstream META program that outputs constant data consumed by a downstream (cross-compiled) TARG program. This novel two-stage build flow ultimately yields highly-optimized executable images for resource-constrained MCU platforms:

To summarize:

a hosted upstream META program manipulates CONFIG parameters as variables
a downstream TARG program consumes these (constant) parameters when cross-compiled
a novel build flow which ultimately yields highly-optimized firmware for resource-constrained MCUs

[zigem-dev/workspace]
$ zigem compile -f em.core/em.examples.basic/FiberP.em.zig
compiling META ...
    board: LP_EM_CC2340R5
    setup: ti.cc23xx://default
compiling TARG ...
    image sha: 93aad9c9
    image size: text (1560) + const (0) + data (40) + bss (4)
done in 6.96 seconds

Further exploration

Depending upon your background and interests, we present you with several different paths for increasing your understanding of the Zig•EM framework:

More sample programs...

visit other sample programs in the em.examples.basic bucket
load these programs onto target hardware, following these instructions

Go deep

familiarize yourself with the other em.** buckets, as well as the ti.** content
starting with AppLed, plumb down through all units contributing to its implementation

Behind the curtain...

get dirty inside of em.zig and meta-main.zig, found in the em.lang bucket
offer up guidance on how to better leverage the Zig language and its build system

Happy coding !!!