Skip to main content

ZMK State Of The Firmware #5

路 8 min read
Pete Johanson
Pete Johanson
Project Creator

Welcome to the fifth ZMK "State Of The Firmware" (SOTF)!

This update will cover all the major activity since SOTF #4. That was over a year ago, so lots to cover!

Recent Activity#

Here's a summary of the various major changes since last time, broken down by theme:

Keymaps/Behaviors#

Since last time, there have been several new powerful keymap features and behaviors added, including several asked for features, such as tap-dance and macros.

Caps Word#

petejohanson added the caps word behavior, i.e. &caps_word, in #823 that allows toggling a mode where all all alpha characters are sent to the host capitalized until a non-alpha, non-"continue list" keycode is sent. This can be useful for typing things like CONFIG_ENABLE_CAPS_WORD without having to hold down shift. This is similar in spirit to using the caps lock key, but with the added benefit of turning itself off automatically.

Key Repeat#

petejohanson added the new key repeat behavior in #1034 to allow repeating the last sent key-press again, including any modifiers that were applied to that key press. It can be added to your keymap using the simple &key_repeat reference.

Macros#

petejohanson, taking heavy inspiration on the initial work from okke-formsma, added macro support in #1168. Several common patterns are documented, but one example, changing the underglow color as you activate/deactivate a layer, looks like:

ZMK_MACRO(layer_color_macro,  wait-ms = <0>;  tap-ms = <0>;  bindings      = <&macro_press &mo 1>      , <&macro_tap &rgb_ug RGB_COLOR_HSB(128,100,100)>      , <&macro_pause_for_release>      , <&macro_release &mo 1>      , <&macro_tap &rgb_ug RGB_COLOR_HSB(300,100,50)>;)

Tap Dance#

kurtis-lew worked diligently to add the tap-dance behavior in #1139, allowing different behaviors to be invoked based on the number of times a user taps a single key in their keymap, e.g.

/ {    behaviors {        td0: tap_dance_0 {            compatible = "zmk,behavior-tap-dance";            label = "TAP_DANCE_0";            #binding-cells = <0>;            tapping-term-ms = <200>;            bindings = <&kp N1>, <&kp N2>, <&kp N3>;        };    };
    keymap {        compatible = "zmk,keymap";
        default_layer {            bindings = <                &td0            >;        };    };};

Conditional Layers#

bcat added conditional layers in #830 as a generalized version of the common "adjust layer" pattern on smaller keyboards.

Example:

/ {    conditional_layers {        compatible = "zmk,conditional-layers";        tri_layer {            if-layers = <1 2>;            then-layer = <3>;        };    };};

Combos#

mcrosson added the layer specific combos in #661, so users can make certain combos only triggerable when the layers set for the combo are active.

This is used by the ZMK implementation of ARTSEY extensively.

Sticky Keys#

okke-formsma updated sticky keys in #1122 to add the ignore-modifiers; property; when set, sticky keys won't release when other modifiers are pressed. This allows you to combine sticky modifiers, which is popularly used with "callum-style mods".

Hold-Tap Improvements#

jmding8 added an additional positional hold-tap configuration in #835 to help certain sequences produce the expected results.

jmding8 also added an additional hold-tap flavor: tap-unless-interrupted in #1018 which works very well with the new positional hold-tap config.

okke-formsma implemented retro-tap hold-tap property in #667

okke-formsma also added quick-tap-ms hold-tap property in #655

Apple Device Compatibility Improvements#

Pairing#

petejohanson did some sleuthing and fixed a long standing problem with inconsistent pairing with macOS in [#946]](https://github.com/zmkfirmware/zmk/pull/946). With the changes, macOS more reliably pairs with ZMK devices.

Consumer (Media) Codes#

Another persistent bug that Apple users experienced was related to crashes and problems with keyboard configurations, that was traced to an issue with ZMK's HID usage that was fixed by petejohanson in #726.

Debounce Enhancements#

joelspadin applied some major enhancements to our debouncing approach to allow fine grained control of our debouncing in #888, including allowing eager debouncing which can reduce key press latency.

Split Improvements#

Behavior Locality#

The long awaited locality enhancement was finally merged by petejohanson in #547, allowing more fine grained control of where certain behaviors are invoked. Some key improvements thanks to the changes:

  • RGB Underglow behaviors now run globally, so enabling/disabling RGB, changing the color, animation, etc. applies to both sides of a split properly.
  • Reset/Bootloader behaviors now run wherever the key was pressed. For example, adding a &bootloader reference to the peripheral side of a split will now put that side of the split into the bootloader when pressed.

Split Connections#

petejohanson also added fixes to improve split re-connection for certain scenarios in #984, helping ensure splits properly connect when one side or the other is reset.

Hardware Support#

Backlight#

bortoz added single color backlight support in #904 for those keyboards that have it as an alternative to RGB underglow.

E-Paper Display (EPD) Driver#

petejohanson worked with LOWPROKB to add support for the E-Paper Displays (EPD) in #895 used in keyboards like the Corne-ish Zen.

nRF VDDH Battery Sensing#

joelspadin added a new sensor driver to support battery charge calculation by sensing voltage on the VDDH pin on nRF52 chips in #750, which is particularly useful for designs using "high voltage mode" with that SoC.

Miscellaneous#

Documentation#

dxmh and caksoylar have joined the ZMK organization to help with documentation, and have been doing an amazing job adding new docs, and leading reviewing docs related PRs to free other contributors up to focus on other areas. It's been an incredible addition to ZMK!

NKRO Support#

petejohanson's work on the HID foundation also included adding support for full NKRO HID in #726 that can be enabled by adding the following to your .conf file for your config:

CONFIG_ZMK_HID_REPORT_TYPE_NKRO=y

Power Profiler#

It's been live for a while, but nicell added an amazing power profiler in #312 to allow users to estimate their battery life for various hardware configurations.

Min/Max Underglow Brightness#

malinges added support for configuring min/max underglow brightness in #944 by setting the values in your .conf file as percentages of full:

CONFIG_ZMK_RGB_UNDERGLOW_BRT_MIN=20CONFIG_ZMK_RGB_UNDERGLOW_BRT_MAX=80

This can be useful to be sure that lowering brightness doesn't set the brightness to zero, and raising the brightness doesn't consume too much power.

Zephyr 3.0#

petejohanson helped prepare and test the upgrade of ZMK to Zephyr 3.0 in #1143. The updated Zephyr release brings with it some key BLE stability fixes, as well as various other core improvements that improve ZMK. This was a huge undertaking!

New Shields#

New Boards#

Board/Shield Metadata#

nicell and petejohanson worked together in #883 to settle on a metadata format that is used to document every board and shield. This now drives automatic generation of our supported hardware page and our more nuanced GH Actions automation for testing changes to ZMK.

Coming Soon!#

Some items listed in the last coming soon section are still under active development.

  • RP2040 support
  • Peripheral rotary encoder support
  • Caps/Scroll/Num Lock LED support
  • Mouse Keys
  • Wired split support
  • More modular approach to external boards/shields, custom code, user keymaps, etc.
  • More shields and boards

Statistics#

Some statistics of interest for ZMK:

  • GitHub (lifetime stats)
    • 105 Contributors
    • 791 Closed PRs
    • 849 Stars
    • 832 Forks
  • Discord Chat
    • 3430 total registered
  • Website (last 30 days)
    • 35.9K page views
    • 3.29K new users

Thanks!#

As we approach the two year birthday for ZMK, I am reminded of how far we have come in such a short time, in large part thanks to the amazing community that has grown around it. I am so grateful to have so many contributors, testers, and user believing in the project and helping make it a joy to work on.

Zephyr 3.0 Update

路 6 min read
Pete Johanson
Pete Johanson
Project Creator

I'm happy to announce that we have completed the work to upgrade ZMK to Zephyr 3.0!

petejohanson did the upgrade work to adjust ZMK for the Zephyr changes.

  • Moving to Zephyr's UF2 build integration that was submitted upstream by petejohanson
  • Additional color-mapping property needed for ws2812 LED strep devicetree nodes
  • Zephyr core API changes, including delayed work, USB/HID
  • Adjust for pinctrl changes on stm32
  • Fixes for power management and log formatter changes

Getting The Changes#

Use the following steps to update to the latest tooling in order to properly use the new ZMK changes:

User Config Repositories Using GitHub Actions#

Existing user config repositories using Github Actions to build will pull down Zephyr 3.0 automatically, however to build properly, the repository needs to be updated to use the stable Docker image tag for the build:

  • Open .github/workflows/build.yml in your editor/IDE

  • Change zmkfirmware/zmk-build-arm:2.5 to zmkfirmware/zmk-build-arm:stable wherever it is found

  • Locate and delete the lines for the DTS output step, which is no longer needed:

      - name: ${{ steps.variables.outputs.display-name }} DTS File    if: ${{ always() }}    run: |      if [ -f "build/zephyr/${{ matrix.board }}.pre.tmp" ]; then cat -n build/zephyr/${{ matrix.board }}.pre.tmp; fi      if [ -f "build/zephyr/zephyr.dts" ]; then cat -n build/zephyr/zephyr.dts; fi
note

If you created your user config repository a while ago, you may find that your build.yml file instead references a zephyr-west-action-arm custom GitHub Action instead. In this case, the upgrade is not as direct. We suggest that instead you re-create your config repository to get an updated setup using the new automation approach.

VS Code & Docker (Dev Container)#

If you build locally using VS Code & Docker then:

  • pull the latest ZMK main with git pull for your ZMK checkout
  • reload the project
  • if you are prompted to rebuild the remote container, click Rebuild
  • otherwise, press F1 and run Remote Containers: Rebuild Container
  • Once the container has rebuilt and reloaded, run west update to pull the updated Zephyr version and its dependencies.

Once the container has rebuilt, VS Code will be running the 3.0 Docker image.

Local Host Development#

The following steps will get you building ZMK locally against Zephyr 3.0:

  • Run the updated toolchain installation steps, and once completed, remove the previously installed SDK version (optional, existing SDK should still work)
  • pull the latest ZMK main with git pull for your ZMK checkout
  • run west update to pull the updated Zephyr version and its dependencies

From there, you should be ready to build as normal!

Board/Shield Changes#

The following changes have already been completed for all boards/shields in ZMK main branch. For existing or new PRs, or out of tree boards, the following changes are necessary to properly work with the latest changes.

RGB Underglow#

Zephyr's WS2812 led_strip driver added a new required property. When adding underglow to a board, you now must also add a color-mapping property, like:

led_strip: ws2812@0 {  compatible = "worldsemi,ws2812-spi";  label = "WS2812";
  /* SPI */  reg = <0>; /* ignored, but necessary for SPI bindings */  spi-max-frequency = <4000000>;
  /* WS2812 */  chain-length = <10>; /* number of LEDs */  spi-one-frame = <0x70>;  spi-zero-frame = <0x40>;  color-mapping = <LED_COLOR_ID_GREEN                   LED_COLOR_ID_RED                   LED_COLOR_ID_BLUE>;};
note

Standard WS2812 LEDs use a wire protocol where the bits for the colors green, red, and blue values are sent in that order. If your board/shield uses LEDs that require the data sent in a different order, the color-mapping property ordering should be changed to match.

Display Selection#

Zephyr moved to using a chosen node named zephyr,display to select the display device to be used with LVGL, the underlying display library we use.

For example, for a shield with:

&pro_micro_i2c {  status = "okay";
  oled: ssd1306@3c {    compatible = "solomon,ssd1306fb";    reg = <0x3c>;    label = "SSD1306";    width = <128>;    height = <32>;    segment-offset = <0>;    page-offset = <0>;    display-offset = <0>;    multiplex-ratio = <31>;    com-invdir;    segment-remap;    com-sequential;    prechargep = <0x22>;  };};

You would add a chosen node like:

/ {  chosen {    zephyr,display = &oled;  };};

USB Logging#

Zephyr unified the way the console/logging device is selected, removing the hacks that special-cased the USB CDC ACM output. Now, the CDC ACM device is configured in the devicetree as well. To ensure that USB logging properly works with custom board definitions, two sections of the <board>.dts file need updating.

Underneath the USB device, add the CDC ACM node:

&usbd {  status = "okay";  cdc_acm_uart: cdc_acm_uart {    compatible = "zephyr,cdc-acm-uart";    label = "CDC_ACM_0";  };};

Then, an additional chosen node (near the top of the file) will mark the CDC ACM device as the console:

/ {  chosen {    ...    zephyr,console = &cdc_acm_uart;  };  ...};

UF2 Builds#

Previously, to get ZMK to build a UF2 image to flash to a given board required adding a CMakeLists.txt file that added a custom post build command. Now, the only thing necessary to have Zephyr build a UF2 is to add the following to your <board>_defconfig file:

CONFIG_BUILD_OUTPUT_UF2=y

If updating an existing board, be sure to remove the previous CMakeLists.txt file to avoid generating the UF2 twice during a west build.

For more details on the implementation, see zephyr#31066.

STM32 Clock Configuration#

Clock configuration moved to devicetree as well, out of the Kconfig files. Here is a sample config for a board that uses the HSI for the PLL source:

&clk_hsi {  status = "okay";};
&pll {  prediv = <1>;  mul = <12>;  clocks = <&clk_hsi>;  status = "okay";};
&rcc {  clocks = <&pll>;  clock-frequency = <DT_FREQ_M(72)>;  ahb-prescaler = <1>;  apb1-prescaler = <2>;};

After adding the nodes, be sure to remove the clock/PLL related configuration from the <board>_defconfig file.

Seeeduino XIAO#

The Seeed(uino) XIAO has gained in popularity for use on smaller boards, and gained more traction with the release of the new XIAO BLE board, powered by the popular nRF52840 SoC. As part of the 3.0 update, we've also more fully integrated the XIAO and XIAO BLE to make it easier to build keyboard (shields) using either controller.

Future Hardware#

One of the exciting items that's one step closer as part of this work is support for Raspberry Pi Pico/RP2040. With Zephyr 3.0 merged, this start the process for getting those controllers/chips supported by ZMK. Follow the issue to keep track of progress. This will also enable us to support the XIAO compatible Adafruit Qt Py RP2040 and XIAO RP2040.

Thanks!#

Thanks to all the testers who have helped verify ZMK functionality on the newer Zephyr version.

Zephyr 3.0 Update Preparation

路 One min read
Pete Johanson
Pete Johanson
Project Creator

As preparation for completing the work to upgrade ZMK to Zephyr 3.0, users with user config repositories who wish to avoid future build failures with their GitHub Actions workflows can take steps to adjust their repositories now.

GitHub Actions needs to use our latest Docker image to ensure continued compatibility with the ZMK codebase on Zephyr 3.0 (and beyond). You should:

  • Open .github/workflows/build.yml in your editor/IDE
  • Change zmkfirmware/zmk-build-arm:2.5 to zmkfirmware/zmk-build-arm:stable wherever it is found

Once the changes are committed and pushed, the build will run as expected.

A future blog post will outline the complete Zephyr 3.0 changes once that work is finalized.

note

If you created your user config repository a while ago, you may find that your build.yml file instead references a zephyr-west-action-arm custom GitHub Action instead. In this case, the upgrade is not as direct. We suggest that instead you re-create your config repository to get an updated setup using the new automation approach.

Zephyr 2.5 Update

路 3 min read
Pete Johanson
Pete Johanson
Project Creator

I'm happy to announce that we have completed the work to upgrade ZMK to Zephyr 2.5!

A big part of this work was some major refactors and improvements by innovaker to our zmk-docker Docker image and GH Actions automation.

  • Faster build times with improved caching.
  • Integration tests which automatically verify new images.
  • PRs to the repo now build properly and run the tests as well.
  • Build images for multiple target architectures, e.g. zmk-build-riscv64, all in parallel.
  • Nightly builds to be sure we're pulling in the latest OS/package updates, to ensure we keep our images up to date, address any reported vulnerabilities, etc.
  • Faster upgrade paths for future Zephyr SDK and Zephyr versions.

In addition, petejohanson did the upgrade work to adjust ZMK for the Zephyr changes.

  • Updated to newer devicetree/driver Zephyr API
  • Adjustment for Zephyr pinmux changes
  • Fixes for power management, LVGL, and formatter changes

Getting The Changes#

Use the following steps to update to the latest tooling in order to properly use the new ZMK changes:

User Config Repositories Using GitHub Actions#

Existing user config repositories using Github Actions to build will pull down Zephyr 2.5 automatically, and should work, fine as is. However, to upgrade to the newer Docker image, you should:

  • Open .github/workflows/build.yml in your editor/IDE
  • Change zmkfirmware/zmk-build-arm:2.4 to zmkfirmware/zmk-build-arm:2.5 wherever it is found
note

If you created your user config repository a while ago, you may find that your build.yml file instead references a zephyr-west-action-arm custom GitHub Action instead. In this case, the upgrade is not as direct. We suggest that instead you re-create your config repository to get an updated setup using the new automation approach.

VS Code & Docker (Dev Container)#

If you build locally using VS Code & Docker then:

  • pull the latest ZMK main with git pull for your ZMK checkout
  • reload the project
  • if you are prompted to rebuild the remote container, click Rebuild
  • otherwise, press F1 and run Remote Containers: Rebuild Container
  • Once the container has rebuilt and reloaded, run west update to pull the updated Zephyr version and its dependencies.

Once the container has rebuilt, VS Code will be running the 2.5 Docker image.

Local Host Development#

The following steps will get you building ZMK locally against Zephyr 2.5:

  • Run the updated toolchain installation steps, and once completed, remove the previously installed SDK version (optional, existing SDK should still work)
  • pull the latest ZMK main with git pull for your ZMK checkout
  • run west update to pull the updated Zephyr version and its dependencies

From there, you should be ready to build as normal!

Thanks!#

Thanks again to innovaker for all the hard work, and to all the testers who have helped verify ZMK functionality on the newer Zephyr version.

ZMK State Of The Firmware #4

路 7 min read
Pete Johanson
Pete Johanson
Project Creator

Welcome to the fourth ZMK "State Of The Firmware" (SOTF)!

This update will cover all the major activity since SOTF #3.

Recent Activity#

Here's a summary of the various major changes since last time, broken down by theme:

Keymaps/Behaviors#

Since last time, there have been several new powerful keymap features and behaviors added, including one of the most asked for features, combos!

Combos#

The initial combos work has landed! The amazing okke-formsma has once again delivered another powerful feature for ZMK. Combos are "position based", and are configured in a new toplevel node next to they keymap node in user's keymap files.

An example, that would send the ESC keycode when pressing both the first and second positions on your keyboard:

/ {    combos {        compatible = "zmk,combos";        combo_esc {            timeout-ms = <50>;            key-positions = <0 1>;            bindings = <&kp ESC>;        };    };};
note

Combos currently are "global", and not scoped to a given active layer. There is future planned work to allow enabling a certain combo for only certain active layers.

Sticky Keys (One-Shot Mods/Layers) Behavior#

okke-formsma also contributed the initial "sticky keys" behavior, which can be used for functionality sometimes called "one shot mods" or "one shot layers". In your keymap, this would like like:

&sk LEFT_CONTROL

for a sticky key/modifier, or:

&sl NAV

for a sticky layer.

&to Layer Behavior#

mcrosson contributed the new &to layer related behavior. This can be used to completely replace the active layer with a new one.

This is most frequently used when using multiple core base layers with different layouts, e.g. QWERTY and DVORAK, to switch between them.

Grave Escape Behavior#

okke-formsma added an implementation of the "Grave Escape" behavior, developing a more generic "mod-morph" behavior to do so. Adding

&gresc

to your keymap will send ESC when pressed on its own, but will send ` when pressed with a GUI or Shift modifier held.

RGB Underglow Color Selection#

mcrosson updated the RGB Underglow behavior to allow binding an explicit color selection to a key position.

Keymap Upgrader#

joelspadin completed the Keymap Upgrader which can be used to update your keymap to using the latest supported codes, and move away from the old deprecated codes.

If you've made keymap customizations, please make sure to run your keymaps through the upgrader, since the old deprecated codes will be removed in a future version of ZMK.

Displays#

There has been lots of work to get display support complete enough for use by end users. Although not quite ready for prime time, it is incredibly close, and we are looking forward to having the last few items completed and the feature documented!

Idle Blanking#

petejohanson added idle blanking for displays, which ensures they will go blank, and into low power mode, after a short period of inactivity from the user. This ensures we avoid burn-in for OLEDs, and helps improve battery life.

Battery and Output Widgets#

petejohanson implemented the first two complete, dynamic "widgets" for the displays for ZMK, adding a small battery indicator, which includes charging status, and a small output indicator, showing the currently active output (USB or BLE). When using BLE, the indicator also shows the active profile slot, as well as if the profile slot is open, awaiting connection from the paired host, or is actively connected to the host for that profile slot.

Highest Layer Display#

mcrosson has contributed the next display widget, showing the highest active layer in the keymap. petejohanson then added a small follow up to allow layers in keymaps to add a label property to each layer, e.g. label = "Nav"; and have that label be displayed in the widget instead of the numeric layer number.

WPM#

New contributor allymparker added our fourth widget, a words-per-minute display! This widget work also included creating the core state logic for tracking the WPM.

For now, this widget is only working on the central side of split keyboards.

Miscellaneous#

Zephyr 2.4#

innovaker is at it again with some crucial core fixes, helping prepare and test the upgrade of ZMK to Zephyr 2.4. The updated Zephyr release brings with it some key BLE stability fixes, as well as various other core improvements that improve ZMK. This was a huge undertaking!

BLE Deadlock Fixes#

petejohanson was heads down diagnosing and fixing a deadlock issue on BLE that was frustrating and plaguing many users. After finally pinpointing the underlying root cause, he developed a fix and roped in many testers on Discord to help stress test things before merging.

Central/Peripheral Selection#

Previously overriding the selection of left as central, and right as peripheral for wireless splits required making local edits to the configuration files, and maintaining them in a ZMK fork.

petejohanson updated the config files to allow users to override this in their <board>_left.conf/<board>_right.conf files in their user repos.

Improved Docker Containers#

As part of the Zephyr 2.4. prep work, innovaker, along with lots of testing and input from mcrosson, developed a brand new pair of Docker images which is now published to Docker Hub as zmkfirmware/zmk-build-arm and zmkfirmware/zmk-dev-arm.

The previously blogged VSCode + Docker integration, as well as our GH Action build automation was all moved over to the new images.

Settings Debounce#

nicell contributed settings debounce work, to help avoid unnecessary extra writes to flash when making various changes that should be saved, such as the active BLE profile, external VCC on/off, etc.

New Shields#

New Boards#

Sponsorship#

Since it's inception, quite a few users have inquired whether they could sponsor any of the contributors involved in ZMK. Although we are not intending to directly fund any individual contributors for their work on ZMK, there is good that can come from folks sponsoring ZMK.

You can see the full discussion on #497, but some items that are being considered with sponsorship funds:

  • Hiring a designer to complete the logo/mascot work.
  • Creating stickers to send as thank-yous to first time contributors.
  • Hosting costs for GitHub Pro.
  • Other hosting costs, e.g. Docker Hub.

For anyone looking to contribute, you can find the ZMK Firmware project is now set up on Open Collective.

Coming Soon!#

Some items listed in the last coming soon section are still under active development.

  • A power profiler page for the website, to help users estimate their battery life for a given keyboard - Nicell
  • Behavior "locality", allowing improved split usage for things like &reset, and controlling external power and RGB underglow for both sides - petejohanson
  • More modular approach to external boards/shields, custom code, user keymaps, etc.
  • More shields and boards

Statistics#

Some statistics of interest for ZMK:

  • GitHub (lifetime stats)
    • 389 Closed PRs
    • 199 Stars
    • 163 Forks
  • Discord Chat
    • 702 total registered
  • Website (last 30 days)
    • 11.5K page views
    • 1K new users

Thanks!#

Thanks again to the numerous contributors, testers, and users who have made working on ZMK such a pleasure!

ZMK State Of The Firmware #3

路 6 min read
Pete Johanson
Pete Johanson
Project Creator

Welcome to the third ZMK "State Of The Firmware" (SOTF)!

This update will cover all the major activity since SOTF #2. This edition comes a bit later than planned, but the amount of features and changes will hopefully make it worth it!

Recent Activity#

Here's a summary of the various major changes since last time, broken down by theme:

Keymaps/Behaviors#

Tons of activity related to keymaps, so we'll go into more detail this time.

Codes Overhaul#

innovaker completely overhauled the set of available codes for keymaps, and simultaneously has created beautiful documentation to help users visualize the codes, and also understand if they are supported on their particular operating system.

This also laid the foundation for the other keymap related changes that are now available.

Modified (Shifted) Codes#

okke-formsma added the ability to apply modifiers to a code, e.g.:

&kp LC(C)

which sends Control + c when pressed. This feature is often used on smaller keyboards to achieve "shifted keycodes", e.g. LS(N1) to send a !. To make this easier, in addition to all the normal codes, we now have defines for common shifted codes, e.g. EXCL for !, AT for @, etc.

To learn more, check out the Modifiers documentation.

Simplified Key Press Behavior#

In previous versions of ZMK, users needed to be careful to select between the &kp and &cp behaviors in their keymaps, depending on whether the particular keycode they wanted to send was in the "HID consumer page" or not. Forcing users to understand the difference and get this right was awkward and error prone.

petejohanson and innovaker have reduced this complexity. Users can now simply use &kp with all available codes and ZMK will handle sending the right events to the connected host.

Power Management#

Several important power management features have been added to ZMK, helping save power for many use cases.

BLE Battery Level Reporting#

Nicell added the necessary driver and core code to send BLE battery level notifications to hosts that support displaying them. Testing seems to show this works with Windows and GNOME, but macOS does not display the battery info.

External Power Control#

megamind4089 added a new driver and behavior to allow users to toggle (on/off) the external power supplied by boards such as the nRFMicro and nice!nano that have specialized hardware for this purpose.

With this change, you can add

&ext_power EP_TOG

to toggle (on/off) the power to external hardware like RGB underglow or OLEDs. Check out the external power control docs for more info.

Deep Sleep#

petejohanson has contributed the initial deep sleep support to ZMK. This work also included some automatic power savings by switching to PORT events on the nRF52 chips, which reduces the idle power draw, even without deep sleep. Deep sleep is currently not turned on by default, but will be soon.

Miscellaneous#

Output Selection#

joelspadin added output selection to allow selecting whether to send output over USB or BLE if both are connected. This should now help avoid having "double keypresses" when your keyboard is plugged into a host.

Bootloader Corruption Fix#

Nicell has already blogged about this, but for those that missed it, a major, and incredibly difficult to pin down bug involving corruption of the bootloader on devices using the Adafruit nRF52 bootloader has been fixed by Nicell. If you've encountered this bug, flashing the latest firmware should prevent it from reoccurring. Unfortunately, due to the nature of this fix, you will need to re-pair your keyboard with your hosts, as the fix involves changing where settings are stored in the flash of the controller.

Official USB Product ID#

petejohanson has gotten an official USB product ID assigned to the ZMK Firmware. For anyone looking to uniquely identify a USB device running the ZMK Firmware, you can match on:

  • Vendor ID: 0x1d50
  • Product ID: 0x615e

We are incredibly grateful that Openmoko Inc., in the wake of discontinuing the openmoko projects, has made this an option for OSS projects.

Development: Remote Docker Container Integration#

idan contributed VSCode devcontainer integration to make it easier for developers to build and develop ZMK without having to do complicated local toolchain setup and configuration. This also opens up some amazing future flexibility for things like GitHub Codespaces.

There's some follow up tweaks necessary for better supporting using this with user config repositories, which will be available soon.

New Shields#

New Boards#

Testing#

There has been an amazing amount of testing from various users as we develop new features. In particular, we'd like to give a shout out to tominabox1 who has been tireless in providing detailed and thorough testing of changes as they are being developed.

Coming Soon!#

Some items listed in the last coming soon section are still under active development.

  • OLED work, including battery and USB/BLE connection status - petejohanson
  • One shot mod/layer behaviors - okke-formsma
  • A power profiler page for the website, to help users estimate their battery life for a given keyboard - Nicell
  • A keymap converter to automatically update keymaps to the new codes and use of &kp everywhere - joelspadin

Statistics#

Some statistics of interest for ZMK:

  • GitHub (lifetime stats)
    • 210 Closed PRs
    • 116 Stars
    • 101 Forks
  • Discord Chat
    • 363 total registered
  • Website (last 30 days)
    • 8.5K page views
    • 766 new users

Thanks!#

Thanks again to the numerous contributors and users who have made working on ZMK such a pleasure!

Fixing the Mysterious Broken Bootloader

路 8 min read
Nick Winans
Nick Winans
Contributor

Recently I was able to fix the "stuck in the bootloader" issue in #322 that had been plaguing us for quite some time. I want to go over what the issue was, how the issue was diagnosed, and how it was fixed.

Background#

What exactly is the "stuck in the bootloader" issue? Seemingly randomly, users' keyboards would suddenly stop working and when they would reset their keyboard they would get put into the bootloader instead of back into the firmware. This would require the user to re-flash the firmware again to get into the firmware. That wouldn't be so bad except for the fact that once this occurs, every reset would require the user to re-flash the firmware again. The only way to really fix this issue was to re-flash the bootloader itself, which is a huge pain.

Going into this, all we knew was that this issue was most likely introduced somewhere in the #133, which added Bluetooth profile management. We've had quite a few attempts at trying to recreate the issue, but we never were able to get it to happen consistently.

Diagnosing the issue#

This issue had been happening sporadically for the past month, and I finally decided to dig in to see what was going on. We started in the Discord and discussed what was common between all of the people who have experienced this issue. Everyone who had this issue reported that they did quite a bit of profile switching. This lined up with the possible connection to the Bluetooth profile management pull request.

Pinpointing the cause#

I had a hunch that this was related to the settings system. The settings system is used by profile Bluetooth switching, and the settings system works directly with the system flash. Based on this hunch, I tried spamming the RGB underglow cycle behavior on my main keyboard. Sure enough after a couple minutes, I got stuck in the bootloader. I was even able to reproduce it again.

This was an important discovery for two reasons. First, I was able to recreate the issue consistently, which meant I could set up logging and more closely monitor what the board was doing. Second, this more or less proved that it was specifically the settings system at fault. Both Bluetooth profile switching and RGB underglow cycling trigger it, and the one common piece is they save their state to settings.

Settings system overview#

To understand what's going wrong, we first need to understand how the settings system works. Here's a diagram to explain the flash space that the settings system holds for our nRF52840 based boards (nice!nano, nRFMicro, BlueMicro).

Settings Diagram

The settings flash space lives at the end of the flash of the chip. In this case it starts at 0xF8000 and is 0x8000 bytes long, which is 32KB in more comprehensible units. Then due to the chip's architecture, this flash space is broken into pages, which are 0x1000 bytes in size (4KB).

The backend that carries out the settings save and read operation in ZMK is called NVS. NVS calls these pages sectors. Due to how flash works, you can't write to the same bytes multiple times without erasing them first, and to erase bytes, you need to erase the entire sector of flash. This means when NVS writes to the settings flash if there's no erased space available for the new value, it will need to erase a sector.

Logging discoveries#

So first I enabled logging of the NVS module by adding CONFIG_NVS_LOG_LEVEL_DBG=y to my .conf file. I repeated the same test of spamming RGB underglow effect cycle and the resulting logs I got were this:

[00:00:00.000,671] <inf> fs_nvs: 8 Sectors of 4096 bytes[00:00:00.000,671] <inf> fs_nvs: alloc wra: 3, f70[00:00:00.000,671] <inf> fs_nvs: data wra: 3, f40// A bunch of effect cycle spam[00:02:34.781,188] <dbg> fs_nvs: Erasing flash at fd000, len 4096// A bunch more effect cycle spam[00:06:42.219,970] <dbg> fs_nvs: Erasing flash at ff000, len 4096// A bunch more effect cycle spam// KABOOM - bootloader issue

So at start up, we can see that the 8 sectors of 4KB are found by NVS properly, however, I wasn't sure what the second and third lines meant, but we'll get back to that. Nonetheless the next two logs from NVS showed erasing the sector at 0xFD000 and then erasing the 0xFF000 sector.

Erased Sectors

It's really odd that the third to last sector and the last sector are erased, and then shortly after the bootloader issue is hit. I really had no explanation for this behavior.

Reaching out to Zephyr#

At this point, I nor anyone else working on the ZMK project knew enough about NVS to explain what was going on here. Pete Johanson, project founder, reached out on the Zephyr Project's Slack (ZMK is built on top of Zephyr if you weren't aware). Justin B and Laczen assisted by first explaining that those alloc wra and data wra logs from earlier are showing what data NVS found at startup.

More specifically, data wra should be 0 when it first starts up on a clean flash. As we can see from my earlier logging on a clean flash I was instead getting f40. NVS is finding data in our settings sectors when they should be blank! We were then given the advice to double check our bootloader.

The Adafruit nRF52 Bootloader#

Most of the boards the contributors of ZMK use have the Adafruit nRF52 Bootloader, which allows for extremely easy flashing by dragging and dropping .uf2 files onto the board as a USB drive. Every bootloader takes up a portion of the flash, and in the README explains that the first 0x26000 is reserved for the bootloader with the nRF52840, and we've properly allocated that.

However, there isn't a full explanation of the flash allocation of the bootloader in the README. There's a possibility that the bootloader is using part of the same flash area we're using. I reached out on the Adafruit Discord, and Dan Halbert pointed me towards the linker map of the nRF52840. Let's take a look.

FLASH (rx) : ORIGIN = 0xF4000, LENGTH = 0xFE000-0xF4000-2048 /* 38 KB */
BOOTLOADER_CONFIG (r): ORIGIN = 0xFE000 - 2048, LENGTH = 2048
/** Location of mbr params page in flash. */MBR_PARAMS_PAGE (rw) : ORIGIN = 0xFE000, LENGTH = 0x1000
/** Location of bootloader setting in flash. */BOOTLOADER_SETTINGS (rw) : ORIGIN = 0xFF000, LENGTH = 0x1000

Here's a diagram to show this a bit better.

Adafruit Bootloader Diagram

We've found the issue! As you can see from the red bar (representing our settings flash area), we've put the settings flash area right on top of the Adafruit bootloader's flash space. Oops!

This also shines some light on why NVS erased 0xFD000 and 0xFF000 sectors. It's possible there was no flash written to 0xFD000 because the bootloader didn't use up all of that space it has, and then there possibly weren't any bootloader settings set yet, so 0xFF000 could be used and erased by NVS too.

After erasing 0xFF000, NVS probably next erased a rather important part of the bootloader that resulted in this issue at hand. In my opinion, we're pretty lucky that it didn't delete an even more vital part of the bootloader. At least we could still get to it, so that we could re-flash the bootloader easily!

The solution#

Now that we've found the issue, we can pretty easily fix this. We'll need to move the settings flash area back so that it doesn't overlap with the bootloader. First we calculate the size of the of flash area the bootloader is using.

0x100000 (end of flash) - 0x0F4000 (start of bootloader) = 0xC000 (48KB)

So the bootloader is using the last 48KB of the flash, this means all we need to do is shift back the settings area and code space 0xC000 bytes. We'll apply this to all of the .dts files for the boards that were affected by this issue.

        code_partition: partition@26000 {            label = "code_partition";-           reg = <0x00026000 0x000d2000>;+           reg = <0x00026000 0x000c6000>;        };
-       storage_partition: partition@f8000 {+       storage_partition: partition@ec000 {            label = "storage";-           reg = <0x000f8000 0x00008000>;+           reg = <0x000ec000 0x00008000>;        };

And with those changes, we should no longer run into this issue! In the process of these changes, we lost 48KB of space for application code, but we're only using around 20% of it anyways. 馃帀

ZMK State Of The Firmware #2

路 3 min read
Pete Johanson
Pete Johanson
Project Creator

Welcome to the second ZMK "State Of The Firmware" (SOTF)!

This update will cover all the major activity since SOTF #1, preparations for the upcoming Hacktoberfest activity, and a current open call for community feedback on a ZMK mascot.

Recent Activity#

So much going on in ZMK!

New Shields#

New Boards#

Hacktoberfest Preparation#

Hacktoberfest is a yearly celebration of open source, which encourages participation in OSS, especially from new contributors.

The ZMK contributors have been busy preparing for folks to join in on the fun by contributing to ZMK!

  • There is now a basic Contributing Guide to help newcomers get oriented, and get up to speed.
  • The Hacktoberfest issue label will help participants discover good issues to work on. (The existing good first issue label also helps with this)

We're looking forward to the launch of Hacktoberfest!

Mascot Selection Feedback#

The ZMK project would like to settle on a mascot! We're soliciting community feedback as part of the process before a final mascot is selected.

The current mascots up for consideration are:

  • Griffin
  • Peregrine Falcon
  • Zapata Wren
  • Zorro (south american fox)

If you're interested in helping with the decision, head over to Issue #195 and add a reaction!

Coming Soon!#

There still lots of activity in ZMK, and plenty of exciting upcoming changes.

Statistics#

Some statistics of interest for ZMK:

  • GitHub
    • 115 Closed PRs
    • 64 Stars
    • 48 Forks
  • Discord Chat
    • 186 total registered
  • Website (last 30 days)
    • 7.4K page views
    • 474 new users

Thanks!#

Thanks again to the numerous contributors and users who have made working on ZMK such a pleasure!

ZMK State Of The Firmware #1

路 3 min read
Pete Johanson
Pete Johanson
Project Creator

Welcome to the first ZMK "State Of The Firmware"!

With interest and Discord activity growing, it seemed important to lay out the progress made recently, current major bugs/showstoppers, and planned next steps.

Recent Activity#

There's been lots of various activity in ZMK land!

  • Nicell (nice!nano creator) contributed initial RGB Underglow (#64) support to ZMK.
  • Tons of documentation work.
  • Refactoring (#73, #74) of keymaps to make them simpler for users.
  • Mod-Tap Behavior (docs coming!) is much improved (#69) and usable now.
  • An initial setup.sh script was created, allowing users to quickly bootstrap a "user config" setup and push it to GitHub, where GitHub Actions will build the firmware for you.
  • Corne shield (#80) shield definition was added.
  • Initial encoder support (#61) was added.

Bugs and Showstoppers#

Despite the flurry of activity, there are still some serious bugs and show stoppers that potential ZMK users should be aware of:

  • Bluetooth Related - There are several key bugs and fixes needed, including one complete show stopper:
    • Fully working split wireless is not working. In particular, both split halves can properly pair, but once they do so, pairing with the _central_ host will not work. Workaround: You can currently have both halves pair, and use USB on the central side (Left side right now for Kyria, Corne, Lily58) and receive HID events over USB. - Fixed in 8b61beb.
    • BT bond information is not currently stored to the devices, so after powering off or restarting your device, users need to re-pair
  • USB - There is one important USB related bug which is a showstopper:
    • The Zephyr USB stack does not have a built in queue for endpoint data being written. As a result, HID events sent by ZMK are sometimes dropped, or lost. - Fixed by careyk007 in #93.

Next Steps#

There's plenty of places to go next! To help keep folks in the loop for what's next, I've created a Core Functionality project/kanban board in GitHub, where users should be able to get some visibility into items being focused on.

Of course, at the top of that list currently is the above bugs/showstoppers, and then from there, I hope to:

  • Work on power management.
  • Improve our documentation on various aspects of the system, mostly around:
    • End user documentation for understanding how to use ZMK, better installation docs, etc.
    • Developer focused documentation, so interested contributors can start building out more behaviors and ZMK functionality.
  • Implement true "hold" detection, useful for several behaviors such as Mod-Tap and Layer-Tap.
  • Hopefully acquire a proper and official USB Product ID for use for the project.
  • Fun things that come up along the way!

Thanks!#

A big thanks for everyone who has shown interest in the project, tested things, asked questions, and contributed PRs (Nicell, CrossR, careyk007).

WIP

路 One min read
Pete Johanson
Pete Johanson
Project Creator

This blog is a work-in-progress as I work on basic docs + blog on this nascent keyboard firmware.

As is, there are more features missing from ZMK than features it has. As always with pre-alpha software, if something breaks, you get to keep both halves! (especially if it is a split KB)