A few months ago, I completed a project to build an entire USB keyboard from scratch. This included electronic circuit design, PCB design, firmware coding, CAD design, assembly and usage. The final result is my daily driver work keyboard, which I affectionately call “KeeBee”:

A few project goals:

1. Implement the circuit myself
2. Write the keyboard firmware
3. Learn about how the USB protocol works

For my day job, I spend most of my time building cloud software that’s many layers removed from real running hardware. It’s extremely safisfying to peel back some of the abstractions, and get closer “to the metal” building real electronic devices I can physically touch and use.

Research & CAD Design

I knew I really liked the OLKB Planck and Preonic style boards. They feature a nice minimal ortho-linear layout, that’s very compact. I also knew that I wanted to use Cherry MX Brown switches. With those two design components in mind, I started playing around with key layouts in OpenSCAD. OpenSCAD is a great open source CAD design tool that functions more like a programming language than a WYSIWYG point and click mouse CAD tool.

Using the dimensions from a Cherry MX datasheet, I hacked up a keyboard plate design, and then added switches and keycaps to get a feel for what the layout would look like. The top plate sits above the keyboard PCB and serves as a good switch stabilizer.

Top plate design:

With keycaps added:

Prototype Circuit and Firmware Design

I chose an STM32F042K6T6 as the main keyboard microcontroller. It’s around $3 per chip in individual quantities, and has just enough pins to implement a 69 key scan matrix (32 pins in total). It sports an ARM Cortex M0 processor, and has a dedicated USB peripheral for sending out USB bits without tying up the main processor bit-banging out USB signals. I bought a Nucleo prototype development board of this chip for experimenting with the chip before I integrated it into my PCB design. The Nucleo was easy to use on a breadboard, and power directly with USB.

I breadboarded out a small 4 key circuit, to test out the diode based circuit I had researched. Ignoring the USB side of the equation, The first step was to just get the Cherry switches to reliably turn the 4 corresponding LEDs on and off when the button is pressed.

Keyscan Matrix circuits are a technique to use when you have more switches than you have pins on your microcontroller.

After I got the keyscan matrix implementation to my liking, it was time to work on the USB side of the equation.

The inner loop for the firmware is basically:

1. Scan all keys in the button keyscan matrix.
2. Map the button locations to their respective key symbols, using the currently selected layout (QWERTY, Dvorak, etc.)
3. Take the mappings, and generate USB HID Report packets and send them out the USB peripheral.
4. Set an LED on the keyboard to on if a key is pressed, off if not.

From main.cc:

static void scan_and_update() {
    scan_matrix.Scan(key_scans, row_count, column_count);
    keyboard.SendReport(
        key_pipeline.MapKeyScans(key_scans, key_count));
    update_key_press_status();
}

int main() {
    Init();

    status_led.SetOk(true);
    while (true) {
        scan_and_update();
    }
}

The keyboard.SendReport component is the piece that actually sends the USB packets to the host. I struggled a lot to get USB working correctly. There are a lot of finnicky layers to the USB protocol that require accurate timing, and correct device identification. I ended up needing to fire up Wireshark to sniff USB packets coming back and forth to my Linux laptop, in order to debug where things were getting lost on the wire. Most of my googling was pretty useless at this part of the build: suggestions I found had suggestions like, “You probably have a faulty USB device, you should get a new one.” When you’re the one actually trying to build the USB device, this isn’t really helpful. I was left reading through very large USB specifications that contain a lot of terminology that was pretty unfamiliar to me.

After mucking about for awhile, I was able to get the 4 key keyboard to correctly identify itself as a USB HID (Human Interface Device) to my laptop, and make sure all my key presses were being correctly mapped to the machine:

Getting a USB vendor and device id requires paying a good bit of cash, so if you’re just doing something as a hobby, you’ll need to hijack a similar device ID. I thought “Gear Head” sounded cool, and they make a keyboard, so I went with that one.

Schematic and PCB Design

With some working firmware on a working prototype, it’s time to put the schematic and PCB design into KiCAD and get an actual printed circuit board made. Now that I had proven the schematic design worked, it was relatively straight-forward to connect everything together schematic wise:

After building out the schematic and selecting part footprints, we need to layout the actual physical PCB:

KiCAD has a neat feature that lets you preview your PCB in 3D:

There are lots of great tutorials on how to use KiCAD. I started with Chris Gammell’s excellent Getting to Blinkey 4.0 youtube series, where he takes you through building a LED blinker PCB circuit in KiCAD from start to finish.

PCB and Component Ordering

Once I was reasonably satisfied with the schematic and PCB design, I started placing a bunch of orders:

1. All the board components from the keyboard’s Bill of Materials: Switches, LEDs, diodes, microcontrollers, etc. I like to use DigiKey for most of my electronic components.
2. The PCB itself. There are a lot of really great PCB prototype manufacturing services out there that will do small run PCB fabrication for really cheap. I’ve had great experience with OshPark and JLCPCB. For this project, I went with JLCPCB because of the board size cost, and because they let me pick a blue solder-mask board.
3. Any other cases, etc. For this project, my brother in law was able to laser cut the top and bottom keyboard plates from 1/4” acryllic sheets. There are other great online laser cutting and 3D printing services for case components, if you don’t have access to the equipment.

PCB arrival day is the the best:

JLCPCB is very affordable. This design was less than $30 shipped DHL from China, and took a little over a week to arrive after submitting my gerber files for order.

My brother in law took my DXF files from OpenSCAD and tossed them in the laser cutter:

Final Assembly

With all the pieces ordered and fabricated, it was time to put the final keyboard together. I started with PCB component assembly: I used a soldering iron for the larger electronic components and a hot air rework station for the small surface mount components like the STM32 microcontroller.

Total component build time for a board was roughly 3 hours - most of the time was spent soldering the 70 diodes and switches.

I added a JTAG debugger pin header to the PCB, which I used to plug in a JLINK Edu mini to flash the microcontroller with the firmware with OpenOCD.

From there, it was final testing, and plate assembly:

My son thought it made a great train for his animals:

Conclusion

From initial idea to final assembly, this project took about 3 months time. It was extremely rewarding to make something that I still use everyday at work.

All the project files are up on GitHub, including firmware source code, PCB schematics, Bill of Materials, and CAD models.

Thanks for reading, and happy hardware hacking!

Placement new is a feature currently being discussed for the Rust programming language. It gives programmer control of memory allocation and memory placement, where current memory allocation implementations are hidden behind compiler internals via the Box::new interface. Controlling memory allocation is useful in many different applications, but a few that come to mind:

1. Arena allocators: Pre-allocating big chunks of memory from the operating system up front, and arbitrating memory ownership directly in your application. This helps avoid context switches into the kernel for certain memory use-cases.
2. Embedded allocators: Allocating chunks of memory from well-defined memory locations. Useful when you need memory to come from very specific hardware addresses.

This is Rust’s answer to C++ placement new, allowing one to control not only when and how memory is freed, but also where it is allocated and freed from (Thanks Holy_City for the clarification here!)

How heap allocation in Rust works now

Heap allocation is hidden behind the Box type in Rust stable. When you instantiate data within a Box type, you’re placing the data onto the heap, and the Box type stores an internal pointer to that heap allocated data.

// Count will be placed on the heap
let heap_data = Box::new(Count { num: 1 });

If you dig around in the source of the Box type, you can see some hints at why ‘placement new’ might be useful.

Let’s see how Box::new is implemented:

#[stable(feature = "rust1", since = "1.0.0")]
#[inline(always)]
pub fn new(x: T) -> Box<T> {
    box x
}

It looks like memory allocation, and lifting into the box type is hidden behind this box keyword. Searching up on the box keyword yields the Rust Documentation on unstable Box Syntax and Patterns. This unstable feature allows you to use the box keyword to instantiate allocated Boxes directly on the global heap.

What does the Drop implementation for Box look like?

#[stable(feature = "rust1", since = "1.0.0")]
unsafe impl<#[may_dangle] T: ?Sized> Drop for Box<T> {
    fn drop(&mut self) {
        // FIXME: Do nothing, drop is currently
        // performed by compiler.
    }
}

I was expecting to see some unsafe dealloc calls here, but it looks like this work is being done somewhere in compiler internals.

How Placement New Fits

If you need to customize how heap allocation works, ideally, you’d be able to hook into the box keyword or use similar syntax (and avoid C++’esque library solutions). This is exactly what Rust’s ‘placement new’ feature gives us. We need a way to use that fancy box syntax (or something similar), and implement our own memory placement strategies.

The work on ‘placement new’ is broken down into a few different efforts that need to come together to make all of this work:

• A syntax extension that allows the programmer to specify where they’d like the memory placed. (RFC#1228),
• A Placer trait that would allow custom allocation / placement implementations, that returns a Place type from the Placers required make_place function.
• Desugaring logic that transforms the syntax into straight forward Rust code that calls the correct Placers.
• Implementations of existing implicit allocation / placement strategies, including a BoxPlace for the default Box heap allocation strategy.

Placement New Examples

There’s no clear consensus on how placement new syntax will work yet, but there are many options being discussed in RFC#1228. A few different options being discussed:

Overloaded ‘box’ syntax

We could overload the previously mentioned box syntax above, and allow it to take a place expression:

let arena_ref = box arena 5;
let heap_ref = box HEAP 5;

Left Arrow Syntax

Left arrow syntax, follows the syntax PLACE_EXPR <- VALUE_EXPR:

let arena_ref = arena <- MyStruct::new();
let heap_ref = HEAP <- MyStruct::new();

Placement ‘in’ syntax

Placement ‘in’ syntax uses the ‘in’ keyword to define the placement location.

let arena_ref = in arena { MyStruct::new() };
let heap_ref = in HEAP { MyStruct::new() };

Placer examples

The Placer implementation for ExchangeHeapSingleton (the default box heap heap allocation method) implementation that was repealed from Rust looked something like this:

fn make_place<T>() -> IntermediateBox<T> {
    let layout = Layout::new::<T>();

    let p = if layout.size() == 0 {
        mem::align_of::<T>() as *mut u8
    } else {
        unsafe {
            Heap.alloc(layout.clone()).unwrap_or_else(|err| {
                Heap.oom(err)
            })
        }
    };

    IntermediateBox {
        ptr: p,
        layout,
        marker: marker::PhantomData,
    }
}

impl<T> Placer<T> for ExchangeHeapSingleton {
    type Place = IntermediateBox<T>;

    fn make_place(self) -> IntermediateBox<T> {
        make_place()
    }
}

And the corresponding Drop implementation:

impl<T: ?Sized> Drop for IntermediateBox<T> {
    fn drop(&mut self) {
        if self.layout.size() > 0 {
            unsafe {
                Heap.dealloc(self.ptr, self.layout.clone())
            }
        }
    }
}

In this case, a Heap type handles all the unsafe alloc / dealloc, and the Place returned from the Placer is of type IntermediateBox.

Current feature status

It’s not clear yet when all these things will land, especially given the uncertainty around the placement syntax. Some of the initial work that was commited to Rust unstable (including syntax extensions and the Placer protocol traits) was subsequently removed in light of further design discussions needing to be had. Either way, I think placement new is an important feature for Rust. Adding explicit (but not required) control points to the internals of Rust will make it more appealing for certain use cases, including embedded and other applications that have special memory control requirements. I’m very much looking forward to this feature landing in Rust nightly.

There are many things to like about the Dropwizard Framework, but if you’re like me, you might want to “own your main” function. A normal Dropwizard application gives you a run hook as part of its Application parent class, but in the end your code is still subservient to the Dropwizard framework code.

One pattern that is very good for organizing small logical services in your application is to use Guava’s Services to break your application into service level groupings (And avoid the drinking the microservice kool-aid prematurely). Guava services give you a common operational interface to coordinate all your in-process logical components. In this model, the Dropwizard web server is no more important than my periodic polling service, or my separated RPC service, and so on. I’d like my Dropwizard web stack to be a peer to the other services that I’m running inside my application. It only takes two steps to make this work.

Step 1: Create the Guava Service

Create a new class that extends AbstractIdleService. When we implement the startUp method in the service, we will need to handle the key bootstrap setup that normally is handled within the Dropwizard framework when you invoke the server command.

import com.codahale.metrics.MetricRegistry;

import com.google.common.util.concurrent.AbstractIdleService;
import com.google.inject.Guice;
import com.google.inject.Inject;

import com.google.inject.Injector;
import io.dropwizard.setup.Bootstrap;
import io.dropwizard.setup.Environment;

import org.eclipse.jetty.server.Server;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class DashboardService extends AbstractIdleService {
    private static final Logger logger = LoggerFactory.getLogger(DashboardService.class);
    private final DashboardApplication application;
    private final DashboardConfiguration config;
    private final MetricRegistry metrics;
    private Server server;

    @Inject
    public DashboardService(final DashboardApplication application,
                            final DashboardConfiguration config,
                            final MetricRegistry metrics) {
        this.application = application;
        this.config = config;
        this.metrics = metrics;
    }

    @Override
    protected void startUp() throws Exception {
        logger.info("Starting DashboardService");
        final Bootstrap<DashboardConfiguration> bootstrap = new Bootstrap<>(application);
        application.initialize(bootstrap);
        final Environment environment = new Environment(bootstrap.getApplication().getName(),
                                                        bootstrap.getObjectMapper(),
                                                        bootstrap.getValidatorFactory().getValidator(),
                                                        metrics,
                                                        bootstrap.getClassLoader());
        bootstrap.run(config, environment);
        application.run(config, environment);
        this.server = config.getServerFactory()
            .build(environment);
        server.start();
    }

    @Override
    protected void shutDown() throws Exception {
        logger.info("Stopping DashboardService");
        server.stop();
    }
}

Step 2: Reinitialize logging inside the Dropwizard Application

public class DashboardApplication extends Application<DashboardConfiguration> {
    @Override
    public void run(DashboardConfiguration config,
                    Environment environment) {
        reinitializeLogging(environment);
    }

    /**
     * Because Dropwizard clears our logback settings, reload them.
     */

    private void reinitializeLogging(Environment env) {
        LoggerContext context = (LoggerContext) LoggerFactory.getILoggerFactory();
        try {
            JoranConfigurator configurator = new JoranConfigurator();
            configurator.setContext(context);
            context.reset();
            String logBackConfigPath = System.getProperty("logback.configurationFile");
            if (logBackConfigPath != null) {
                configurator.doConfigure(logBackConfigPath);
            }
        } catch (JoranException e) {
            throw new RuntimeException("Unable to initialize logging.", e);
        }
    }
}

Start your Guava Service

Now you have a nicely contained Guava service that you can manage alongside your other Guava service that aren’t necessarily Dropwizard related. You can startAsync and stopAsync the service to start and stop the web server, even while other services are still running.