# hp-mark Exists Now

15-2-2021

I climbed down a rabbit hole recently. Nearly half a year working on hp-mark. The past 84 days even skipping blog post and newsletter writing.

Progress has been steady. It works now. hp-mark measures positions, like:

Accuracy is not fantastic. The image above is a lucky one, with only 3 mm of error. A more commonly seen error is around 35 mm.

A lot of hp-mark work remains. I ticked 2 boxes, and added 6 new ones in the roadmap today:

Aquire camera 6D pose (this includes defining our world coordinate system)
Aquire effector 6D pose
Detect all markers on 95% of training images
Take image ourselves upon request, don't rely on other programs to take image first
Create a continous stream of position measurements (video?)
Get a statistical idea about size of error
Respond to RepRapFirmware/Duet request for position measurement
Integrate a second camera, to reduce error

I'm a bit overwhelmed still. I need to look backwards a bit.

### What Have I Done?

I've been surprised.

I chose C++, OpenCV, and a very traditional approach for this project. I just wanted the most basic code that would mecanically find circles among pixels, and then apply static geometric equations to transform circles into a nozzle position. No AI stuff, no GPU stuff, no fancy hardware control, just $$f(x) \rightarrow y,$$ where $$x$$ is an image, and $$y$$ is a nozzle position.

Such an old and explored problem! I felt confident.

Some things that took long for me to program, in descending order:

• Good enough ellipse detection. OpenCV's SimpleBlobDetector was ok, but ultimately not robust enough for our purposes. I was lucky to find ED_Lib, and spent a little over a month integrating it into hp-mark. For example, I had to fight a little to get sub-pixel accuracy in the center and size data. More about that here.
• Geometry. A photon bounces off a sphere, through a pinhole, and onto an image sensor. Many more follow. They create a pattern, a projection, on the image sensor. Given the position of the sphere, what's the equation of its projection? Or vice versa; given a projection, what's the position of the sphere? Which projected point correspond to the sphere's center? I derived all relevant equations by hand and wrote tests that wouldn't go green before I was exactly right. To bootstrap the chain of testable geometric functions, I derived the most basic equation in two different ways, and tested that the two derived functions gave exactly the same results. The system would never be good if the theory it rested upon wasn't sound, so I spent the time to make this perfect.
• Filtering among ellipses. The improved ellipse detector found lots of ellipses. Getting hp-mark to determine which ellipses represent real markers, was hard. To be real useful, hp-mark must be able to identify all its markers almost all the time, so I spent extra time here as well. I will return to this point in the future, but we're at least finding all the markers ca 80% of the time now.
• Color recognition. The human brain is full of heuristics that helps us determine color, in a wide variety of situations. Very fine details of camera settings, light/shadow, and way of definition, throws a simplistic computer program off very easily. The strongest heuristic I was able to program, in which all colors are relative, still can not identify a non-marker in a set of six real and one false marker. Still, I was able to keep color as a marker categorizer that lets us avoid having to put QR-codes on our markers. I fought a bit for the aesthetics.

I feel like I'm almost too tired of it to even write about this stuff now. But before I climb out of this rabbit hole, I can show you something about how the ellipse detector works internally:

In the image above, we see that one marker was missed, despise ED_Lib's many different heuristics. I tried to tune the heuristics for the hp-mark use case. See how that went, here.

Ultimately failing to tune the heuristics convinced me to put background plates below my markers. That is what I'm currently working on. They will probably end up looking something like this:

The final background design will probably be flat and circular, and hp-mark will use the edge of the background, as well as the marker itself, to determine its position, further improving both robustness and accuracy of the system.

But not today. I'm so tired of that kind of programming. Let's celebrate the restuls at the top of this post for a while first. And let's rest.

- tobben

Hangprinter Campaign: Bountysource Salt

Hangprinter Project: [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], [40], [41], [42], [43], [44], [45], [46], [47], [48], [49], [50], [51], [52], [53], [54], [55], [56], [57], [58], [59], [60], [61], [62], [63], [64], [65], [66], [67], [68], [69], [70]

Hangprinter Project Homepage: hangprinter.org

Print Issue Solution Filter Project: [1], [2], [3], [4]

Sourcing RepRappro Mendel in Larvik: [1], [2], [3], [4], [5], [6], [7]

Archive: 2014, 2015, 2016, 2017, 2018, 2020, 2021