This is continuing from my previous post on moving Acorn to Metal.

Over the past week I've made some pretty good strides on having Acorn use Metal 2 and the new Core Image APIs introduced in 10.13 High Sierra. I've reworked how the pixels draw from the canvas NSView subclass to the display by introducing a little shim view which rests above the canvas and does the actual rendering. (Previously the canvas was an NSOpenGLView subclass). So if you're running Acorn on MacOS 10.12 or earlier you will get the previous OpenGL rendering, and if you're on 10.13 or later you'll get the super fast Metal 2 + IOSurface rendering.

And wow is it buttery smooth. Opening up a 100 megapixel image and then panning, brushing, and zooming around is super fun and has made this work 100% worth it. I've also found a couple of other optimizations that will also help folks out on previous OS releases as well, so there are wins to go all around.

Of course, this is all just testing on my dev machines for the time being. Who knows what's going to happen when it gets out onto everyone's machines- GPUs are notorious for being flaky. But with that in mind I've also added a preference to switch back to the OpenGL renderer, as well as a software renderer (and because of the architecture I came up with, was only about a dozen lines of code to support).

All my regression tests are now run against the both renderers as well, and I found a couple of instances where drawing deviated from CPU to GPU, but it wasn't anything unexpected.

So what's left to do?

So far, I've not been able to get deep color to display correctly when using Metal and IOSurface. I can set the pixel format to MTLPixelFormatBGR10A2Unorm, and the first few frames render correctly, but then things quickly go south from there with the introduction of funky florescent colors and black boxes showing up. I've got more digging to do, but I think it might actually be issues with IOSurface and not Metal. That's just more of a hunch at this point though.

The other show stopping bug I'm running into is drawing through MTKView isn't happening as frequently as I'd like when I seem to be taking up a lot of CPU power. I have a feeling this has to do with my ignorance on Core Animation and some sort of transaction system I'm not familiar with.

This problem mainly shows up when using the magic wand tool. Using the magic wand you can click on the canvas and drag out to change the tolerance for the pixels you'd like to be selected. For each mouse event I get I recalculate the tolerance, perform the operation on the pixels, and then tell the view to update the changed region.

If I've got the renderer setup to use OpenGL or software, then I get a 1:1 mapping of the tool calling setNeedsDisplayInRect: to update a region, and then drawRect: being called to actually do the new drawing.

If I'm using Metal, then I can call setNeedsDisplayInRect: 20 times before drawRect: is actually ever called. And it only seems to happen when I stop moving the mouse and the load on the CPUs go down. This makes it look like things are going slower then they actually are, or that the tool isn't even working.

I'm also seeing something which might be related to this when using the Window ▸ Zoom Window menu item. When Zoom is called and I'm using Metal, drawRect: is never called during the animation an instead my image is squished or expanded, depending on which way the window is moving. That's no good.

But everything else is good. Really good in fact.