CoreML Programming with Xamarin.Mac and F#


Although Xamarin technologies are most commonly used to develop mobile applications, Xamarin.Mac makes it easy to use your preferred .NET language to develop desktop Mac apps. There are many C# examples in Xamarin’s mac-samples directory on Github, but it’s also easy to use F# to develop desktop Mac apps as well.

Currently, there is no support for Storyboards when working with F# and Xamarin.Mac, so this leaves you with two options: develop your UI in code or manually add the links between the Storyboard XML and your F# code. Both are straightforward, but in this example we’ll build the UI in code.

The following code shows a simple custom NSView that the app uses to set the values of the input parameters:

The class is typical of those that bridge the worlds of F# and the Mac. The AllowNullLiteralAttribute allows null references to the type, while CLIEventAttribute and OutletAttribute are used to map a native message to a .NET event. In this case, the assignments to spinner.Target and spinner.Action specify that the system should call the method associated with the Selector “SpinnerValueSelected” when the spinner’s value changes. That association is created with the OutletAttribute applied to this.SpinnerValueSelected. The code retrieves the text value, converts it to a floating point value, and triggers the Updated event.

Programming for the Mac will be very familiar to those who are accustomed to programming for iOS: you have an entry point that creates an NSApplicationDelegate which has one or more NSViewController objects that, in turn, have some number of NSView objects. However, it isn’t quite as simple as swapping NS for UI as the prefix for your objects! MacOS has a longer history than iOS and there are more legacy approaches and quirks than there are in iOS. For instance, NSViewobjects are not necessarily backed by CoreAnimation layers, and instead of NSViewController objects, you may work with NSWindowController objects.

Using CoreML with F#

The app has three InputView objects that specify the number of solar panels, greenhouses, and acreage of a property for sale on Mars. The app uses a (very simple) CoreML model file to estimate the price of such properties. CoreML is a framework introduced in MacOS High Sierra and iOS 11 for inferencing (making calculations) with machine learning models trained using other tools. At this point, you can neither train nor access the internals of a CoreML model.

To use a machine learning model with CoreML, the first step is to convert the model from the native training library to a CoreML .mlmodel file. This is done using Apple’s CoreML Tools or an open-source extension. The second step is to “compile” the .mlmodel using xcrun coremlcompiler compile model.mlmodel outputfolder. For a detailed walkthrough of training and converting an ML model to CoreML and Tensorflow Android Inference, see my MSDN Magazine article.

A compiled CoreML model results in a folder, e.g., MarsHabitatPricer.mlmodel that contains binary definitions of the model structure and weights. You may load that model from the Web (and that would be fine for a small model such as this), but you generally load it as a resource with code, such as:

You’d want to return an Option or a Maybe in a robust app, but, for learning purposes, we’ll just throw an Exception if there’s a problem.

CoreML apparently has its own internal datatypes for representing values, so instead of directly passing the input values, you need to wrap them in an IMLFeatureProvider instance that provides a mapping from strings to values. In this case, we have three features: the number of solar panels, the number of greenhouses, and the acreage. Those values are stored in mutable variables and, when any one of them is modified, we trigger an InputsChanged event:

As you probably figured out, we subscribe an instance of this MarsHabitatPricerInput type to each of the InputViewobjects’ Updated events, so changing the input values leads to triggering the MarsHabitatPricerInput.InputsChangedevent. That, in turn, leads us to:

We pass the MarsHabitatPricerInput object to the MLModel object’s GetPrediction method. Again, we raise an exception on an error and otherwise retrieve the model’s “price” output, which we know to be a Double. The details of the model’s input and output strings and datatypes come from the training-and-conversion process.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>