Flutter, an ideal framework for Algorithmic Art?

Flutter is a cross platform framework for Apps. On most of its target platforms it uses a low level renderer called Skia to draw user interfaces, rather than wrapping the native platforms UI primitives. This means that you can go very deep with just a single language/context. This also seems to allow for much easier porting to a new platform, as more of the code is portable. Indeed Flutter seems to be in a much stronger position on MacOS and Linux compared to React Native. The Web is also a target. But more on all of this later.

It uses a language called Dart, which can run in two modes. The first is an interpreted mode for hot stateful reloading, so there is a great developer experience, you make a change and see it almost immediately without having to rebuild your App. If you are used to web development this may not seem impressive, but native mobile apps can often take (literally) minutes to compile. The second mode is a compiled one for production (so at runtime the code is 'native'). Dart is very easy to learn, especially with the excellent tooling that Dart and Flutter ship.

A lot of these features are of massive importance to creative coding. Being able to quickly make changes and see the effect is essential for iterating on artwork. Being able to deploy to a desktop platform for an exhibit, mobile apps for engagement and the web for promotion would be ideal. Performance is something we don't want to hold us back.

There are some gaps, to fill in, but Flutter offers a remarkably good basis for creating 2D, interactive or procedural work. It is easy, friendly, fast, easy to change and can be delivered to many platforms and contexts.

Out of the box experience for Algorithmic Art

In Flutter to do fully custom drawing we subclass CustomPainter and implement a paint method:

  @override
  void paint(Canvas canvas, Size size) {

  }

This Canvas has the broad range of standard drawing functionalities you might expect for example drawImage, drawRect and drawVertices.

Canvas APIs make a lot of use of Paint to control the way drawing happens. Here we can control things like fill and stroke, blend mode, antialiasing and more. We can also set a shader which allows for gradients or image drawing.

We include this in an App with a CustomPaint widget (in Flutter we call components Widgets). The size is controlled either by a child widget (useful for when it is dynamic), or can be specified explicitly.

Capturing Images

One interesting feature that is quite straightforward to add is capturing parts of your UI as images. For any kind of Algorithmic Art application this is great, as we'd often want to provide some way to export creations. It requires a bit of setup with a GlobalKey to identify the thing you are interested in, but the key parts look like:

  static const platform = MethodChannel("your channel name");
  GlobalKey globalKey = GlobalKey();

  Future<void> _capturePng() async {
    final RenderRepaintBoundary boundary = globalKey.currentContext!.findRenderObject()! as RenderRepaintBoundary;
    final ui.Image image = await boundary.toImage(pixelRatio: 4.0);
    final ByteData? byteData = await image.toByteData(format: ui.ImageByteFormat.png);
    final Uint8List pngBytes = byteData!.buffer.asUint8List();
    await platform.invokeMethod("savePng", pngBytes);
  }

// And in your build method (the part that creates the UI)
  @override
  Widget build(BuildContext context) {
    return RepaintBoundary(
      key: globalKey,
      child: // stuff to snapshot
    );

Notice how the pixelRatio allows us to control the quality. This is assuming that a MethodChannel has been set up to allow for the platform to save the image (i.e. a platform specific implementation), though you could likely avoid this by using a package to do it for you. So we can easily do fully custom drawing and (relatively) easily export these drawings.

Standard and fancier (mobile) UI

We can do various other things such as simple animations with TweenAnimationBuilder or we can do more interactive experiences based on taps or gestures for mobile devices.

Here is a very simple example of dropdown menu. Unlike many creative coding frameworks this (and a lot more) is very easy to create from built in components.

And here is a simple example of adding sliders to control parameters.

We also benefit from Flutter's massive reach in terms of platforms, good performance and hot stateful reload, which allows us to quickly iterate. In some ways this is already ahead of things like Processing/P5.js. Though there are some gaps to fill.

Filling in the gaps and tips on using Dart

How can improve the AX (Artist Experience)? Flutter gives us a lot of great low level tools for drawing. Indeed out of the box we are not too far off the experience of something like Processing. We also have a bunch of great higher level components for UI such as buttons, form fields, layouts and way, way more, as this is the main intention of Flutter.

I tackled the same problem for the 2D Canvas Web API with Solandra so in running into this challenge I decided rather than just trying to solve for a single project I should create a library. Hmmmm. At least having created a similar thing before I felt like I had a very good understanding of the goals. And Dart (plus Flutter) actually has a standard library and a bunch of built in features for things like 2D Maths, so it should be a lot easier this time?

For the first version I wanted to tackle the following:

• Better Colour: Flutter only(?) supports RGB(A), this is terrible for generative uses; it is okay it you are just copying something a designer sent.
• Canvas aware iteration: something from Solandra, basically I should be able to iterate over the Canvas in intuitive ways without having to write a bunch of repetitive boilerplate.
• A higher level shape/Path class. Flutter supports a basic Path but I can't do anything other than describe something to be drawn with it.
• Pseudo Randomness: I'd want a simple, consistently seeded set of methods for common distributions.

I also wanted to make things concise where possible and always allow for dropping down to the lower Canvas/Paint level.

Colour

I created a helper to create a Color from hue, saturation, lightness and (typically optionally exposed) alpha.

Color fromHSLA(double h, double s, double l, double a)

This seems to me to be the clearly best of the commonly used ways of constructing colours. As is fairly commonly the case the hue goes from 0 to 360 (as in degrees), the other values from 0 to 100. This seems to me like a good set of choices that are quite easy to intuitively think about.

This is typically exposed via APIs like (In Dart you can optional positional parameters in square brackets like this, or named parameters which are optional by default):

background(double h, double s, double l, [double a = 100]) {

Solandra has helpers like setFillColor that allow you to set a fill colour to be used when filling Paths. (This is just updating a Paint which you can use directly when required for more specific things.)

Iteration

In the original Solandra I added a bunch of methods to iterate over a Canvas. This can make it much easier to start drawing something and allow you to focus on your work. So you can easily create something like:

Here is the Flutter version of forTiling (there are a bunch of similar methods available):

  void forTiling(
      {required int n,
      bool square = true,
      double margin = 0,
      bool columnFirst = true,
      required Function(Area area) callback}) {

In the original TypeScript version I used a callback with 4 arguments. In TypeScript function parameters can be substituted for something more general, a function with fewer arguments, because of how arguments work in Javascript. In Dart this isn't possible, so I added a Area class that gives the useful information for these kinds of functions to work with (notice the this. shorthand that means we don't have to write out so much constructor boilerplate).

class Area {
  Point<double> origin;
  Size delta;
  Point<double> center;
  int index;

  Area(this.origin, this.delta, this.center, this.index);
}

So in action this might look like:

final sol = Solandra(canvas, size);
sol.clipped();
sol.background(40, 20, 90);
sol.forTiling(
    n: 32,
    margin: 0.05,
    columnFirst: columnFirst,
    callback: (area) {
        sol.setFillColor(area.index.toDouble(), 70, 40);
        sol.fill(SPath.rect(at: area.origin, size: area.delta));
    });

Even this simple example hopefully shows how a bunch of things like adding a margin or tiling over a canvas become very simple. You can also see how it borrows some ideas from Processing (having a fill colour, set via setFillColor and ease of the background fill).

Shapes

In Flutter we can easily draw a specific shape. But what if we want to work with a shape by smoothing it out, exploding it into fragments and so on. For this I created SPath. Let's just jump straight into an example. What if I wanted to take a 20 sided regular polygon, split it into triangular segments, explode those segments into triangular shards a couple of times; now take all those shapes and colour based on index. Before I show you 'the' answer, first consider what that would be like to implement from scratch. Imagine how fiddly, difficult and error prone it would be.

In Solandra this is just:

SPath.regularPolygon(radius: size.minimum * 0.4, at: sol.center, n: 20)
  .segmented
  .expand((p) => p.exploded(scale: 0.75, magnitude: 1.1))
  .expand((p) => p.exploded(scale: 0.75, magnitude: 1.3))
  .forEachIndexed((i, p) {
    sol.setFillColor(i * 5, 80, 70);
    sol.fill(p);
  });

This is both concise and agile (easy to change details or even structure of). (expand is the Dart version of flatMap i.e. it takes a List, a function from item of List to new List and concatenates the results.) One other nice operation you can perform on an SPath is the Chaikin algorithm for smoothing out shapes: this cuts each corner to nicely smooth things out. Just a few iterations will produce something that looks nicely curved.

Pseudo Randomness

Solandra supports a bunch of standard pseudo random operations. randomPoint is interesting as it takes into account the canvas size.

Point<double> randomPoint()
double randomAngle()
double random()
int randomInt(int max)
bool randomBool()
double gaussian({double mean = 0, double sd = 1})
int poisson(int lambda)
T sample<T>(List<T> items)
List<T> samples<T>(List<T> items, int count)
doProportion(double proportion, Function() callback)
proportionately(List<Case> cases)
Point<double> perturb(
    {required Point<double> at, required double magnitude})
List<T> shuffle<T>(List<T> items)

The use of seed is consistent and I think this covers most common distributions. It also includes helpers for randomised control flow such as doProportion (do a thing this proportion of times) and proportionately when there are a set of cases to pick from randomly.

Helpers

I added some basic canvas helpers:

  clipped()
  double get aspectRatio
  double get width
  double get height
  Point<double> get center

As well as bunch of extensions on Point which make things easier or do common mathematical things.

Point<double> rotate(T angle) =>
      Point(x * cos(angle) - y * sin(angle), x * sin(angle) + y * cos(angle));

Point<double> get normalised {
final m = magnitude;
return Point(x / m, y / m);
}

Point<double> pointTowards(Point<double> otherPoint,
    {double proportion = 0.5}) {
return this + (otherPoint - this) * proportion;
}

double dot(Point<double> other) => x * other.x + y * other.y;

Offset get offset => Offset(x, y);

Solandra Flutter

An early version is already available and quite usable: Solandra (Flutter). You can also see, thanks to Flutter's support for Web a live demo. I did find it remarkable how a few demos I only ran on MacOS, could be very easily deployed to a website, without any changes. The initial blank screen/load time is a little unfortunate; I haven't yet looked at how to improve it. It appears to cache however, so subsequent loads are fast.

Dart/Flutter Tips

• Dart is kind of Java flavoured Javascript
• But has some cool innovative features: a cascade operator .. allows you to chain many calls on one object and every class is an interface (so no need for that common OOP thing of writing out both an interface and class all the time)
• You need semicolons at the end of lines
• The type goes in front of the variable or function
• But often can be inferred and you can use var for variable, final for something you won't change and const for something that could be created at compile time.
• forEachIndexed exists but you must import 'package:collection/collection.dart';
• expanded is the Dart version of flatMap
• Dart/Flutter come with lots of built in things for e.g. Point
• (2D) Point is generic, but I think for many creative code uses it is easiest to use Point<double> everywhere.
• Try to use standard things like the existing Size class
• Return types should be given, void if it doesn't return something
• Custom drawing is not clipped by default, Solandra includes clipped() for this.
• See also my article on Flutter in (late) 2021

Conclusions

Flutter seems remarkably well suited as a platform for creative coding. I can't think of an alternative that delivers on all of:

• Wide cross platform targets, with high quality
• Performance is good for fully custom, fully animated 2D drawing
• Hot reloading + Declarative UI

If you are used to Processing or P5.js you should give it a try for your next project. If you are completely new to creative coding it would likely be better to use something with more examples. While Dart is easier and better than Java, you will struggle to find examples of creative coding.